Wednesday, October 13, 2010

Network-Transparent Interfaces

I'm thinking of a system designed by a bunch of smart people for use over networks. It is intended to allow for a separation between clients and servers such that applications are written for and hosted on a server and can be accessed from a remote (or local) client. It allows for complex graphics, sophisticated interaction, and it is all (roughly) standardized by a public, non-profit group. The servers host sophisticated software and each user's data. The clients can communicate with and access the data and programs from anywhere in the world, regardless of the machine the client is actually running on, provided the client software follows the standards. Can you guess what I'm describing? That's right! It's the X Window System.

Oh, you guessed the World Wide Web? Huh. How'd you get that idea?

(I intentionally reversed which was client vs server in reference to X, but my point stands)


...

So I recently began noticing some disturbing similarities between X11 and the WWW. Unfortunately, this came about by thinking about the failings of each within a relatively short time span. The more I thought about it, the more I realized that the Web seems to be trending towards the same failure that X hit a long time ago. People seem to fall into a mode of thought where thin clients are all the rage and will solve all the world's problems. It was far more appropriate in the days when X was designed, since the majority of computers were dumb terminals attached to some mainframe and/or were too weak to do anything interesting. It's really not true today.

Some parts will always be true. Big servers will always be more powerful than desktop machines (barring some complete revolution in the way computers work, but we can't plan for that anyways, so I'm ignoring that possibility), and there is some merit to centralized data storage accessible anywhere. The advent and increasing ubiquity of MIDs (Mobile Internet Devices) and smartphones plays towards the "clients that are too weak to do anything interesting" angle, but they get continually stronger. History shows that thin clients have their own slew of difficult problems, and I'm not convinced the current trends in the progression of the Web are truly addressing that.

I could (and probably will, at some point) go into what I think the Web is doing wrong, but right now I want to consider first what both X and the Web are trying to address: network-transparent graphical programs.

The intent, as I see it, is to create a standard system to allow for network-transparent graphical programs in a heterogeneous environment. The division of labor between the two ends of the network connection changes based on the application, though as a general rule the communication between the two should be kept to the minimum required. I don't know that a client-server model is a good way of describing such a scenario, since both ends of the connection have elements of each. As such, I will refer to the user-facing end as the front end, and the computer-only side as the back end. In the case of the Web, the browser would be the front end, and the web server would be the back end; in the case of X, the X Server is the front and the individual applications are the back.

The front end primarily provides the tools and abstractions necessary to allow for the creation of sophisticated user interaction and graphics. This is the more sophisticated and difficult bit of engineering, as the tools it provides need to be general enough to allow for as many options as possible to developers while still providing a sophisticated tool set to make the simple cases easy. The Web largely succeeds at the latter, but fails at the former, largely due to incomplete standards and the lack of a reference implementation. X accomplishes the reverse. Anyone who has attempted to write for X will attest to its needless difficulty.

On the most basic level, any graphics system needs to provide means to draw to a screen or set of screens (or something else, but the most common case is a monitor/television) and the means for interaction with that screen. In the traditional case, the interaction comes from a keyboard and mouse. That was a safe assumption twenty years ago, but the advent and popularity of smartphones and other touch-oriented devices prove it to be a fallacy today (and that's ignoring the more creative input devices these days). Even the assumption of a single screen isn't safe, many people run multiple monitors and devices such as Wacom tablets attached to their PCs. In many cases, those screens are hooked together into a larger virtual screen, but that may not always be ideal. A modern front end would need to allow for all of that and make the system general enough to be as future-resistant as possible, since future-proof is largely impossible.

In order to generalize output as much as possible, the system needs to allow for arbitrary shapes and sizes of output screens, and make the system as extensible as possible to allow for things such as braille displays or holograms (if and when those are available) or any other interesting outputs that people can come up with. It's true that graphics won't always apply to this, but I would argue that a good system for user interaction should support as many modes of interaction as possible (don't even get me started on the state of sound in the existing systems).

Generalization applies just as much to the input system as the output.  Some inputs need to be closely tied to the outputs, in such cases as touch screens.  Others can and should be divorced from the graphics system, such as joysticks or control pads.  Mice need to have a screen to make any sense (unless someone thinks of something really interesting to do with them), but keyboards are just as useful outside of a graphical environment as in one.


This is in-depth enough to warrant its own ramblings (or series thereof), so I'm going to stop with the front end for now.  Human-computer interaction is a maddening problem, and I can see why all attempts so far have been less than perfect.  This ties together all levels of computation, and I'm beginning to think that any monolithic system is doomed to failure.  Unix philosophy strikes again.  This is definitely worth more thought.

The back end itself is largely uninteresting for the purpose of this discussion.  Though there are a lot of really interesting problems with efficient data storage and algorithms, it's really more general computer science than interface considerations in particular.

The other really interesting (and somewhat maddening) problem is the communication between the two.  In keeping with my earlier thought, I'm going to go out on a limb and say that there is no "one true method" for communicating between the back and front ends.  If it is across a network, then the set of necessary optimization criteria are completely different from those with communication within a local machine.  The only real commonality is conservation of bandwidth, but the specific solution to that may very well differ between implementations.  The only real statement I'm prepared to make on this is that the difference between the two should be completely invisible to any developers of apps for the system (and really, as much of the system itself as possible, look at KIO for inspiration).

Well, now that I've driven myself slightly (more) insane, I think I'm going to end this.  I have a lot of food for thought, and little more to actually say at the moment.  I started out with plenty I wanted to go over, such as competitors to X such as Quartz and NeWS, and ideas to pull from those systems, but over the course of writing this I've gone in a very different direction.  More to ponder, really.

Until next time, whenever that may be.