Interaction Design Section   Living Section   About Bruce Tognazzini

While the battle lines of pro- and anti-Aqua have been drawn in blood, no one seems to be looking at the real problem, and that is that Apple, in OS-X, is doing nothing other than running a 10-year-old and 20-year-old interface together at high speed.

Apple has one advantage shared with Microsoft—control of the interface—and one advantage that is unique—control of the hardware. Apple seems unwilling or unable to take advantage of either.

The next big movement in interface technology, just as with the last, will result from a marriage of hardware and software changes. The current interface resulted from two important hardware innovations, the bit-mapped display and the mouse. Both were achieved by 1965, with the first successful commercial GUI arriving some 19 years later.

In the ensuing years, little has happened to the displays, except they’ve gotten bigger (and, in the case of palmtops, smaller). Lots has happened to input devices, but you wouldn’t know it looking at a Macintosh. The only changes to input technology that have taken place recently around Apple are the crippling of the keyboard and the farcical round mouse that has made Apple the laughingstock of the industry.

Ironically, the only real advance in input technology has come from the software-only company, Microsoft, with their introduction of the wheel mouse. (We had a similar mouse prototyped at Apple more than ten years ago. Didn’t seem worth marketing it, though. After all, the standard one-button mouse is perfect, isn’t it?)

Where could Apple be next year if it were truly interested in developing a next-generation interface? Let’s take a look.

Keyboard

Easy wins. Anyone who uses the mouse has faced the frustration of having to repeatedly press the Return key while flipping from dialog to dialog. Why isn’t there a Return key on the left side of the keyboard, so we wouldn’t have to keep dropping the mouse? Why aren’t there, likewise, an extra Del and Delete key? These could all occupy a new column on the left edge of the keyboard.

Gesture

The clutter of words, icons, and buttons that obscure our screens today are the result of the severely limited vocabulary of the mouse. The only word it knows is "click," so you have to find an instance of the word you want to convey to the computer, then say "click" while you hover over it.

Consider how many words you would have to paste up in your office to be able to carry out a conversation using this method. And consider how slow and frustrating that conversation would be.

Gesture is the obvious next step in input. Companies, such as Palm Computing, with far weaker hardware than Apple have been using it now for years. Why is it missing from the Macintosh, and why is it missing from OS-X? Apple brags of having the hardware advantage; it’s time they used it.

OS-X should ship with a tablet or horizontal flat panel that will enable gestural input. The mouse is dead; it’s time for a change.

Integrated Phone/Voice Recognition

By having integrated telephony, not only can users eliminate one source of clutter on their physical desktop—the phone—they will suddenly start wearing a lightweight headset. That headset is the entry into vocal interaction. Where gesture leaves off, spoken commands can begin. Is voice-recognition up to the task? It is barely up to it for continuous speech recognition, but most of what we need doesn’t require either continuous speech or a rich vocabulary. Most of what we need to do involves issuing one of a few dozen commands in any one context. For example, in the context of Photoshop, the user might speak anything on a menu or recite the name of a tool. Several dozen words define the entire vocabulary.

Imagine the flow that would occur were you able to just work, freed from the constant task of changing or modifying the action of tools.

Reader Erik Neu sees an additional synergy:

In my observation, the average (or even the better-than-average) office worker does not master even the more basic features of their telephone (like conference calling and transferring) because of the clunky, non-intuitive and non-standard interface. It would be SOOOO much better and easier to invoke these features from a PC interface.

Screen Objects

Apple is feeling pressure to eliminate the spatial finder not because someone repealed people’s natural affinity for spatial interfaces, but because the 1984 Macintosh model has run out of steam. Why? Because, in those days of 128K machines with a single floppy and no hard disk, no one thought to provide complex objects. Instead, people got documents, applications, and folders. Can you imagine an office that only had documents and folders? Where would you sit? Where would you store things? It is little wonder that people can become lost.

What are needed are a series of new, more powerful objects.

Piles

Apple holds a patent on this one. Developed by Gitta Salomon and her team close to a decade ago, a pile is a loose grouping of documents. Its visual representation is an overlay of all the documents within the pile, one on top of the other, rotated to varying degrees. In other words, a pile on the desktop looked just like a pile on your real desktop.

To view the documents within the pile, you clicked on the top of the pile and drew the mouse up the screen. As you did so, one document after another would appear as a thumbnail next to the pile. When you found the one you were looking for, you would release the mouse and the current document would open.

Piles, unlike today’s folders, gave you a lot of hints as to their contents. You could judge the number of documents in the pile by its height. You could judge its composition very rapidly by pulling through it.

Folders

Apple has one new kind of folder in OS-X, the Package, which, in essence, enables an application to contain within it all the little tiny files necessary for it to run.

However, regular files appear to continue to be as weak and ineffective as ever. I should be able to tell, by looking at the outside of a file, how much material is within, how old the folder is, and how long it has been since I’ve opened it. I’m not talking about surrounding it with text; I’m talking about using that 24-bit color space to add visual attributes. For example, the more the folder contains, the thicker it should appear. The older it is, the deeper the color should become, with age cracks appearing after several years. The time since I’ve opened it could be represented by cobwebs or dustiness.

File Cabinets

Many of the larger projects I work on could take good advantage of a file cabinet, a sort of project-level structure. Different drawers might hold email vs. graphic designs vs. specifications vs. prototypes. Or I might divide it by the people working on it, with a drawer for each. By displaying labels on the drawers when there are few or tool-tips when there are many, people could access the sets of folders within. It would knock out one layer of hierarchy by bringing the second layer to the surface.

Scrapbooks

I proposed scrapbooks in Tog on Software Design as a handy place to organize the materials for projects, etc. Building on the concept of piles, scrapbooks offer the ability to casually organize incoming material and to provide notes and other context-enhancements.

As outlined in Tog on Software Design, a scrapbook can be:

A scrapbook can contain:

Notebooks

Notebooks would be a step up from a scrapbook, perhaps occurring as a project nears fruition.

People

People should also be first-class objects in a modern interface. I should be able to drag a document onto an icon-sized photo of a colleague and have it automatically sent to the default address. (Of course, I could option drag a document to it to reveal all addresses, or I could just double click it to do the same.)

Active Documents

Microsoft has been inching closer and closer to compound documents, and Apple itself took a brief run at it several years ago, but it is time for the real thing. The large-corporation monopoly will not be broken in the computer industry until we have the kind of cooperative development in the end-user space that we are seeing in the Linux system software space. This calls for a document platform on which different developers can supply various engines for text editing, for graphics editing, for spell checking, for "spreadsheeting" (to coin a phrase), etc., etc., etc.

Projects

Up until now, I've been discussing objects that might appear on the desktop. The desktop itself, however, has shrivelled a bit. We need a new, larger desktop.

Sun has done this by introducing a "virtual desktop" that is several times bigger than the portion visible on the screen, along with a little map viewer that lets you see where you are and drag to where you want to go. In my experience, users find this troubling.

Quicken has what I believe to be a much more successful model, with a series of master tabs, always visible across the top, pulling up one environment or another. The user easily moves from the checking/savings account environment to the investment environment to the report environment. Quicken was also clever enough to enable the user to open any account in any environment, so if you need to compare stock purchases to checks written for same, you can open your checking account in the investment environment, etc.

I would love to bring up the AskTog environment every month and find all my windows and tools just lying there waiting for me.

Conclusion

This is not an exhaustive list of objects and capabilities that should be incorporated in the interface. These are just a few of the more obvious.

OS-X may work out and it may not, but let’s not confuse it with an advance in user interface. At best, it will catch up with itself, should they decide not to cripple the Finder.

The move to a true advance in user interface is long overdue and, with Microsoft’s success with the thumb mouse, we just might soon be seeing the software-only company leading the way.

Don't miss the next action-packed column! Receive a brief notice when new columns are posted by sending a blank email to asktoglist-subscribe@yahoogroups.com. return to top

Contact Us:  Bruce Tognazzini   Copyright Bruce Tognazzini.  All Rights Reserved