User Interface Management Systems (UIMSs) have now gained acceptance in the research and business communities. Unfortunately, they are still not widely available or used. This paper proposes three reasons for this: that they are too hard to use, that they are still too limited in the types of interfaces that they can create, and that they are not portable with respect to different machines, operating systems and graphics systems. UIMSs can be made easier to use by avoiding programming-language-like techniques, and they can be made more functional by providing the ability to create Direct Manipulation interaction techniques and to integrate them into modern interfaces. UIMSs can be made more portable by inventing new input models and graphics packages that are appropriate for highly-interactive interfaces and that can work in various environments.

User Interface Management Systems (UIMSs) [17,19,24] have gained general acceptance in the research and business communities. This can be seen by the large number of papers on UIMSs that appear in the SIGGRAPH and SIGCHI conferences and the rising number of UIMS products. Unfortunately, most user interfaces are still created by hand without the use of appropriate tools. Why is this? This paper proposes three unsolved problems with current UIMSs which, I believe, strongly contribute to their lack of popularity. The first is that current UIMSs are very hard to use and often require the user interface (UI) designer to learn a special purpose programming-like language. The second problem is that UIMSs typically are fairly limited in the types of interfaces that they can create. The third problem is that most UIMSs are not portable.

One avowed goal of UIMSs is to allow professional UI designers who are not programmers to create user interfaces. Most UIMSs, however, use a textual specification with a rigid syntax that is much like a programming language. This problem might be addressed by using techniques such as those found in easy-to-use Direct Manipulation [21] interfaces in new UIMSs.

Unfortunately, it is hardest to make UIMSs that can create the modern, Direct Manipulation style of interfaces that are popular for most new systems. For example, more work needs to be done so that UIMSs can be used to create the interaction techniques used in Direct Manipulation interfaces. In addition, it must be made easier for the user interface designer to control multiple interaction techniques that are available and operating at the same time.

Another important problem is that UIMSs typically only work on one particular machine with one particular operating system and with one type of graphics package. This is typically because there is no standard graphics package that provides the necessary primitives needed by UIMSs and because existing device independent graphics packages are too slow for use in Direct Manipulation interfaces.

There does not seem to be any general agreement about what should be called a User Interface Management System, beyond a general feeling that it is something that helps build user interfaces. The model for UIMSs presented by Tanner and Buxton [23] shown in Figure 1 is an important start and shows where various parts fit into the general picture. For example, the graphics supplied with most window systems and the Macintosh Toolbox [1] can be classified as the module library by itself. Is this enough to qualify as a UIMS? Does it matter?

UIMSs can also be classified on a different axis by how they allow the designer to specify the interfaces (see Figure 2). This is closely tied to how easy the system is to use since non-programmer UI designers will have to learn programming techniques (if not actual programming languages) for the systems towards the left of the axis.

Some systems provide a library and require the designer to program in a conventional language (as in typical window managers). This would usually not be classified as a UIMS. However, some other systems provide some sort of framework that directs and helps the designer in this programming. These systems are often classified as UIMSs even though conventional, general-purpose programming languages are used. Examples of this are object-oriented systems like GWUIMS [22] and Apple's MacApp [20] (which creates Macintosh programs). These provide standard objects for handling user interfaces which the designer modifies to create a particular user interface. The next class of UIMSs invents a language of its own that the designer must learn in order to create the user interface. This language may be very abstract such as BNFs [16], or it may be more ``direct'' and declarative as in COUSIN [8]. The next class has tools that allow the designer to present a graphical description of the desired interface. These range from very program-like (e.g. Transition Diagrams [11]) to very demonstrational (e.g. Peridot [13] which uses Direct Manipulation techniques to create Direct Manipulation interfaces). The assumption is that systems to the right of the axis will be easiest for UI designers to use.

A problem with most available UIMSs is that they do not address Direct Manipulation (DM) interfaces. These interfaces are characterized by: (a) a general lack of syntax, (b) multiple interaction techniques that are available at the same time (e.g. a number of menus and the user can choose which one to use), (c) a reliance on novel and rapid use of non-keyboard input devices (e.g. a mouse), (d) semantic feedback inside inner loops of interaction techniques, and (e) elaborate graphics.

UIMSs which support Direct Manipulation interfaces must deal with the following issues, some of which are elaborated on in this section:

Some UIMSs support DM interfaces by providing a hand-coded library of interaction techniques and just allowing the designer to ``glue'' them together (see Figure 1). Some of these require the designer to use a textual language to specify where the interaction techniques are positioned on the screen—COUSIN [8], and Apollo's Dialogue—whereas others, such as Menulay [4], Trillium [9] and GRINS [18], allow this to be done graphically.

Another approach is taken by the Peridot UIMS [13]. Peridot allows the designer to create the interaction techniques themselves using examples to demonstrate what the screen picture should look like and what the end user should do to perform the interaction (see Figure 3). The system attempts to infer what the general case is that covers the specific examples using some simple rule-based Artificial Intelligence techniques. Peridot was inspired by work in the areas of Visual Programming and Programming by Example [12]. Examples of other interesting recent systems in these areas include Editing-by-Example [15], Graphical Thinglab [3], Rehearsal World [6], and SmallStar [7].

Work is also needed to provide appropriate mechanisms to handle multiple interaction techniques that are available at the same time (so the user can choose which one is desired), and also multiple interactions that are operating at the same time (such as different devices under each hand). Having multiple interactions available is a cornerstone of mode-free interfaces, and using multiple devices in parallel has been shown to increase users' efficiency and satisfaction with systems in some experiments [5]. This concurrency problem is compounded by the fact that the application program must be structured so that it can deal with the parallelism. This might require research into appropriate operating system structures for the kind of parallelism needed by user interfaces. Hill's Sassafras system [10], which uses a production-like system, is one experiment in this area, and another is Peridot, which uses ``active values'' [13] which are like variables except that dependent graphics are updated immediately when the associated values change.

A related issue that is not usually handled by UIMSs is to support semantic feedback [14], which is when the application is involved in feedback during mouse movement and inside other ``inner loops'' of interactions. For example, an object being dragged might jump to legal points (this is often called ``gravity'' and an advanced form is ``Snap-Dragging'' [2]). These types of feedback are sufficiently complicated that the UIMS cannot provide them locally without access to the application's data structures. This is closely related to issues of semantic error checking and semantic default values which are not usually handled by existing UIMSs. Overall, this can be classified as the desire for fine-grain control whereas most UIMSs only allow coarse-grain control. Constraints as used in GRINS [18] and the active values of Peridot [13] will help alleviate this problem, but an important efficiency question remains: can the UIMS provide for semantic feedback that is fast enough to keep up with the mouse and other input devices?

Almost every UIMS that exists today is tightly tied to a particular operating system, computer, and graphics package. This is because there are no appropriate high level interfaces that can shield the UIMS from having to directly manage the various input devices such as the mouse and keyboard and from directly creating pictures on the screen.

To make UIMSs be more portable, we must deal with the following issues, some of which are elaborated on in this section:

One important reason for the lack of portability is that none of the existing or proposed graphics standards (CORE, GKS, PHIGS, CGI/VDI, etc.) are appropriate for creating Direct Manipulation interfaces. Most are too slow to get an appropriate level of feedback, and too large to fit easily into many computers, but more importantly, the input model in all standards is essentially the same and is widely perceived as wrong for highly-interactive interfaces. Unfortunately, there seems to be no good alternative model, so current UIMSs must directly manage the mouse to provide Direct Manipulation interfaces. The UIMS community should be investigating new models for input devices so a graphics package can be created that provides a higher level of abstraction.

Obviously, an appropriate model for output graphics must also be provided, but this seems easier to generate. The most difficult problem here is providing a high enough level of abstraction along with sufficient speed, but the appropriate general principles seem to exist.

Along with the input and output models, a standard interface to a window manager will probably be needed for portability. The current effort at creating a Window Manager standard (X3H3.6) will probably help with this, but it will also need new input and output models. In addition, it would benefit from the advice of the UIMS community since they will probably be affecting the user interfaces that can be created.

User Interface Management Systems have arrived. There are a fairly large number of them available and a number of products have been created using them. The UIMS community now has three important goals: to make UIMSs easier to use by designers, to increase the functionality of UIMSs so that they can create the most popular Direct Manipulation styles of interfaces, and to allow them to be more widely available. To achieve the first two goals, more work needs to be done on graphical specification techniques and parallelism. The third goal requires that more work be done on making UIMSs portable which includes defining appropriate input models for highly interactive interfaces. These efforts will hopefully allow User Interface Management Systems to be as widespread and essential for creating user interfaces as parser-generators are for creating compilers.

I want to thank Xerox Canada, Inc. for the donation of the Xerox workstations and Interlisp environment which supported the work on Peridot. This research was also partially funded by the National Science and Engineering Research Council (NSERC) of Canada. I would also like to thank SIGGRAPH for funding my attendance at the UIMS conference. For help and support with this paper, I would like to especially thank my advisor, Bill Buxton, and also Bernita Myers, Peter Rowley, Ralph Hill, and Ron Baecker.