11Hill.tioga
Rick Beach, February 15, 1987 8:00:26 pm PST
Some Important Features and Issues
in User Interface Management Systems
Ralph D. Hill
Dynamic Graphics Project
Computer Systems Research Institute
University of Toronto
Toronto, Ontario, Canada
M5S 1A4
ralph@toronto.EDU
(416) 978 6986
This work was financially assisted by the Natural Sciences and Engineering Research Council of Canada and the University of Toronto. We also gratefully acknowledge a grant from Xerox Canada, Inc. which provided the equipment that made the implementation of Sassafras possible.
Abstract
Some key features that are believed to be required in future User Interface Management Systems (UIMSs) are presented and discussed, as are some key philosophical issues. These features and issues concern the range and quality of user interfaces that can be supported, and the ease with which the user interfaces can be implemented and modified.
The arguments presented are based on experience gained with the Sassafras UIMS. Sassafras is a prototype UIMS that was explicitly designed to support some features that were not common in UIMSs, but were believed to be necessary to support many styles of interaction.
Overview
In the past few years many User Interface Management Systems (UIMSs) have been proposed (see [Thomas & Hamlin 1983] and [Pfaff 1985] for general information on UIMSs). Many are effective tools for rapidly prototyping user interfaces to interactive systems. The question is no longer whether such tools are practical, but what features are needed to maximize their usability, and what are the key issues in their future development.
Based on experience gained with Sassafras (a prototype of a powerful UIMS intended to support a wide range of interaction styles [Hill 1987a, 1987b]), a list of desirable UIMS features, and some issues in the design of UIMSs has been assembled. The features are:
· Concurrency
Support for concurrency in the user interface, primarily to support user interfaces that allow multiple input devices to be used simultaneously.
· Communication
Support for communication among the modules that make up the user interface at run-time.
· Suspended Time Editing
Support for an editor that allows execution of a user interface to be suspended, its specification altered, and then execution resumed from the point of suspension, running the new specification.
The issues that will be discussed are:
· Matching the UI to the Application
It is often suggested that a UIMS can be used to put a variety of user interfaces onto an existing application. This can be done only if the UI specification and the application routines are compatible.
· Separation of User Interface from Application
A fundamental assumption of a great deal of UIMS work is that the UI can be separated from the application. Separation is often useful, but it can limit user interface quality. The goal is to separate when possible, without limiting quality.
· Parser vs. Programming Language
Should the portion of the user interface dealing with the syntactic level specification be more like a parser than a programming language? Most existing UIMSs are biased toward the parser approach, but this is not always best.
· Non-Programming UI Implementor
Should the user of a UIMS be a non-programming UI implementor, as it is often suggested is possible? Or, should the user have some limited programming skills? Experience with Sassafras suggests that the latter is more appropriate for some interface styles.
The next section will briefly discuss Sassafras and some of the interfaces that have been implemented with Sassafras. Then the features and issues listed above will be discussed with reference to this experience. Finally, some concluding remarks will be made.
Experience with Sassafras
Sassafras [Hill 1987a, 1987b] is a prototype UIMS that has been implemented as a test bed for a new technique for specifying the syntax of dialogues, and a new run-time structure for UIMSs. The syntax specification technique is based on a list of rules, where each rule specifies an action to be taken when the system is in a given state and the user supplies a specific unit of input. The run-time structure allows an interactive system to be decomposed into modules that effectively execute in parallel, and communicate and synchronize via asynchronous message passing.
The key features that Sassafras is intended to demonstrate are: support for concurrency within the implementation of the user interface, and support for elaborate communication among the run-time components of an interactive system. Concurrency is supported both in the syntactic level specification language, and the new run-time structure. The new run-time structure also supports the run-time communication.
Sassafras has been used to implement several user interfaces, including the user interface to the run-time portion of Sassafras. For the purposes of the following discussion, the three most significant interfaces are: (these are demonstrated in [Hill 1986].)
· Sassafras Control Panel
The Sassafras control panel was originally coded in a traditional programming language, and was re-implemented with Sassafras, once Sassafras became stable. It provides experience in retrofitting a UIMS based user interface to an existing application. The key problems with this are: the application routines had to be modified to present a uniform interface to the user interface, and had to be extended to support many features in the new UI.
· Two-Handed Paint
This is a paint program that allows the user to control the position of the (electronic) paint brush by using the mouse with one hand, while controlling the colour of paint and the thickness of the brush with a touch-sensitive tablet operated with the other handed. This provides an obvious example of a case where a user interface that allows multiple dialogues to proceed in parallel may be superior to a traditional interface.
· Room Layout
This is really a sequence of four interfaces. The first is a close approximation to the furniture positioning in a room program described in Chapter 2 of ``Fundamentals of Interactive Computer Graphics'' by Foley and van Dam [1982]. The subsequent three interfaces are modifications of the first. The original interface is moded, with a hierarchy of four menus. The fourth interface is modeless with real-time feedback and a direct manipulation [Shneiderman 1983] style of interaction. In order to test the theory that UIMSs can be used to retrofit user interfaces to existing applications, the application routines were developed after looking at the pictures that show the rough behaviour of the Foley and van Dam interface, but before reading the details. Thus, they were designed with general knowledge of the task, but without specific knowledge of the user interface.
It was found that the application routines had to be extensively modified in order to support the Foley and van Dam interface. The application routines implemented a specific model of interaction with the user interface (based on the notion of current selection, and supporting real-time feedback), that was incompatible with the basic model of interaction supported by the Foley and van Dam interface. This suggests that it may not always be possible to retrofit a specific UI to an existing application, as the application and the UI may not be well matched.
Some desirable UIMS features
Concurrency in the user interface
Concurrency is necessary in the implementation of multi-threaded dialogues. These arise in user interfaces where the user is free to manipulate multiple input devices to address several (possibly related) tasks at the same time. For example, in a CAD application, the user may be allowed to drag a part into position (say, with a mouse manipulated with one hand), while selecting the size, rotation, colour or some other parameter(s) of the part with some other device manipulated with the other hand. Similarly, in a paint program, a user may paint with one hand, while controlling paint attributes with the other. Buxton and Myers [1986] have demonstrated that allowing users to performs two tasks simultaneously may speed up their performance and thus this feature should be supported in UIMSs.
Concurrency can also be a useful tool for structuring the implementation of the UI. In many direct manipulation interfaces the user can perform a wide variety of actions, depending on which item the user has selected to manipulate. If these interfaces are implemented as a single dialogue, a complex structure can result. By implementing them as several dialogues that are able to run in parallel, where the item selected determines which dialogues (if any) are running, a more modular, easily understood implementation results.
Concurrency can be used to improve the structure of user interfaces in another way. If a UIMS that supports concurrent dialogues is used, the UI can be implemented as a translator that carries on one dialogue with the user while carrying on another with the application routines. This can result in a more easily understood implementation, and can improve the responsiveness of the interface by overlapping interactions with the user and the application routines. The control panel interface was implemented in this way, and it gains responsiveness and has a clearer structure as a result of this approach.
For all of the above reasons, concurrency is an important feature that should be supported in UIMSs. Sassafras demonstrates that it is practical to do this.
Communication
Most existing UIMSs have a run-time structure where the UI is allowed to call the application routines, or the application routines call the UI. Neither approach allows a good balance between the two components. The problem is that if the only form of communication is procedure call, the communication will tend to be one-sided as the callee can only send information by returning, and must return to the calling routine. A balanced approach is needed as the UI will often have to communicate commands and arguments to the application routines, and the application routines will have to communicate information to the UI such as range limits and defaults that change as the state of the application routines change. If communication in either direction is sacrificed, then interface quality is compromised.
This two-way communication between the UI and the application is used in both the control panel and layout interfaces. In the control panel interface, commands are sent to the application routines in response to user input, and the application routines supply information that is used in the pop-up menus that the UI presents to the user. Tanner and Buxton [1985] describe picking as a potentially difficult problem in UIMSs. In the room layout interfaces, two-way communication is used to support picking (selection of objects that have already been placed by pointing and then clicking the mouse button). In the case of the layout programs the user interface tells the application routines where the user has attempted to pick, and the application replies with the object that the user picked (or error information). Thus the solution is simple if appropriate communication facilities are provided.
Two-way communication is also needed between the user interface and the routines or modules that are used to collect input from the user (the Interaction Modules or IMs). These are the standard packages that support keyboard input, pop-up menus, and other standard forms of input. These packages are effectively subroutine libraries. There must be some way to send parameters to them (to customize their behaviour), and to receive input from them.
In the paint program described above, the IMs report mouse activity and interaction with the touch tablet. The UI tells the interaction modules that deal with the touch tablet and the mouse what information they should report. If the UI could not tell the IMs how to behave, the UIs would have been much more difficult to construct.
Suspended time editing
Suspended time editing is the ability to suspend the execution of a user interface, alter its specification, and then resume execution from the point of suspension, running the new specification. This feature allows a user interface designer to rapidly install changes in a user interface and then immediately test the changes in the context that inspired them. Assuming a UIMS is to be used to support a style of interface development where a prototype is constructed and then iteratively tested and improved, this feature is very useful. It makes the loop very fast, and eliminates the tendency to procrastinate on the installation of some changes because the effort is not believed to be worth the improvement in interface quality.
During the development of the interfaces described above, Sassafras was modified to support suspended time editing. This feature was found to be extremely useful in the refinement of user interfaces.
Issues in the design of UIMSs
Matching the UI to the application
It is often suggested that UIMSs can be used to retrofit user interfaces to existing applications with little or no change to the applications routines. This is likely true, if the new UI can be tailored to the existing application. However, if the goal is to add a new UI of a specific style, then the application routines may have to be extensively modified. For example, the original application routines for the first version of the room layout program had a model of behaviour that was incompatible with the specification of the Foley and van Dam interface. The application routines had to be extensively modified before the UI could be completed.
The style of interaction supported by the user interface is not independent of the functionality of the application routines. The UI cannot support functionality at the user interface that is not supported in some way at the interface between the UI and the application routines. There are two potential problems:
1) It may be difficult to add a specific new UI to an existing application if the new UI is to implement a specific style of interaction that is incompatible with the existing application routines.
2) The application routines may not support all of the functionality that is need to support the user level commands.
The only solution to the first problem is a major rewrite of the application routines, or a change to the UI specification. Both of these are undesirable. In the case of new applications, there must be communication between the UI and application designers during development, to ensure that their respective designs are not incompatible.
The second problem is not as serious. If the UI can be easily modified, then it is natural to expect that useful commands will be discovered and quickly implemented at the UI level. If a powerful programming environment is used to implement the application routines, it should not be difficult for the application implementor to keep up with extensions to the user interface.
Separation of user interface and application
Much UIMS research, and many systems, are predicated on the assumption that the user interface can be separated from the application and the two developed separately [Olsen & Dempsey 1983, Tanner & Buxton 1985]. The meaning of ``separation'' is not clear. It could be logical separation, in the sense that the user interface and the application routines do not share data, and have only a limited ability to communicate with each other. Alternatively, the separation could be physical, with the specification of the user interface being in a different file (or files) than that of the application routines. The two forms of separation are independent, hence there are four possible combinations. The best combination is physical separation without logical separation. This allows the most appropriate language to be used for each component, and allows the various components of the interactive system to communicate as needed. For example, the room layout interface relies heavily on physical separation to help modularize the specification and to control complexity. It also uses a tight logical coupling to improve feedback when the user is moving pieces of furniture. Traditional run-time structures have prohibited the use of a combination of physical separation and tight logical coupling.
Parser vs. programming language
Many UIMSs are built around some form of parser and are predicated on the assumption that a key function of the UIMSs is the parsing of user input. Thus, the language used to specify the user interface is a language for formally specifying languages and does not have most of the features of traditional programming languages. For many simple interfaces (e.g., the control panel interface) this is sufficient.
For more complex interfaces, such as the paint and advance room layout interfaces, the parser based approach becomes clumsy and awkward. The problem is that simple computations must be frequently performed on user input or system state. This is particularly true of direct manipulation style interfaces. For example, the two-handed paint program must modify the size of the brush, performing additions to it and keeping it in the range of legal sizes. If the user interface specification language does not support this type of operation application routines must be used. This greatly increases the size and complexity of the specification and reduces its legibility. A solution is to extend the user interface specification language so it becomes a simple programming language with special control constructs to assist in dialogue specification.
Non-programming UI implementor
Throughout the UIMS literature (e.g. [Buxton, et al. 1983] and [Feldman & Rogers 1982] and the advertising for most commercial UIMSs), it is commonly claimed that UIMSs should allow non-programming user interface experts to design and implement user interfaces. When dealing with visual aspects of the UI, such as icons and screen layout, programming skills are not needed. But for some parts of the UI development a limited-programming user interface expert would be better than a non-programmer. The above issues are arguments in favour of requiring the user of a UIMS to have some programming skills. Also, when testing and debugging it is often hard to know if the problem is in the application routines or the user interface, hence, some programming knowledge would be of use in locating errors or when consulting with the application programmer(s).
Thus, the goal should not be to eliminate programming, but to provide the UI-designer with an appropriate set of tools. These tools could require the use of some programming skills. In the development of the example interfaces, it was found that simple arithmetic operators, conditional branch and function call were the only constructs needed. It is likely that non-programmers could quickly learn to deal with these constructs (the more complex and error prone concepts of iteration and recursion are not needed). Thus it reasonable to ask that the user interface expert have some basic knowledge of programming.
There are, however, some UIMSs that require no programming (e.g., Trillium [Henderson 1986], the University of Toronto UIMS [Buxton, et al. 1983], and Peridot [Myers & Buxton 1986]). The cost of being free of programming is a loss of generality. This cost may be acceptable in many cases.
Summary and conclusions
Three desirable features that should be supported by UIMSs have been described. These are: concurrency, communication and suspended time editing. These can improve the quality of user interfaces developed with a UIMS and can speed this development. The Sassafras UIMS demonstrates that these features can be efficiently supported.
Four key issues in the design and use of UIMS have also be presented. These concern: matching of the UI to the application, separation of the UI from the application, whether the dialogue specification language is a parser or a programming language and whether the UI implementor should be a non-programmer. These issues are derived from assumptions or beliefs about the way in which UIMSs can be used. Experience with Sassafras suggests that these assumptions and beliefs are not always true. This suggests that the design and implementation of future UIMSs should not rely on these assumptions and beliefs.
Acknowledgements
The design of Sassafras, and the choice of features and issues presented in this paper owes a lot to many discussions with Bill Buxton, and to [Tanner & Buxton 1985]. This presentation has been improved as a result of discussions with Brad Myers, Peter Rowley and Alison Lee. Special thanks to SIGGRAPH for financing my trip to the Seattle workshop on Software Tools for User Interface Development.
References
[1] Buxton, W., Lamb, M.R., Sherman, D. and Smith, K.C. Towards a Comprehensive User Interface Management System, Proceedings of SIGGRAPH'83 (Detroit, Mich., July 2529, 1983). In Computer Graphics 17, 3 (July 1983), 3542.
[2] Buxton, W. and Myers B.A. A study in two-handed input. In Proceedings CHI'86 Conference on Human Factors in Computing Systems (Boston, April 1317, 1986), ACM, New York, 321-326.
[3] Feldman, M. and Rogers, G. Toward the Design and Development of Style-Independent Interactive Systems. Proceedings of Human Factors in Computer Systems, (Gaithersburg, Maryland, March 1517, 1982), ACM, New York, 111116.
[4] Foley, J.D. and van Dam, A. Fundamentals of Interactive Computer Graphics, Addison-Wesley, 1982.
[5] Henderson, D.A. Jr. The Trillium User Interface Design Environment. In Proceedings CHI'86 Human Factors in Computing Systems (Boston, April 1317, 1986), ACM, New York, 221227.
[6] Hill, R.D. Sassafras Demonstration Video Tape [videotape]. (1986), Dynamic Graphics Project, CSRI, University of Toronto, Toronto, Ontario, Canada, M5S 1A4.
[7] Hill, R.D. Supporting Concurrency, Communication and Synchronization in Human-Computer Interaction. PhD thesis, University of Toronto. To appear January 1987.
[8] Hill, R.D. Supporting Concurrency, Communication and Synchronization in Human-Computer Interaction — the Sassafras UIMS. To appear in ACM Transactions on Graphics, 1987
[9] Myers, B.A. and Buxton, W. Creating Highly Interactive and Graphical User Interfaces by Demonstration, Proceedings of SIGGRAPH'86 (Dallas, Texas, August 1822, 1986). In Computer Graphics 20, 4 (August 1986), 249258.
[10] Olsen, D.R. Jr. and Dempsey, E. SYNGRAPH: A Graphical User Interface Generator, Proceedings of SIGGRAPH'83 (Detroit, Mich., July 2529, 1983). In Computer Graphics 17, 3 (July 1983), 4350.
[11] Pfaff, G.E., ed. User Interface Management Systems, (Proceedings of the IFIP/EG Workshop on User Interface Management Systems, Seeheim, Federal Republic of Germany, October 1983), Springer-Verlag, 1985.
[12] Shneiderman, B. Direct manipulation: a step beyond programming languages, IEEE Computer 16, 8 (1983), 5769.
[13] Tanner, P.P. and Buxton, W.A.S. Some Issues in Future User Interface Management System (UIMS) Development. In User Interface Management Systems, Pfaff G.R. (ed.), Springer-Verlag, Berlin, 1985, 6779.
[14] Thomas, J.J. and Hamlin, G. Graphical input interaction technique (GIIT) workshop summary, Computer Graphics 17, 1 (January 1983), 530.