Directions for User Interface Management Systems Research Mark Green Department of Computing Science University of Alberta Edmonton, Alberta, Canada 1. Introduction Over the past five years there has been considerable growth in the area of User Interface Management Systems (UIMS). This interest has resulted in a number of experimental UIMS, and a few commercial products. In this paper three possible directions for user interface management system research are proposed. The two main purposes of this paper are to stimulate research in UIMSs, and to provoke discussion on which directions this research should take. As a result, in the discussion of the three research areas the emphasis is on the open research problems and not how they might be solved. 2. High level descriptions Existing UIMSs are based on detailed descriptions of the user interface to be constructed. These descriptions cover screen layout, the dialogue between the user and the computer, and the interface between the user interface and the other parts of the application program. These descriptions tend to be at a fairly detailed level. Quite often the user interface designer must specify the positions of menus, and the order of the menu items. The dialogue between the user and the resulting user interface must be specified in detail, usually in the form of a grammar or transition network. In many ways dialogue specification has not changed significantly since the work of Newman in 1968 [10]. Existing UIMSs can significantly increase the productivity of user interface designers. But, there are still a number of problems with them. Further increases in designer productivity will only be the result of radical changes in UIMSs. Most of the existing UIMSs have basically the same functionality, and they provide this functionality in basically the same way. Therefore, moving from one UIMS to another will not drastically effect the productivity of the designer. The only way to increase the designer's productivity is to provide a new type of UIMS. Most existing UIMSs tend to be hard to learn. This can be inferred from the size of the manuals for commercial products, such as the Apple Macintosh or SUN Workstation. The time required for the first Macintosh applications to appear in the marketplace is another indication of the time and effort required to learn these systems. In summary, we need a new approach to UIMSs which will produce UIMSs that are easier to learn and use. One solution to the above problems is to use a higher level representation of the user interface to be constructed. What form should this representation take? We can approach this question by considering the information that is required in order to construct a user interface. One piece of information we need is a description of the problem the user is trying to solve. This is a high level description covering the tasks the user must perform in order to complete his job. Note that this description does not cover how the user would perform these tasks in a particular user interface. This part of the description is called a task model, and several notations have been developed for it. A formal notation for task models is presented in [7]. Another important piece of information that is required for the construction of a user interface is a description of the functionality of the application routines. These routines are called by the user interface in order to perform application specific computations. Any of the formal software specification languages could be used for describing this part of the program. The final piece of information we need in order to construct a user interface is the properties of the potential users. These properties include the manual dexterity of the users, typing abilities, level of computer expertise, preferred interaction modes, and any other factors that might effect their interaction with the resulting user interface. Currently, once these three pieces of information are collected the user interface designer starts designing the user interface. He or she usually starts with a paper design, and then moves to a UIMS to produce the detailed design and implementation of the user interface. My first proposal is that we start developing a new generation of UIMSs that directly produce the user interface given the task model, application functionality, and user properties. That is, given a very high level description of what the user interface should do, the UIMS will automatically design and implement the user interface. This type of system will definitely increase the productivity of the user interface designer. In order to further illustrate this concept consider the following scenario. The user interface designer constructs a task model for the problem the user is faced with. This task model includes the objects the users manipulate in the task domain, and the operators they use to manipulate them. The objects are described by listing their properties, and the operators are defined in terms of pre- and post-conditions. Similarly the routines in the application are defined in terms of pre- and post-conditions. A description of the data structures maintained by the application is also required. Based on the properties of the user community the UIMS must perform the following operations. First, the visual representation of each of the objects in the task model must be produced. This visual representation could be selected from a catalogue of display techniques based on the information to be displayed and the properties of the user. Next, each of the operators in the task model must be mapped into a sequence of application routines that implement the operator. This can be done by finding a sequence of applications routines that have the same pre- and post-conditions as the task model operator. This is basically a planning problem. Finally, the arguments to the task model operators are used to generate the interaction techniques in the user interface. For each argument there must be an interaction technique that can be used to enter its value. This interaction technique can be selected from a catalogue based on the type of input value and user properties. The automatic design of user interfaces is beyond the state of current technology. Advances must be made in several areas before a UIMS of this nature can be produced. Such a UIMS will need to use automatic programming techniques [2,3]. Automatic programming techniques have not developed to the point where they can produce a program as large as a user interface. On the other hand, user interfaces form a well defined domain which is ideal for some of the automatic programming techniques. Some of the programming by example systems that have been developed for user interfaces form a start in this direction [9,12]. Another area where further research is required is in codifying our knowledge of user interface design. This codification is particularly important at the presentation level. This codification would contain rules for laying out menus and screens, selection of input techniques, selection of output techniques, and the best ways of designing dialogues for particular classes of users. This type of research could lead to the development of expert systems for user interface design. There could be a number of problems with this type of UIMS. First, it is not clear that such a UIMS could be constructed. Second, there is the possibility that this type of UIMS could produce user interfaces that are not very usable. Even if the UIMS is 99% perfect, it will still produce user interfaces that are not acceptable. As a result, the user interface designer may need some way of modifying the user interfaces that are produced by the UIMS. Third, the user interfaces produced by the UIMS may be highly stylized. It may be able to produce an acceptable user interface, but the UIMS may ignore the best user interface for an application. 3. Interface to other parts of the program In a large application the user interface is only part of the application program. The application designer may need to use a number of other tools in the production of the program. Some examples of these tools are database management packages, communications packages, and distributed computing packages. Each of these tools will have its own special notations for describing its part of the application, and the application programmer will need to master all of these notations. In some cases the application programmer may need to describe the same aspect of the program in several notations. One area where there could be considerable overlap is database and user interface. The database management system is responsible for the information manipulated by the program. The application programmer must produce a schema for the database that describes how the data is (logically) organized and how the programmer can access it. On the other hand, the user interface is responsible for transferring information between the user and the computer system. In most cases this information will be stored in the database, or be commands that modify the database. Thus, the application programmer is forced to describe the same (or similar) information in both the database schema and the description of the user interface. It would seem natural to describe this information once, and store it in one place (to guarantee its consistency). The user interface must communicate in some way with the rest of the application program. The database schema already describes the data maintained by the program and how it is accessed. The schema could be augmented to describe the routines called by the user interface. In a large number of applications the user is mainly interested in manipulating a database, therefore, it seems natural to describe these routines in the database schema. The UIMS can then use this information to construct the interface between the user interface and the other parts of the program. My second proposal is that a combined notation for user interfaces and database schemas be developed. This will allow the application designer to produce the high level design of the database and the user interface at the same time. This will save the designer some time and effort, and will decrease the number of notations that he must learn. Most database systems provide an interactive query facility, which is a form of user interface. Thus, incorporating part of the user interface into the database system is not a radically new idea. Some work has already been done in the user interface area on combining user interfaces and databases, for example [11] and [1]. 4. User interface evaluation While existing UIMS assist in the implementation of user interfaces, they provide very little aid in the design of user interfaces. In many ways the design of user interfaces is harder (and more critical) than their implementation. One area where the UIMS could help in the design process is with the evaluation of the design. This evaluation could occur either before or after the user interface is implemented (or both). My third proposal is that UIMSs should be extended to cover the evaluation of designs. The implementation of a user interface can be instrumented to produce a protocol while the user is interacting with it. This protocol contains a time stamped record of each interaction between the user and the user interface. Some UIMSs already produce this type of information, for example [5] and [8]. Producing a protocol is not very difficult, but doing something meaningful with it is a major research problem. The protocol file usually contains a large amount of raw data, and it is very difficult to analyze this data by hand. Buxton has developed a program that allows the user interface designer to play back the protocol file in order to observe the user's actions in a controlled way. Tools that could analyze the protocol in order to determine the time required to perform commands, and the number and types of errors would be very useful. The UIMS has available a description of the dialogue, this information might be useful in the analysis of the protocol. The evaluation of a user interface before it is implemented is more difficult. Techniques have been developed for evaluating user interfaces given formal descriptions of them. Some of these techniques are reported in [4] [6], and [7]. Since UIMSs require a formal description of the user interface, these techniques could be incorporated into them. The user interface designer could then ask the UIMS to evaluate his design and possibly suggest ways to improve it. These evaluation techniques could also be part of the automatic user interface design system described in section 2. In this context they could provide a way of optimizing the designs produced by these systems. An area related to user interface evaluation is the use of style guides. Sometimes a collection of programs must have very similar user interfaces. In this case a style guide is developed to describe how the user interfaces must look and behave. If this style guide is written in a formal notation the UIMS could check for and possibly enforce the style. For example, each time the user interface designer modified the design, the UIMS would check to ensure that the new design agrees with style guide. If it doesn't, the designer will be informed and his modifications might be rejected. 5. Conclusions In this paper three proposals for research in UIMSs have been presented. These proposals are: 1) High level systems for automatic user interface design. 2) Integrated notations for databases and user interfaces. 3) The incorporation of design evaluation into UIMSs. The first of these proposed research areas is by far the hardest, and it is not clear that it will produce usable systems within the near future. As a result, it can be viewed as a long term goal for UIMSs. The other two research areas could produce usable results in the near future. In both of these areas there has been pioneering work that could lead to major advances. Some of the ideas put forward in this paper are hard to defend, and I don't intend to defend them (I might not even agree with them). One of the main purposes of this paper is to provoke discussion on the directions UIMS research should take, and stimulate thought on the future form of UIMSs. If some readers (violently) disagree with what I have said, then this paper has served its purpose. Acknowledgements I would like to thank Gurminder Singh for his comments on some of the ideas put forward in this paper. None of the bad ideas are his fault. I would also like to thank the Natural Science and Engineering Research Council for financial support. References [1] Armstrong W.W., M. Green and P. Srirangaptna. A Database Management System and Associated Tools for a General Design Environment. Proceedings of the 1985 Canadian Conference on VLSI, (1984), 183187. [2] Barstow D.R. Knowledge-Based Program Construction, North-Holland, New York, 1979. [3] Biermann A.W., G. Guiho and Y. Kodratoff (ed.), Automatic Program Construction Techniques, Macmillan, New York, 1984. [4] Bleser T. A Formal Specification and Computer-Assisted Evaluation of Person-Computer Interfaces, MSc thesis, George Washington University, 1983. [5] Buxton, W., Lamb, M.R., Sherman, D. and Smith, K.C. Towards a Comprehensive User Interface Management System, Proceedings of SIGGRAPH'83 (Detroit, Mich., July 2529, 1983). In Computer Graphics 17, 3 (July 1983), 3542. [6] Card S.K., T.P. Moran and A. Newell, The Psychology of Human-Computer Interaction, Lawrence Erlbaum Associates, Hillsdale NJ, 1983. [7] Green M. The Design of Graphical User Interfaces, Technical Report CSRI-170, Computer Systems Research Institute, University of Toronto, 1985. [8] Green, M. The University of Alberta user interface management system, Proceedings of SIGGRAPH'85 (San Francisco, Calif., July 2226, 1985). In Computer Graphics 19, 3 (July 1985), 205213. [9] Myers, B.A. and Buxton, W. Creating Highly Interactive and Graphical User Interfaces by Demonstration, Proceedings of SIGGRAPH'86 (Dallas, Texas, August 1822, 1986). In Computer Graphics 20, 4 (August 1986), 249258. [10] Newman W.M. A System for Interactive Graphical Programming, Proceedings 1968 SJCC, (1968), 4754. [11] Olsen D.R. An Editing Model for Generating Graphical User Interfaces, Graphics Interface'86 Proceedings, (1986), 6670. [12] Singh G. Programming by Example for Graphical User Interfaces, Proceedings 1986 CIPS Edmonton Section Annual Conference, (October 1986). B10Green.tioga Rick Beach, February 13, 1987 8:57:45 pm PST ��"computergraphics" style��MarkcenterHeader� J�centerFooter�,�Ititle�9Iauthors�Zhead��t� �Ibody��N��N��N��N��N��N��N��N��N��N��N��N��(�*N��N��N��N��N��N��N��N��^Iitem�:O�:O�5�N��N��N�� I reference��i3��P��$� �UP��4�)��yP��P��P��)�,�2��P��P��P��P��A��fP��K�"��|P��D�8��A`Ck