Rick Beach, February 15, 1987 7:54:47 pm PST
Tools and Methodology for
User Interface Development
1. Task force organization and objectives
The members of this task force were: John Bennett, Terry Bleser, Neil Corrigan, Roger Ehrich, Tom Hewett, Przemyslaw Prusinkiewicz, Hank Ramsey, Jim Rhyne, Kurt Schmucker, John Sibert and John Wiberg. We jointly prepared an annotated outline for this report during the workshop. The conversion of this outline to prose, after we had left Seattle, was done by Jim Rhyne, Roger Ehrich, John Bennett, Tom Hewett, John Sibert and Terry Bleser. In the process, we were again reminded of the inadequacy of present editing, networking and text formatting tools for such a task.
The objective of the task force was to characterize in a broad way the understanding of procedures for enduser interface design and development. Discussion often focused on tools to assist designers and implementers, paying particular attention to research topics of current and future interest. There was a consensus that enduser interface software proceeds through phases of requirements/solutions, design, implementation and testing common to other software and hardware developments.
The primary focus of research has been on implementation, and little progress has been made on methodology and tools for the other phases. We also lack integrated tools for use throughout all of the phases. Consequently, most of the group's effort went to requirements/solutions and design. The group members felt that the most important testing of an enduser interface was its scrutiny by the actual endusers and by human factors experts, and that this testing must occur at the earliest stages of the development, rather than at the end. Auditing of the final software is also essential, but should not be a problem when the design of the interface is cleanly specified.
Perhaps the most important software development bearing on the enduser interface has been the User Interface Management System (UIMS). Since Kasik published his research paper with this title , the pace of research into enduser interface development and tools has greatly accelerated. We begin with an assessment of the state of this development.
2. User interface management systems
For the past several years, research on User Interface Management Systems has focussed primarily on a number of difficult issues in the design of software tools for the implementation of sophisticated human computer interfaces. For example, the Seeheim workshop  examined the architecture of interface software under the assumption of modular separation of the application and its interface. Much effort has been directed to models and languages for dialogue specification [7,9,14,17], on object-oriented programming [23,24,25], and on other techniques for constructing interfaces [3,5,10,15,16,19]. These tools have been primarily tools for programmers. Little support has been provided for designers, human factors researchers or computer science researchers exploring interface technology.
There has been much effort to model the organization of UIMS. The Seeheim model shows how a software system is structured at run time and reflects how the design-time system must be structured to provide this run-time organization. The Seeheim model was based on a concept of dialogue independence in which dialogue and application were loosely coupled by a control component that 1) defined the relationships between dialogue and application and 2) routed tokens back and forth at run-time.
Both the Seeheim model and the concept of dialogue independence have been useful as frameworks. However, as more complicated interfaces supporting window managers, direct manipulation and semantic feedback have been investigated, problems have been revealed . Functional distinctions between dialogue and application have been unclear, and it has been found that shared data models are almost essential . Therefore new models are being sought that incorporate what has been learned about interface structures and that provide a better basis for software tools.
In addition to this growth in complexity of UIMS models and capabilities, builders of experimental UIMS are discovering that their systems are not at all easy to use [15,20]. The problem is partly one of complexity of the tool function, partly of integration and partly of poor enduser interface to the tool.
Both the construction of the tools in the first place and the use of the tools in practice seem to fit within the framework of a software engineering design methodology. This allows trade-offs affecting system qualities seen by the user at run-time to be managed during the construction process using engineering principles. Such a methodology resembles standard software engineering in its gross aspect but is quite different in detail. Interface requirements are rarely as well defined and stable as requirements for computational algorithms. The interface objects and events have a complicated relationship to the application objects and actions. Furthermore, the interfaces that result are subject to evaluation, criticism and redesign by endusers and human factors specialists who do not normally play a role in the design of computational algorithms. Nevertheless, the recognition of the importance of an engineering methodology for managing trade-offs in the design and implementation of human-computer interfaces in itself represents a significant shift of emphasis in UIMS research.
3. General methodology framework
The general framework has three components:
· Exploratory research. Studies in software and hardware technology for enduser interfaces, and in human factors of enduser interfaces,
· Requirements acquisition and conceptual design. The development of interface requirements and design approaches for particular situations,
· Design and development. The detailed design, implementation and debugging of interface software and hardware.
There is a dependency of each component on the ones preceding it for information, but the process typically requires multiple iterations through the components as issues arise . It is strongly recommended by the group members (and many others) that endusers and human factors experts be involved in all components.
A discussion of methodology for exploratory research would be out of place in this report. Such methodologies are routinely practiced in university and industrial research laboratories. However, cooperative efforts between psychologists and computer scientists are extremely valuable as much exploratory software is never adequately tested, and much experimental psychology is not made practical and accessible. Furthermore, the employment of exploratory interfaces by endusers reveals much unexpected information about the adequacy of the design and the ease of use and learning of the interface. This said, we turn to methodologies and tools for the other two components.
4. Methodology and tool integration
Ideally, the development process should flow smoothly through whatever components are needed to resolve issues. This is almost never achieved for a number of reasons, many of which are organizational and are not addressed here. However, one issue is relevant and extremely important, and that issue is the preservation and organization of information gathered during the development process.
Requirements, if documented at all, are stated in plain, unindexed prose. Those requirements which are minor and best understood often constitute the bulk of the prose. It is hardly surprising that designs are often not checked against requirements, and that the checking process is a laborious and error ridden task.
More thought must be given to compatibility and consistency of the methodological components. The data employed in each component must be analyzed, and ways found of recording relationships among requirements, design approaches and the rationale underlying design decisions. This will not only permit partial mechanization of the consistency checking task, but will be an invaluable education tool as the project progresses to the implementation stage.
5. Requirements acquisition and conceptual design
The function of requirements acquisition and conceptual design (RA/CD) is the accumulation of requirements for the enduser interface and an initial design approach for the interface. The purpose of this activity is to obtain agreement between the prospective users of the interface and the developers of the interface on the major issues of scope of effort and general approach, before starting on the design of the entire interface.
The task of requirements acquisition requires a representative sample of the endusers to think clearly, carefully and completely about their needs, and to communicate those needs to the developers. The developers have software, hardware and human factors skills, but they may not have the knowledge of the application domain or task procedures possessed by the endusers. On the other hand, endusers may be unaware of current technology and design principles. Their work habits may impede their ability to supply creative proposals regarding their needs.
The conceptual design task communicates technology and design information back to the endusers, who can judge whether it matches their needs. The participants will begin to see new requirements and new design approaches emerge during the RA/CD task, as each participant comes to understand the knowledge and ideas of the others. The benefits of early analysis of design can be considerable, and the likelihood of the endusers accepting the final implementation is substantially increased.
Conceptual design results from an analysis of the requirements and the generation of enough of a trial design representation that the enduser representatives can judge whether the requirements are understood. Thus, the conceptual design is a sketch that serves to test hypotheses about interface presentation and performance.
In order to achieve a coherent result, it may be necessary for the requirements analysis and conceptual design to be done by the same group of people. Because the result must be robust enough to support trade-offs during the design and implementation, this group of people must have earned the respect of the developers. It will be useful to augment the team with individuals skilled in understanding and characterizing enduser tasks.
6. General issues
6.1 Formative evaluation
Formative evaluation is a key methodological process by means of which the current conceptual view of the target system is subjected to criticism and analysis to determine needs, inconsistencies, weaknesses and other inadequacies. It also involves monitoring the process and products of system development and gathering enduser feedback for use in refinement and further system development. Summative evaluation is the counterpart process of examining enduser behavior after the target system has been designed and implemented. For a discussion of the differences between formative and summative evaluation, together with some illustrative examples, see .
At the requirements acquisition and conceptual design stage the primary concern is still focused on the determination of a consistent framework that accounts for the system requirements. As this framework evolves, additional questions will be raised about the goals and requirements of the target system until the point is finally reached where design can begin and the UIMS tools can take on an appropriate embodiment.
Members of development teams are becoming increasingly sophisticated in their understanding of issues in ``consistency.'' The everyday notion is becoming refined as considerations of consistent technology, appearance and enduser processes are included in trade-off design decisions. In order to allow for evolution of tool power with increasing innovation in technology and in interaction techniques, it is clear that ``identical'' interfaces are unduly restrictive. What is needed is a consistency that preserves those properties of the enduser interface that support transfer of already acquired factual and process knowledge (about the application, about the use of tools in general). Consistency within the interface enhances both learnability and usability by reducing the frequency of unexpected events that require additional enduser attention. People are willing to acquire new patterns, but only when the advantages provided clearly exceed the cost of learning.
A simple example can illustrate the role of consistency in system specification. Imagine a dialogue specification for move in the sequence: object selection, command selection, target selection. Further imagine a dialogue specification for copy with the sequence: command selection, target selection, object selection.
Functional requirements are met, but conformance with enduser expectations arising from the conceptual similarity of the two commands are not. Now, no team would make such an obvious mistake, but less obvious mistakes are common, even in well designed interfaces. There are several dimensions of consistency:
Interactions involving objects which play similar roles should be similar in structure so the enduser may generalize over roles.
Endusers should be able to perform similar manipulations on objects of the same type.
The same object should not appear in several representations that are visually unrelated.
Components of the interaction such as dragging and selection should always be performed in the same way.
Where a metaphor is employed (e.g. the physical office metaphor) the behavior of objects should be consistent with their real counterparts (e.g. files dragged to the filing cabinet or the wastebasket will disappear from view, while files dragged to the printer will reappear).
There are doubtless other kinds of consistency not mentioned here. The issues which must be addressed are the impact of each kind of consistency and the interactions among them .
Integration of interfaces is another notion that is frequently used but badly understood. It is useful to modularize function in applications, as this permits the endusers to compose functional modules in ways best suited to their task requirements. This is especially true when a system must support many endusers with diverse needs, or when task requirements are continually changing. Relying on endusers to do their own, high level application development may be the only way to avoid the omnipresent application development backlog. The interface should permit endusers to apply their general knowledge about interaction techniques to all modules. It should permit the enduser to move effortlessly from one module to another (flow of control), and to move data effortlessly from one module to another (flow of data). There is a great deal of current focus on rudimentary characteristics of integration, such as whether modules may execute concurrently, how they communicate and share data, whether multiple interfaces (windows) to the enduser are required. It seems clear that such characteristics will be possessed by future systems, and that our ability to build suitable enduser interfaces will be limited by our understanding of enduser problem solving behavior.
6.4 Performance baseline data
Part of understanding the function of the target system involves understanding the needs of the endusers (the designer of the interface, the person using an application through the interface). Given these needs (which might involve learnability, speed of interaction, error rates, expertise level, physical capabilities or environment), there are common performance baselines that have to be considered in the system formulation in order to lead to good designs later on. For example, if there are requirements for very rapid interaction with low error rates, one might begin to formulate ways to make use of direct manipulation concepts. Direct manipulation allows immediate feedback on many kinds of errors so that they are seen by the enduser and fixed immediately.
The major obstacle in this area is simple lack of information. Lots of claims are made, but surveys of the human factors literature reveals that very little is known about enduser preferences or the degree to which particular interaction technologies satisfy performance needs.
6.5 Choice of media
Another aspect of conceptual formulation of a system involves an analysis of alternative media for the system. Given the needs of the enduser, the system designer can make use of baseline data and technological expertise to select alternative physical media that may lead to viable solutions. For example, if the enduser is a worker in a noisy factory environment, many types of voice interfaces might be ruled out. However, there may be new sensor technology known to the designer that might permit limited vocabularies in a voice interface. This information then leads to thumbnail sketches of possible interface instances. As these sketches are constructed, additional information may be needed to determine whether certain technologies are indeed consistent with other system requirements.
6.6 Debugging the requirements document
Once a representation has been captured in text and data, experience shows that it will need to be revised. Finding bugs in concepts, correcting descriptions and discovering omissions are all sensitive to the fact that there are no accepted formats that communicate to both designers and endusers. Repeated iterations with the endusers will be needed to compensate for this lack.
7. Style decisions in enduser interface
There is often a tendency to overdesign at this stage. It is important to have frequent contact with the endusers, and to focus on high level design issues such as:
· Will the interface presentation be physical, iconic, textual or some combination of the three?
· Will direct manipulation, text entry, point and select, gesture or some other interaction style be used?
· What presentation media will be used — monochrome display, color display, voice, tone, special equipment . . . ?
· What user input devices will be used - keyboards, mice, styluses, voice recognition, . . . ?
· Will concurrent enduser actions be allowed?
· Will the interface be enduser configurable, e.g. will the user be allowed to replace one input device with another?
· Will the interface be enduser extensible?
We have assumed that active involvement by endusers is feasible, and this is not always the case. If, for example, the interface is being developed for a generic environment (e.g. an operating system) to support a wide range of applications and endusers, a representative sample of types of endusers may be involved. A similar situation occurs when a product is being developed for a new market, without established endusers.
Mockups of the interface are a great help in getting enduser feedback, but only if they can be done quickly. Presentations can be sketched if they are visual, or examples created if they are auditory. Example input devices may be used in informal trials by the endusers.
If extensible or configurable interfaces are planned, the ability of the endusers to make effective use of these functions must be assessed. Needs of handicapped workers must be considered, and explicit trade-offs made to support them.
The conceptual designers must avoid throwing new technologies into the interface without adequate rationale. Though these technologies may add flash to the design, they can often introduce problems in the development stages which result in late and costly interfaces. Too often, the excitement of working with a new technology will overshadow its rational justification.
8. Tool functions
There are few tools to support requirements acquisition and conceptual design of enduser interfaces. To some extent, this is a consequence of the variety of individual methodologies in use, but the greater cause has been that few researchers have attempted to build such tools. There are organizational obstacles to such projects in universities and large industrial laboratories, mainly the separation of researchers from development projects. For progress to be made, these obstacles must be overcome.
8.1 Requirements acquisition
Requirements acquisition has as its goal the creation of a model of the enduser and organizational requirements for the interface. Model forms include information flow, process representation, task assignment and enduser behavior. There are many techniques used for data collection, only a few of which are amenable to computer support (e.g. instrumentation of existing interfaces).
A few attempts to structure the requirements acquisition process around a model have had some success (e.g. the ISDOS project, Jackson Design). The fact that none of these attempts have gained widespread acceptance probably means that a variety of requirements acquisition techniques will be used in the future. Rather than focus on a semantic model for task analysis, it may be more fruitful to examine general, customizable data organization and retrieval tools. Particular problems that need to be addressed are high level query facilities, rule based tools for analysis, and tools for re-organizing the data when the initial organization proves incomplete (this will inevitably happen).
Interfaces are always developed in conjunction with the application. It is probably not possible to cleanly separate requirements for the interface from requirements for the application. This means that the interface task analysis tools must be identical to or fit nicely into the set of tools used for task analysis of the application function.
8.2 Conceptual design
Conceptual design is the generation and testing of hypotheses about interface presentation and behavior. These hypotheses can exist in two forms, one is the designers' technical specification of the design, and the other is a realization of the design for inspection by the endusers. The designers' technical specification may employ terms like direct manipulation, multiple dialogue threads or iconic presentation which will have little meaning to the endusers. The endusers can react best to example presentation and behavior. An interesting, open problem is a method for stating hypotheses and a tool for realizing these hypotheses as enduser interaction scenarios.
8.3 Enduser visualization
Storyboards are often used to help endusers visualize a set of design hypotheses. General drawing programs can be used to create the presentations for storyboards, but their end product is hardcopy and neither the descriptions behind the presentations, nor the storyboard scenarios are typically recorded with the design data. To this end, mixed media databases could be very helpful, along with tools for animation of the presentations. The animation specifications should be recorded in the database along with the presentations.
The storyboards will be revised, and there will be a need for tools and techniques to keep track of various revisions, tied to observations which prompted the revision. Finally, the tools must be efficient, as conceptual design is a fast paced activity, especially in a group environment.
A more dynamic visualization may be obtained through very high level prototyping, perhaps using a WYSIGYG interface builder such as Peridot , or Trillium . Such prototyping is only feasible on very high performance workstations because the flexibility of the tools trades against the use of lots of CPU cycles. We conjecture that design effectiveness can be increased with investment of capital. Present day examples include the growing popularity of LISP and SMALLTALK machines as prototyping vehicles for software development. Given the dramatic increases in hardware performance, it does not seem out of place to throw MIPS, Megabytes, Megapels and Megamessages at the design problem.
The lack of standard interfaces to input devices is a minor, but serious problem. The ubiquitous asynchronous communication interface lacks speed and places a decoding burden on the realtime software. Perhaps a standard I/O bus or a message based mini-LAN will emerge as the solution. The lack of standard interfaces to display hardware is being solved by standards such as CGI, and by the growing consensus on function in window managers.
8.4 User models
User performance models seem to occur in two flavors: loosely organized collections of facts about enduser behaviors, and formal models of enduser behavior. The success or failure of one of these types of user model depends on the skill with which it is used; examples of good and bad interfaces done with and without employing user models abound.
Loosely organized fact collections are stored in the minds of experienced designers, and result from experience and apprenticeship. A few attempts have been made to codify such knowledge, but by the time this is done, changes in the interface technology may render it obsolete. It would seem that the same kinds of video/audio/text data networks previously mentioned could be used to collect and index user behavior facts.
Formal models exist to predict behavior. Those models which have been thus far developed focus on a relatively low level of enduser behavior (GOMS, Riesner's BNF Grammars), or are constrained to predicting enduser strategy in uncomplicated tasks. There are research problems here of the greatest difficulty. Some researchers believe that Expert Systems tools may be used to build formal enduser models 
8.5 Integration of tools
The requirements and the capabilities created during the execution of this component must match. Integration of tools in a toolkit can be achieved when: all the tools can be used in the same environment, tools are modular, so that new tools can be added and old ones deleted and existing ones modified. In the case of interactive tools, of course, common presentation and interaction techniques should be used so that enduser habits transfer among the tools
Links between requirements and capabilities ought to be recorded so that the completeness of the design and the consistency of the requirements can be assessed. Such links should convey both positive and negative support (i.e. when a design feature works against a requirement). It is extremely important that requirement/capability links carry an explanation and rationale. That a particular set of capabilities meets a requirement may not be obvious at all, especially when the requirement is a general one, such as a specific limit on training required for an enduser to reach a certain level of proficiency with the interface. The linking of requirements and capabilities suggests that similar information models (knowledge bases, semantic models) be used. The development of information models for this purpose is an open research area, scarcely touched at present.
8.6 Support for group process
Most development is a group process. Software engineering has lately been concerned with group development facilities, though the focus has been on tools and procedures for information sharing. Such tools will also be useful for interface design, but interface design has stronger requirements for visualization and high level modeling. Special presentation facilities, such as multiple projection displays, will be useful. Example interaction devices should also be present to allow storyboards to be played out with some realism.
The function of design is the accumulation of specific decisions about the interface appearance, the functioning of the input devices, and the nature of dialogues, together with particular proposals for software to implement these decisions. It is crucial that the design be in harmony with the conceptual design, and meet the requirements set forth in the requirements acquisition and conceptual design phase. It is similarly crucial that there is an appropriate rationale for each decision made during the design phase. Finally, if the resulting interface is to be pleasing to the enduser, an aesthete must control the design as an artistic rather than a rational endeavor.
10. General issues
It is extremely hard to maintain design consistency with the requirements and the conceptual design. Organization factors, such as design being done by a different team, make it difficult to carry the vision and understanding across the boundary between these two phases. The question of whether a particular design decision is consistent with the requirements or the conceptual design phase often cannot be answered because it falls into an area left uncovered, or because the facts needed to decide the question are difficult to extract from the RA/CD documentation. Systematic approaches require compatible, formal representations of both the conceptual design and the design, and this introduces overhead and loss of flexibility in the design process.
Often, the enduser is brought into the design process in an attempt to circumvent these problems. The enduser is shown the proposed design, and asked to determine if this meets the requirements. This places all of the burden on the enduser.
Solutions to these problems involve both methodology and tools. Tools, in particular, help maintain the fine-grained consistency throughout an interface by enforcing the many details that have to be considered to produce a well-crafted implementation. The methodology must also incorporate procedures for consistency checking, and the practitioners must carry these out.
One approach that has hardly been explored is to treat the consistency checking task as having three components: information retrieval, relevancy determination and consistency analysis. The computer may be used to speed up the retrieval task, with people determining the relevance and consistency of the retrieved facts. This process does not require a formal model, but preserves the flexibility of the designers by using plain or structured text to record information.
Whatever procedures and tools are chosen for managing consistency with the requirements and conceptual design may also be used to check the internal consistency of the design. Formal methods may well be worth their inherent overhead, especially if some semi-automated tools for deriving implementations from the formal representation are available (e.g. interaction techniques with displayed objects derived from known constraints).
10.2 Enduser involvement
As many have pointed out, it is important that the enduser be involved in the design process. Designers, as creative individuals, tend to see a personal view of the needs and behavior of the enduser, and the criticism by the enduser of the emerging design moderates this tendency. Likewise, the enduser may learn something that facilitates creative suggestions regarding the design, i.e. overlooked needs and opportunities. The maintenance of contact through this phase cements the bond of understanding between the designer and the enduser, making a satisfactory conclusion to the project much more likely.
As with conceptual design, the enduser best understands concrete examples of interface scenarios. Demonstrations are very useful, but the effort of their implementation should not exceed the probable utility, and there is some danger the enduser will erroneously conclude the project is near to completion. Prose and picture descriptions are useful as well, and far easier to produce.
One method for effectively using endusers in design is called iterative refinement and has been reported by Whiteside to produce excellent results . The technique consists of having subjects use a partly implemented application system, augmented by a hidden investigator who attempts to map unexpected enduser actions into valid system functions. Such enduser inputs are later analyzed and used to produce an improved interface which is tested again.
Prototyping is frequently done, both to provide a demonstration vehicle, and to test the adequacy and feasibility of a design approach. The prototype is constructed to determine whether enduser performance meets expectations before a full implementation is done. In some cases, a prototype may be used to test the viability of the underlying hardware/software system in supporting the desired interface. Prototypes are very valuable when it is otherwise difficult to provide a rationale for a design decision. The acceptance of the prototype stands in lieu of the rationale for the design decision. This situation usually occurs when a decision has been made to consider a design approach that is novel. Such decisions are made for marketing, or internal research and development reasons, and the prototype provides a way to resolve the risk before too much time and resource has been spent.
Prototyping is only effective if a prototype can be developed much more quickly than a full implementation. A frequently made mistake, especially with workstations, is to prototype on the implementation hardware and system. Unless the prototype is seriously restricted in function, its development time is unlikely to be much shorter than a full implementation. In fact, many systems began life as a prototype and grew into a product. Extensibility and maintainability problems with such systems are common and well known.
11. Style decisions in enduser interface
The design style decisions determine the effectiveness and consistency of the interface, and its implementation cost. It is very easy to make mistakes, and there are few methodologies and tools to assist the designers. An interface design will usually be very complicated, often more complicated than the design of the computation algorithms for the associated application. A minimalist design approach, in which the addition of each new design item demands justification, guards against the introduction of needless complexity. Unfortunately, such an approach is usually unpalatable to the creative individuals involved in the design task.
The many facets of consistency have already been noted: motor skill consistency, presentation consistency, perceptual consistency, interaction consistency and conceptual consistency. At present, the characteristics of each type of consistency are topics for human factors research, thus one cannot recommend methodologies for resolving consistency issues other than study of the interface. Consistency is more likely to result when the designers have a clear and spare mental model of the interface. However, consistency at the design task is exceedingly important. The enduser's impression of the interface as sloppy and confusing will be determined by its lack of consistency.
An interaction paradigm is a perceptual unit of interaction, e.g. the dragging of a file icon onto a trash can icon. The selection of interaction paradigms should proceed with three objectives: providing the interface functionality at the right level of interaction granularity (viz. Buxton's notion of chunking ); minimizing the number of distinct interaction paradigms; making sure that the interaction paradigms fit the capabilities of the endusers. The design of presentation icons and the selection of input and presentation hardware depends on the interaction paradigms being used. There are obvious boundary conditions, as one would not want a design that required five different input devices and three or four different display devices.
Dialogue is the employment by the enduser of one or more interaction paradigms that are related to each other. The flexibility and functionality of the interface depend on the ease with which the enduser can combine and cascade interaction paradigms. Thus, interaction paradigms must be selected which are compatible with each other, e.g. they may apply to the same objects, or the result object of a paradigm may be manipulated by another paradigm. Interface mode shifts, enduser attention shifts and input device shifts in transitions between interaction techniques all impede the enduser's ability to combine and cascade interaction paradigms.
One dialogue related issue seeing a lot of research attention is concurrent enduser actions in the interface. We identified three different types of concurrency in typical use: simultaneous operation of input devices in a single interaction paradigm (e.g. simultaneous position and sizing of a displayed object in a graphical editor ; time-sharing behavior of the enduser in interacting with several tasks (e.g. a multi-tasking system with windows); unordered entry of input values or interaction events in a single paradigm (e.g. the selection of three points and the command to create a circle). A fourth type, simultaneous enduser interaction with unrelated interaction paradigms, is possible but of questionable utility. Concurrent dialogues can be very useful, but they introduce considerable complexity in the interface design and implementation. More research effort should be spent investigating concurrency trade-offs in the interface.
Direct manipulation interaction techniques are appealingly simple. However, their use is largely restricted to manipulation of concrete rather than abstract objects (e.g. particular files as opposed to sets of files characterized by some common attribute). Moreover, they may require more enduser labor and time than linguistic (indirect) interfaces do for some tasks. Reliable guidelines for the employment and design of direct manipulation interfaces do not yet exist, but there is much interest in the problem. Mixing direct and indirect interface techniques is possible but not well understood. Some research in this area indicates that endusers cannot or will not make rapid and optimum mode shifts between interaction paradigms when it would be clearly to their benefit to do so.
The members of this group spent some time working on an operational definition of direct (and indirect) interfaces without success. The distinctions seem to be largely cognitive and perceptual, and not defined by software architecture, presentation design or enduser interaction devices. Everyone agrees that pointing at visible objects, as opposed to naming them, is an example of a direct manipulation interaction paradigm. Beyond this case there is much confusion and debate. Moreover, the debate may be unimportant to the interface designer, who could be better served by reliable guidelines on the functionality, learnability and efficiency of various interaction paradigms.
The style decisions made by designers will profoundly influence the implementation of the interface. A variety of tools exist for the implementation of command language, menu and forms based interfaces. Object oriented programming has been found to be well suited to the implementation of direct manipulation interfaces. Design decisions may not ignore performance considerations, especially for workstations. Finally, the designers and the implementers must agree on the design of the interface, or the implementation is unlikely to match the expectations of the designers. There is a clear advantage to getting implementers involved in the design process at an early stage as participants and critics.
12. Tool functions
The interface designer's task is the selection of presentations and interaction techniques and dialogues which flesh out the details of the conceptual specification. The resulting design should be roughly prototyped for inspection by the endusers.
Because the design process is iterative, the tools should permit a smooth transition from conceptual specification to design (and on to implementation and testing). The smoothness of this transition depends on being able to extend the conceptual specification to an interface design, and on being able to relate design decisions back to conceptual specifications and task analysis.
12.1 Design consistency
Tools have usually focused on a particular design representation, such as Jackson Design or the Entity-Attribute-Relationship model. These is much controversy about the adequacy of such representations, and their flexibility in the design process. Flexibility and adequacy may be preserved by representing information as plain or structured text. The software tools attempt to recover text from the requirements, conceptual design and design documents based on semantically related keywords and phrases, leaving the task of relevancy determination and consistency analysis to the tool user.
The task can be done interactively, if the information database is online, and semantic retrieval tools are available. Portions of the text related to a consistency tester's query are shown on an interactive screen, where they may be dismissed as irrelevant or noted as consistent or contradictory by the touch of a finger. It will be possible and desirable to keep an audit trail of such decisions, both to measure the coverage of the work, and to uncover mistakes.
12.2 Design prototyping
The presentations were roughed out in the conceptual design stage, and details must now be added to them. It will be useful if the tools used to create storyboards and the associated presentations also generate descriptions for the implementation phase to follow. A number of research systems include WYSIWYG generators of presentations (Trillium , Menulay, , Peridot ), whose internal representation of the interface might automatically be converted to implementation code. A number of research problems must be solved, however, before such systems can be widely used.
Prototyping is best done in an environment free from concerns about system limitations and interactive performance. It should be possible to quickly alter design decisions are create a new working prototype. This is easier if very powerful, interpretive software is used on a computer with several times the speed and size of the actual enduser machine.
12.3 Technique libraries
Low level interaction techniques are typically closed packages of procedures and data. There are two reasons for this: (1) a uniformity of base interaction techniques in an interface is encouraged; the tight coupling of input and output gives better performance. Dragging and menu selection are examples of low level interaction techniques.
The choice of a low level interaction technique is governed by the task to be performed and the availability of devices on the enduser system. The decision to include another hardware interaction device in the system must be supported by reasoning that the benefit to the enduser outweighs the confusion and clutter caused by too many interaction devices.
Good programming practices should insure clean interfaces to all techniques in the library. Moreover, the techniques should be classified according to the interaction devices employed, the basic interaction semantics implemented (types of data manipulated), feedback requirements (function and performance of displays) and enduser indications and contra-indications.
It should be easy to add support for new interactions or new input and display devices. The most effective paradigm for this seems to be copying and modifying an existing interaction technique.
12.4 Interaction configuration
Higher level interactions may include multiple states, simultaneous interaction events, complex constraints, communication with the application and elaborate feedback. The complexity of the design problem at this level precludes the simple assembly of interaction technique modules.
Tools employed at this level include dialogue languages, interaction specification by example and constraint systems. The best possible world would allow the designer to create an interaction configuration in whichever tool mode best matched the design task of the moment. However, many hard problems are yet unsolved in the quest to view an interface design in all of these modes,
12.5 Testing of the design
A proposed design must be tested, by building a prototype, by analysis of the design specification, or by the construction of a user model. These approaches are not exclusive, but the latter two require both a formal model and the skills to analyze it, and are rarely practiced. There is some opportunity here for tools which employ a particular formal model and its analysis, but these tools will require training of the designer. In the case a prototype is built, it will be useful to obtain ``hostile'' users as a test group, inviting them to try to ``break'' the interface to the system .
An important task, almost never done, is the estimation of interactive performance of the interface. Design errors resulting in poor performance are almost never caught until implementation is nearly finished, at which time a great deal of effort and expense is required to correct them.
Expert systems technology may permit the construction of intelligent assistants, capable of reducing the burden of making mundane design choices, and of minor customization of chosen techniques. A few research projects are beginning to explore these technologies.
Experiences of other designers, and human factors work provides valuable guidelines for designers. A well indexed knowledge base of experiments and reports would be extremely useful, but this does not presently exist, and would be a mammoth undertaking.
12.7 Integration of tools
There is a considerable advantage in having an integrated data base of analysis, specification and design, if for no other reason than recording relationships among objects and assertions at the different levels in the design process. Whether the individual tools used against this database are integrated is not clear at present.
12.8 The interface designer's interface
It is clear that the interface designer is also an enduser, albeit with special requirements. A common complaint about current interface design and implementation tools is that they are difficult to use. This stems partly from the difficulty of the problem, partly from the considerable programming skills required by today's tools, and partly because interface designers, like the cobbler's children, suffer from badly designed and implemented interfaces.
The interface designer's interface should provide both a final form presentation and an underlying form presentation for the interface objects being designed, and allow the designer to switch rapidly from design definition to design evaluation. WYSIWYG design tools may be very effective here, together with tools to convert the design specification into code. This is obviously possible for example when a drawing package is used to create graphic objects whose interface implementation will be by calls to procedures in a standard graphics package such as GKS or CORE.
The problem of associating action and application function with interface events is only partly solved in WYSIWYG design systems. Virtual input devices that can be manipulated as if they were real make it possible to do some association by the example of performing an operation on the virtual input device and then performing the corresponding operation on the presentation object. Where the association is formulaic, or dependent on the state of other objects, exemplary association does not work well.
Some symbolic specification of interface behavior appears inescapable for the near future. This will require some programming skill on the part of the interface designer, but the task may be made easier by better specification paradigms, such as active objects and declarative specifications.
13. Research topics
Under this heading we present the topics which are considered most critical for future research in terms of the user interface designer's interface. The topics fall into two broad categories. The first includes those research topics which we in the general area of ``tool builders'' should be pursuing. The second includes those topics where research from other disciplines is necessary for our future work.
13.1 Tool building research
In this section we try to isolate the research issues underlying the further development of the UI design tools described above.
We lack formal tools to assist in the conceptual design phase as well as to provide a usable guide for further stages of design and evaluation. First on the agenda would seem to be development of the necessary formalisms for describing conceptual models. This is clearly related to developments in knowledge representation in AI. Given the development of adequate formalisms, the next step is to capture them in automated tools.
We need to find ways to incorporate developments in knowledge based systems into the interface design process and into end user interfaces. Two possible means of accomplishing this are:
· Knowledge based systems encompassing some aspect of the design process can serve as advisors to the designer. Systems with knowledge of the application domain can be incorporated into the interface to act as advisors to end user's.
· One of the major efforts of any user interface design is the development of a good on-line help system. Some of the tools and methods of knowledge engineering can be applied here.
Currently tools exist for use during design, implementation, execution and evaluation of user interfaces. While further developments are needed in all of these areas, it is also important to consider the integration of these tools over the entire life cycle.
We were unable to agree among ourselves about the meaning of the increasingly popular term ``direct manipulation.'' This leads us to believe that we are in need of a more useful and formal definition than currently exists.
The interface designer is faced with an increasing range of choices of input and output media for user interfaces. What constitutes a ``good'' choice of media for a given application? How can the selection of ``natural'' or ``appropriate'' media be encouraged?
Adaptive interfaces change dynamically depending on recent actions by the end user. Issues involved in adaptive interfaces include more than just how to build them. How does one avoid violating the ``principle of least astonishment''? Is it possible to build an adaptive interface that is inobtrusive, that does not make the user feel he is being controlled by the computer?
There is a continuing need for better tools to analyze performance with user interfaces as part of the evaluation process. This includes data intensive operations such as analyzing event history traces. This appears to be another area where techniques developed in some branches of AI might be useful.
The interface designers toolkit can be viewed as a collection of separate tools which are used independently. What is the value of integrating these tools, and what level of integration is necessary.
There are a host of documentation issues which are related to user interface tool research. Examples are use of on line documentation, automated documentation generation, and testing documentation for understandability and for correctness related to the interface design.
13.2 Research we need from others
There are many research questions which we feel are best attacked by those in other fields. Following are a few examples of such issues.
In order for us to build better on line help systems we must understand the kinds of help which prove most useful to end users. This involves understanding potential sources of confusion, as well as a better understanding of the learning process. Similar knowledge is needed by documentation designers and by those of us who would like to build automated documentation aids.
We need better models of end users, both cognitive and behavioral models. Our current tools are primarily based on an implied user model rather than an explicit one. This makes it difficult to evaluate the effectiveness of the tools in terms of satisfying the end users needs.
The framework in which this group worked resembles the software life cycle, but is substantially different. Requirements for an algorithm may be understood in advance and remain relatively stable during the remaining phases. Requirements for an interface are only partly understood in advance, and will be developed as the interface is developed. These important differences mandate an iterative rather than sequential methodology. As the enduser is the ultimate judge of the acceptability of the interface, it will be very advantageous to have enduser involvement in all phases of interface development.
The lack of good methodologies and tools for requirements acquisition, conceptual design and design is the primary concern of this task force. Some researchers have confronted these issues [26,28], but more ideas are needed, as is more experience with methodologies and tools. In this report, we have attempted to point out areas of weakness in our knowledge, in the hope that others will find challenges there as well.
Some organizations are beginning to pursue interface standards and guidelines. These efforts will help codify our present state of knowledge, and uncover areas needing further work. Much more research is needed before interface development can become an engineering discipline. Individuals with special training and skills will be needed to practice this discipline . The emergence of an interface engineering discipline depends critically on collaboration among software designers, artists and human factors designers to create the needed body of knowledge, and to initiate and develop educational programs.
The greatest impetus to this development process will come from general recognition of the benefits of good interface engineering. This recognition will not come until we interface designers include tangible and measurable objectives for enduser performance improvements in our design criteria, and demonstrate techniques for building interfaces that meet and exceed these criteria. A happy and productive computer user is the indicator of our success.
 Bennett, J.L. Collaboration of UIMS Designers and Human Factors Specialists. In this issue, Computer Graphics 21, 2 (April 1987).
 Bier E. and Stone, M. Snap-Dragging. Proceedings of SIGGRAPH'86 (Dallas, Texas, August 1822, 1986). In Computer Graphics 20, 4 (August 1986), 233240.
 Borning, A. Defining Constraints Graphically. In Proceedings CHI'86 Conference on Human Factors in Computing Systems (Boston, April 1317, 1986), ACM, New York, 137143.
 Buxton, W. An Informal Study of Selection Positioning Tasks. Proceedings Graphics Interface '82, National Canadian Graphics Association, (1982), 323328.
 Buxton, W., Lamb, M.R., Sherman, D. and Smith, K.C. Towards a Comprehensive User Interface Management System, Proceedings of SIGGRAPH'83 (Detroit, Mich., July 2529, 1983). In Computer Graphics 17, 3 (July 1983), 3542.
 Buxton, W. Chunking and Phrasing and the Design of Human-Computer Dialogues. Proceedings of the IFIP World Computer Congress, 1986.
 Ehrich, R.W. and Williges, R.C. Advances in Human Factors/Ergonomics: Human Computer Dialogue Design, vol 2, Elsevier, New York, 1986.
 Gould, J.D. and Lewis, C.H. Designing for usability—Key principles and what designers think, Communications of the ACM 28, (1985), 300312.
 Green, M. The University of Alberta user interface management system, Proceedings of SIGGRAPH'85 (San Francisco, Calif., July 2226, 1985). In Computer Graphics 19, 3 (July 1985), 205213.
 Henderson, D.A. Jr. The Trillium User Interface Design Environment. In Proceedings CHI'86 Human Factors in Computing Systems (Boston, April 1317, 1986), ACM, New York, 221227.
 Hewett, T.T. The role of iterative evaluation in designing systems for usability. In People and Computers: Designing for Usability, Harrison M. and Monk A. (eds.), Cambridge University Press, Cambridge, 1986, 196214.
 Hewett, T.T. and Meadow, C.T. On Designing for Usability: An Application of Four Key Principles. In Proceedings CHI'86 Human Factors in Computing Systems (Boston, April 1317, 1986), ACM, New York, 247252.
 Hudson, S.E. and King, R. A Generator of Direct Manipulation Office Systems. ACM Transactions on Office Information Systems 4, 2 (1986), 132163.
 Kasik, D.J. A user interface management system, Proceedings of SIGGRAPH'82 (Boston, Mass., July 2630, 1982). In Computer Graphics 16, 3 (July 1982), 99106.
 Myers, B.A. and Buxton, W. Creating Highly Interactive and Graphical User Interfaces by Demonstration, Proceedings of SIGGRAPH'86 (Dallas, Texas, August 1822, 1986). In Computer Graphics 20, 4 (August 1986), 249258.
 Nelson, G. Juno, a Constraint-Based Graphics System. Proceedings of SIGGRAPH'85 (San Francisco, Calif., July 2226, 1985). In Computer Graphics 19, 3 (July 1985), 235243.
 Olsen, D.R. Jr. and Dempsey, E. SYNGRAPH: A Graphical User Interface Generator, Proceedings of SIGGRAPH'83 (Detroit, Mich., July 2529, 1983). In Computer Graphics 17, 3 (July 1983), 4350.
 Olsen, D., Buxton, W., Ehrich, R., Kasik, D., Rhyne, J. and Sibert, J. A Context for User Interface Management, IEEE Computer Graphics and Applications 4, (Dec 1984), 3342.
 Olsen, D.R., Dempsey E.P. and Rogge R. Input-Output Linkage in a User Interface Management System, Proceedings of SIGGRAPH'85 (San Francisco, Calif., July 2226, 1985). In Computer Graphics 19, 3 (July 1985), 225234.
 Olsen, D.R. Jr. Larger Issues in User Interface Management. In this issue, Computer Graphics 21, 2 (April 1987).
 Pfaff, G.E., ed. User Interface Management Systems, Springer-Verlag, 1985.
 Rich, E.A. User Modeling via Stereotypes, Cognitive Science 3, (1979), 329354.
 Schmucker, K.J. Object-Oriented Programming for the Macintosh, Hayden Book Company, Hasbrouck Heights, NJ, 1986.
 Schulert, A.J., Rogers, G.T. and Hamilton, J.A. ADM—A dialog manager. In Proceedings CHI'85 Human Factors in Computing Systems (San Francisco, April 1418, 1985), ACM, New York, 177183.
 Sibert, J.L., Hurley, W.D. and Bleser, T.W. An Object-Oriented User Interface Management System, Proceedings of SIGGRAPH'86 (Dallas, Texas, August 1822, 1986). In Computer Graphics 20, 4 (August 1986), 259268.
 Wasserman, A.I and Shewmake, D.T. Information System Design Methodology. Journal of the American Society for Information Science, (1980), 524.
 Wixon, D., Whiteside, J., Good, M. and Jones, S. Building a User Defined Interface. In Proceedings CHI'85 Human Factors in Computing Systems (San Francisco, April 1418, 1985), ACM, New York, 2427.
 Yunten, T. and Hartson, H.R. A SUPERvisory Methodology And Notation (SUPERMAN) for human-computer system development. In Advances in Human Computer Interaction, H.R. Hartson, ed., Ablex, Horwood, NJ, 1984, 243281.