Introduction
The designer of a graphical user interface must carefully choose the set of graphical objects which can be manipulated and the operations which can act on these objects. Furthermore, it is common that some operations are appropriate on some kinds of object but not others. Thus, the design of a user interface often requires partitioning objects into sets, each with its own set of commands. For simplicity, the designer wishes to keep the number of sets small. On the other hand, the designer would also like to keep small the number of operations provided on each object set so that the behavior of that set is easy to understand. If a good deal of functionality is to be provided, then these are conflicting goals.
Likewise, it is possible to reduce the number of operations which a user must learn by providing a small set of powerful operations, rather than a large set of primitive operations. Here again one can go too far -- a very powerful operation may require so many arguments, that the user will have trouble invoking the command. Such a command can be broken into several commands which the user can perform in sequence. However, this too causes difficulties.
Whenever a user interface dialogue requires that several commands be performed in sequence, we must decide how to provide error recovery. In a singly-threaded dialogue (the user must complete all of the steps of one command before invoking another), we must be prepared to allow the user to undo and redo any of the steps completed so far, or to abort the sequence entirely, and we must be sure that the user understands that he is in the middle of a sequence which he must complete or abort before he can invoke other commands. Alternatively, in a multi-threaded dialogue, the user can invoke any command at any time. In this case, we wish to ensure that each command leaves the session in a consistent state -- multi-step commands must produce intermediate results which make sense to the whole system. We wish to consider the multi-threaded case here.
In short, our efforts to decrease the number of object sets may increase the number of the operations provided for each set. Attempts to concentrate functionality into a few powerful operations may produce operations which are hard to use. Attempts to split up powerful functions into smaller ones may require introducing new object types to serve as intermediate results. This triggers a vicious cycle. Below, we discuss these trade-offs in the context of 2D and 3D illustration programs.
First, we will describe the different kinds of objects which a user is commonly allowed to manipulate in an illustration program. Next we will describe a common set of operations which can be performed on these objects. Both the set of objects and the set of operations will, of course, be influenced by systems which already exist. However, we will try to break up the objects and operations into their raw form. The overwhelming numbers of object sets and object operations which result from this treatment will motivate our view of the design process as a process of merging object sets, combining primitive operations into more powerful ones, and breaking up powerful operations into simpler ones, to make the trade-offs described above.
Interface Objects
Part of the design of any illustrator is the specification of the nature of the objects which can be manipulated. Painting programs allow the user to manipulate pixels. Drawing programs allow the user to manipulate points, lines, and curves. Some solid modeling systems allow the user to manipulate the vertices, edges, and faces of polygons. Other solid modeling systems permit the manipulation of quadric half-spaces and their intersections. For the purposes of this discussion, we will rule out pixel-oriented systems. Since our research is motivated by a quadric-surface style solid modeler, Solidviews, we will discuss its objects and operations. Some of our analysis will carry over to the other two kinds of illustrators.
We should keep in mind that one easy way to simplify an illustrator is to delete certain object sets altogether. A quadric surface system, for instance, might not provide operations on vertices, edges, and faces at all.
Here is a rough characterization of the kinds of objects present in a solid modeler:
2D cursor.
3D cursors.
Rays.
Ray tracing intersection points.
Coordinate systems.
Vertices, Edges, and Faces.
Points, Lines, and Planes.
Assemblies.
Construction Points, Lines, Circles, Planes and Spheres.
Selections.
2D cursor. While not always thought of as an object, we choose to view it that way. In Solidviews, we use the hardware cursor provided by the hardware. This cursor moves, we pretend, in the image plane of a 3D viewing volume.
3D cursors. A second kind of cursor, called a skitter, moves around in 3-space. The skitter can be moved in 3-space by moving the 2D cursor, and specifying a mapping from the plane into 3-space (Actually, we need a mapping into 6-space since a skitter has both a position and an orientation). Much of the power of Solidviews is in the nature of these mappings. We provide operations that move the cursor to a point, in a line, in a plane, or (with ray tracing) on the surface of any object in the scene.
Rays. When a particular 2D to 3D mapping is used, Solidviews computes the equation of a ray originating at the eyepoint and passing through the 2D cursor. It may seem odd to consider this ray to be an object which the user can manipulate. However, one operation that we may wish to provide later on is to allow the user to fire rays manually (e.g. by pointing to 2 points which define it, or by pointing to an edge in the scene). In other words, one way possible interface might merge the sets "Ray" and "Line", allowing the same operations to be performed on both.
Ray tracing intersection points. When a ray hits an object's surface, Solidviews computes an intersection point and a surface normal. These intersection points could be an object set with their own operations. In Solidviews, intersection points cannot be manipulated by the user. Instead, the point and surface normal are used to compute the position and an orientation of the skitter.
Coordinate systems. Many illustrators use homogeneous transforms, at least internally, to describe the relative positions of scene objects. Each object can be imagined to have a set of 3 orthogonal axes affixed to it (often at the object's centroid). Operations such as "Rotate object A around the y axis of B's coordinate system" turn out to be quite useful.
Vertices, Edges, and Faces. Systems which permit polyhedral objects usually allow their vertices, edges, and faces to be selected and manipulated.
Points, Lines, and Planes. It may well be possible to talk about, points, lines, and planes which are not the related to the vertices, edges, and faces of polyhedra. For instance, the center of the disk of a cylinder is an important point, the axis of the cylinder is an interesting line, and the equatorial plane of the cylinder is an interesting plane.
Assemblies. Often, a scene is represented as a hierarchy of assemblies. Many operations (translate, assign color, etc.) apply to an entire assembly. For many operations, applying that operation to an assembly has the same effect as applying the operation to each of the parts of the assembly individually. In these cases, the assembly hierarchy acts as a sort of predefined multiple selection.
Construction Points, Lines, Circles, Planes and Spheres. Determining the position of an assembly (or coordinate system, or vertex, or point) may require a set of construction steps (e.g. ruler and compass). The construction process may generate a set of special objects.
Selections. Often, illustrator commands apply only to a set of objects which are "selected". In fact, there may be several kinds of selection (e.g. an object may be selected "to be moved", and a point may be selected as "the place to move to".) From this point of view, a selection is not so much an object as a state which objects may be in. In fact, if a system asks for the command first and then the arguments, the role of the selection may be determined by what the command is and by the order in which the argument is selected. However, in Solidviews, the user may select an object by pointing at its surface. Both the object selected and the position at which the selection were made can be used in commands. In this view, the selection may itself be thought of as an object A (on the surface of another object B), which represents both the object B and a specific position on B's surface.
Selections can potentially greatly increase the number of object sets. For instance, some operations may apply only to "Points selected as a place to move to".
Let me reiterate at this point that we have listed all of the categories above simply to note the possibility of providing distinct operations for each set. Doing so would almost certainly be undesireable as the user would have to learn (and the programmer provide) too many distinct operations. No doubt the reader has already thought of ways to merge categories (e.g. perhaps all operations which apply to points should apply to construction points, and vertices as well). This is exactly the point. Some sort of merging must be done, and the quality of our user interface depends on doing it well.
Merging object sets is difficult because objects can have subtly different behavior. For instance, if we try to merge Edges and Lines, we may have trouble because we cannot count on Lines to belong to faces or to have endpoints. One way to get around these problems is to provide operations which derive an object of one class from an object of the other class. For instance, we can derive Lines from Edges. This presents at least five problems:
1) We must provide the conversion operation.
2) We must provide a way for the user to select the object which is to be converted.
3) The conversion is going to take the user time.
4) Feedback. We probably should draw the derived object. This may obscure the original.
5) We must give the user a way to get rid of the derived object later.
Before we consider which objects sets to merge, then, we should examine closely the operations which we wish to provide, and the operations we will have to provide if we decide to provide conversion instead of merging.
Interface Operations
Of course, the precise set of operations which will be provided by any particular illustrator will vary greatly. Below we list some operations which are quite universal (such as selecting an object or translating an object) and some which are specific to a solid modeler. We hope that our analysis will be useful, with some adjustments, in analyzing other classes of illustrator as well. The classes of operations which we consider are:
Creating a 2D cursor to 3D cursor mapping.
Creating rays.
Transforming objects.
Positioning vertices.
Creating edges and faces.
Creating solid primitives.
Geometric construction.
Selecting objects.
Finding ray-surface intersections.
Although we are trying to present a view of a generic illustrator here, this choice of operation categories already represents some design decisions: We are using ray tracing; both quadric half-spaces and polyhedra are supported; Euler operations are not being used; and so on.
Creating a 2D cursor to 3D cursor mapping: Mapping Type. This operation changes that part of our user interface state which determines how mouse motion will effect the position of the skitter in 3-space. The operation takes a single argument, which is the new mapping type to use. This operation appears to have relatively little interaction with the other operations. However, its implementation can have a significant effect on the way primitive operations can be grouped into more powerful ones. In particular, if the user specifies the mapping type by holding down a particular key while moving the mouse around, then we will be reluctant to have the user hold down yet another key to indicate the kind of selection operation he wishes to perform. Hence, if there are several kinds of cursor mapping and several kinds of selection operation, we will have to ask the user to indicate these choices in several steps.
Creating rays: ray description => ray. We could provide a number of operations with create rays (e.g. from two points, or from the origins of two coordinate systems, or from the axis of a coordinate system or from two vertices), and then provide a set of operations which manipulate rays (e.g. rotate, and translate), and provide a "Cast" operation which fires the ray at something, and finally provide a delete operation to get rid of it. Alternatively, we could decide that rays are procedural objects only -- they are cast once when they are defined and that is the end of it. In this case, we don't need to provide the user with any operations for moving, casting, or deleting rays. We have simply deleted the ray category and distributed its functionality to other objects (e.g. vertices) on which the ray casting procedures act.
Transforming objects: object, transform => side effect. Transformation operations immediately present some difficult choices. What sort of objects can be transformed? How shall the transformation be specified? One set of answers would be: Selected assemblies can be transformed (but not individual vertices, edges, and faces). A translation transformation can be specified with two constructed points. We will discuss below how assemblies come to be selected. We are left with the question of which 2 construction points to use, if their are several. Perhaps we use the first two construction points in the order in which the construction points were added to the scene. This works fine. It is the scheme used in Griffin. Unfortunately, it relies on the existence of a good way to place construction points. In 2D, dragging them around with the mouse is adequate. However, in 3D, it would be useful to be use transformations to place the construction points themselves. This requires being able to select construction points and requires that some set of transformation operations will work on them. One could provide special selection operations and transformation operations, or one could decide that construction points should be considered to be assemblies (e.g. merge the two sets together. Solidviews takes the latter approach as described below.