File: GargoyleInterface.tioga Last edited by Bier on July 8, 1985 10:24:43 pm PDT The Gargoyle Design Document (cont) Eric Bier Maureen Stone The Gargoyle Interface In the above section we described the commands which Gargoyle was expected to handle. In this section we describe how these operations are to be brought forward to the user interface. This design is incomplete and not nailed down. We welcome comments and better suggestions. This section (when completed) will define the following: The TIP Table -- presented first as an overview The Menus -- presented first as an overview Selection Actions -- the user actions required to select and deselect control points, segments, trajectories, outlines, and clusters. Selection Feedback -- how the system shows what has been selected or deselected. Clusters -- Making and Unmaking Curve Editing Actions -- Changing spline type, adding control points, splitting and concatenating, adding and removing continuity conditions. Making and Removing Holes Changing style parameters Actions on all entities -- Move, Copy, Delete, and Transform Undo and Redo Intelligent Dragging and Rotating Symmetry Tools The TIP Table Gargoyle will have a great deal of functionality, will leverage the user's knowledge of Tioga, and will be modeless. As a result, the TIP table will be quite extensive. Here is a first cut: /Cyan/Imaging/gargoyle/mouseActions.press leftMargin: 1.5 in, topMargin: 0.5 in, width: 5.75 in, height: 4.5 in Notice that the third row does not parallel Tioga. We have given up some Tioga's delete and pending-delete accelerators in exchange for interactive dragging and rotation functions, and to be able to add a new segment. In our analogy, adding a new segment is like typing a character in tioga. However, since adding a segment is spatially (mouse) oriented function, it seems sensible to have a mouse button for it instead of using keyboard keys. The Menus Gargoyle menus will not be persistent as in Griffin. They will, however, be hierarchical. The current design is to have a two-level hierarchy where the top level menu is a regular viewers menu (persistent, located at the top of the viewer), while the second level will be pop-up menus. In this way, the user has a persistent level reminding him (in small space) of the available functionality, and a set of large pop-up menus which are out of the way most of the time. The current menu design is shown below. Most aspects are undecided or subject to change including: 1) Whether the menus are just text or include graphical icons. 2) Whether the style menus can be made to display the current style of an object (as in Star) as well as change that style. /Cyan/Imaging/gargoyle/menus.press leftMargin: 0.5 in, topMargin: 0.5 in, width: 7.5 in, height: 7.5 in These operations are not in the menus at all: Select/Deselect (but Extend is) (left, middle, right) Interactive Drag/Rotate/Scale (CTRL middle, CTRL right) Interactive Copy (SHIFT left, SHIFT middle, SHIFT right) Add at caret (CTRL-left) and these seem a little out of place: Undo (also SHIFT-ESC) Redo (also ESC) Extend (also CTRL-V). Special Menus need to be designed for the transformations (since the user must be allowed to type in numbers), and text alignment (which really cries out for a picture of a piece of text with its alignment points showing so the user can just point at the desired one -- Gremlin at UC Berkeley already does this). Selection Actions A difficult problem. Seven different kinds of objects can be selected: Control points, joints, segments, sequences of segments, whole trajectories, whole outlines, and whole clusters. Parts of clusters may be selected without selecting the whole cluster. Furthermore, we are trying to leverage the user's knowledge of Tioga where possible to make the selection mechanism uniform. Here is the current design: Selection operations are performed with all three mouse buttons, by themselves or with SHIFT held down. The left mouse button and the right mouse button (in control point mode) work exactly the same way except that the left mouse button performs an automatic "Deselect All" before any selection is made. The three buttons work as follows: Control-point level selection: Clicking the left mouse button over a single control point or joint, deselects all entities, and makes that entity the sole selected entity. It also puts Gargoyle in "control point mode". With Gargoyle in this mode, clicking the right mouse button single control point or joint adds that entity to the list of selected entities. While this selects control points and joints directly, it selects segments indirectly. A segment is taken to be selected if all of its control points is selected (including both joints). Note: left-clicking in free space positions the caret and deselects all. Right-clicking in free space positions the caret. Segment-level selection: Like text selection in Smalltalk. With Gargoyle in "control point mode": Pushing the right mouse button down over a control point or joint begins the selection of an ordered sequence of control points. The second point approached determines the sense (roughly clockwise or counter-clockwise) of the selection. The last point under mouse before the button is released is the second endpoint of the selection. Again, a segment is taken to be selected if all of its control points is selected (including both joints). Trajectory selections: Holding down the middle mouse button near any point on a trajectory temporarily highlights that trajectory. Moving the mouse button will allow this highlighting to change. When the button is released, all selections are deselected and the highlighted trajectory becomes the sole selection. Gargoyle is placed in "trajectory mode". In "trajectory mode", holding down the right button will highlight the nearest trajectory temporarily and releasing the button results in the highlighted trajectory being added to the list of selected entities. Outline-level selections: If an outline has only one trajectory, then selecting the trajectory selects the outline. In any case, double-clicking with the left mouse button near a trajectory belonging to an outline (or in the interior of an outline if it is filled) will select that outline. This puts Gargoyle in "outline mode". The right mouse button can now be used to extend to other outlines. Top-level selections: Double-clicking the right mouse button near any trajectory (or in the interior of a filled outline) will select the top-level outline or cluster to which that trajectory belongs. This puts Gargoyle in "top level mode". The right mouse button can now be used to extend to other top-level entities. Intermediate nodes: Nodes which are neither top level nor bottom level can be selected by adding a subpart of the node to the list of selected entities (as described above) and then extending that selection with CTRL-V. In Tioga, CTRL-V extends the selection by one level only. In Gargoyle, the extension will extend each time CTRL-V is used until the top level is reached. Recovering from errors: If a selection is made by accident, there are several ways to recover: Start over: Using the left mouse button deselects all and lets you start over. Undo last selection: If no action has been taken since the selection was made, then SHIFT-ESC (BackSpace?) will remove the selected entity which was most recently added to the list. Deselect: CTRL-SHIFT left, CTRL-SHIFT middle, CTRL-SHIFT double-left, CTRL-SHIFT double-middle work just as though a new selection was being made except that a special highlighting is used. When the command completes, the selected point, point sequence, trajectory, outline, or cluster is deselected. Of course, this only has an effect if all or part of the entity was previously selected. Abort: Hit DEL before lifting the mouse button. The previous set of selections will be restored. Switching modes: Gargoyle will toggle back and forth between the mode set by the left-mouse button and fine-grain mode. Double-clicking the right mouse button toggles the grain. This does not permit arbitarity switching between grains, but should be adequate for the common cases: 1) All selections at the same level. 2) Heterogeneous selections of Trajectories and control points. and CTRL-V can still be used to avoid having to used fine-grained selection to select every last control point of a trajectory, outline, or cluster. One last problem: Can a cluster and a sub-part of that cluster be on the list of selected entities as the same time? The answer is probably no. Hence, selecting a superset of a selected entity removes that entity of the selected list. Selection Feedback The first cut at selection feedback involves blackening joints and control points. This form of feedback seems the only way to go when control point sequences are being selected. It is not so obviously good if a whole trajectory, outline, or cluster is selected. The feedback which shows that an object is selected must be somewhat separated from the feedback which shows where control points are so that they may used for alignment, copy-selected, or selected as a sub-sequence. However, these are related since a control point probably shouldn't be selectable unless it is visible. Finally, it should be easy to turn off all of the selection feedback so that they artist can examine the picture drawn so far. One simple scheme is this: 1) Visible control points display as empty squares. Selected control points appear as solid black squares. 2) Control points cannot be selected individually unless they are visible. 3) All or no control points of a trajectory are selected at once. 4) A command, ShowPoints will be provided which makes visible all of the control points of objects on which selections exist. HidePoints will perform the inverse. 5) As an accelerator, "control point mode" commands will automatically make control points visible when they would cause a selection to occur. 6) When an entire trajectory is selected, all of its control points are blackened. This is independent of whether or not the control points are visible. 7) When an entire outline or cluster is selected, all of its control points are blackened. This scheme has the horrible disadvantage of being very noisy. However, it is relatively simple and will do for a start. Clusters The Cluster menu button takes all of the selected entities and makes them immediate children of a new cluster. If partial trajectories, or partial outlines, or partial clusters are selected, this command prints an error message and does nothing. Uncluster assumes that all selected entities are clusters. It removes the top layer of clustering from each. If some selected entities top-level outlines, it simply takes no action on these, but does work on the clusters. If a non-top-level object is selected, Uncluster prints an error message and does nothing. Curve Editing Actions: Borrowing from Tioga Select/Deselect/Extend. See Selection Actions and Selection Feedback above. {Add/Splice} Point. Initially, Gargoyle will use a trajectory creation approach similar to that used in Darlene Plebon's PathTool. In particular, the curve and the control points are shown at the same time. All editing operations "rubber-band". To begin a trajectory: As described in Selection Actions, left clicking in free space positions the caret. CTRL-left click, adds a new segment from the caret to the mouse-position. The type of this segment is initially line-segment but can be changed. After, the line segment has been added, the caret moves to the mouse position. Clicking CTRL-left again will now add a second line-segment, and so on. Splicing in a point. Left-clicking over an edge will position the caret on that edge. CTRL-left click at this point will break the caret-segment into two parts, both of the same spline type as the original segment. The joint between these two new segments will occur at the current mouse position. Moving Points. Points are moved by selecting them and then dragging them (see Intelligent Dragging below). The Merge Problem. One of the nastier unsolved interface problems is the matter of merging joints. We plan to provide a gravity mechanism so that points can be placed exactly on top of other points and on curves. Furthermore, we wish to remember which points were placed this way so that we can take advantage of this information in rendering. Finally, we need a way to specify that a contour is to be closed. Perhaps this can be done as follows: A menu command Close will be provided to close trajectories (rather than using gravity). This command adds a line segment between the two endpoints of the trajectory. A menu command Weld will combine two trajectories together if they already share an endpoint. A menu command Split breaks two trajectories apart at the current caret position. Whenever gravity is used to bring two contours together, this fact is noted but makes no change in the object hierarchy. This touching relationship is a "fragile constraint" as described in the Gargoyle Capabilities section above. Its existence is noted, but no effort is made to preserve it. Gargoyle must notice, however, when a fragile constraint is broken (and should inform the user). Fancy Curve Editing Changing Curve Type. Select the segment or segments to be changed. Pull down the curve type menu and click the desired spline type. Each selected segment will be replaced with a curve of the given spline type, passing throught the same endpoints (joints) as the original. Where possible, the shape of the segment will be preserved (for Conic to Bezier, this might work, for Conic to Straight Line it won't). This is just a beginning plan. Part of the research of Gargoyle is to develop a better scheme. Constraining Joint Continuity. Select a control point sequence. Click the Smooth menu button in the Edit Curve menu. All joints in the control point sequence are now constrained to have continuity of slope. Since they may not currently have this property, the current curve may have to be changed to make this property true. When constrained joints are visible, they display themselves with an X in addition to (instead of?) their hollow box. Clicking the Corner menu button removes the continuity constraint on all selected joints. Extra Points. Each segment may have a set of extra points associated with it. To add an extra point, place the caret and click the Add Point menu command. Extra points are displayed whenever control points are visible. They are displayed in a distinct fashion ('+' sign?) to be distinguishable from control points. Extra points can be used for alignment operations just like control points. (Also see the next paragraph.) Extra points can be deleted with Delete. Intersection Points. Select a number of trajectories. Click the Intersect menu button. Gargoyle will find all places where the selected trajectories intersect (including self-intersections), and will add to each segment extra points representing its intersections with all other segments. These extra points can then be used, along with Split and Weld, to provide some of the functionality which union, intersection, and difference operations will provide some day. Making and Removing Holes Adding Holes. Select any outline which includes a closed (fence) trajectory. Extend the selection to any number of outlines which have only closed (fence) trajectories. Click the AddHoles menu button. Each of the simple outlines donates its trajectory to be a hole in the outline which was selected first. All outlines but the first are destroyed (and information about how they were filled is lost). The donated trajectories maintain their color and line width styles. If any of the arguments is improper, an error message is printed and no action is taken. Removing Holes. Select any number of hole trajectories. Click Delete. All of the holes will go away. Alternatively, click FreeHoles. In this case, each of the holes becomes a simple outline with a single fence trajectory. The fill properties of the holes is taken from the default style. Changing Style Parameters LineEnds, LineWidth, LineColor can be set by selecting a set of segments and invoking the appropriate menu command. AreaColor is set by selecting a set of outlines and invoking the appropriate menu command. Move and Copy Below, we describe how objects can be moved by dragging them into place in a continous fashion. Here we describe certain discrete motions which may be useful. Move. Place the caret on a control point, joint, or segment of interest (or at a point in free space). Move-Select any collection of entities. A special move-select caret will appear as you do this. This caret also can be placed on any object or in free space. When the selection is done, the objects are translated so as to put the move-select caret on the primary caret. Objects may NOT be moved if they are part of something larger (e.g. an outline or a cluster). Copy. Like move select except that copies are made of the copy-selected objects before translating them. Individual control points CANNOT be copied. Sequences of segments, trajectories, outlines, and clusters CAN be copied. Objects may be copied even if they are part of something larger (e.g. an outline or a cluster). A sequence of segments becomes a complete trajectory (and hence a complete outline) when it is copied. Copied trajectories (even holes) become complete outlines when copied. If the primary select caret occurs at a trajectory endpoint, it is probably a reasonable default to automatically weld the copied sequence to the target trajectory (it can always be Split off if this is not the intention). Delete and Transform Delete. Can be invoked with the Delete menu button, or by hitting DEL. All selected objects are deleted. If the fence trajectory of an outline is selected, but not all of the holes are, Delete prints an error message and does nothing. Transform. Translation, rotation (and maybe scaling) can be done with intelligent dragging (see below). Translation, rotation, and mirroring can be done with symmetry tools (see below). However, for those times when the user would prefer to type exactly the desired transformation, we provide a set of transformations which take numerical arguments: Translate: x, y Rotate: Degrees counter-clockwise, origin Scale: x-scalar, y-scalar, origin SkewX: y coordinate of the line which is unaffected, Degrees counter-clockwise. SkewY: x coordinate of the line which is unaffected, Degrees counter-clockwise. MirrorX: x coordinate MirrorY: y coordinate As in Griffin, these commands will come with an option to make a Copy instead of transforming the original. A Get Caret Position command will allow the user to discover interesting coordinates. In fact, we may decide to display the caret position at all times. Undo and Redo Refresh for Dragging -- Animation Below, we discuss how objects will be dragged and how Gargoyle will help the user align objects while dragging them. Before this, we should mention the different effects which dragging can have on the selected entities and how these effects will be displayed on the screen. Dragging a rigid body. During dragging, the objects pop up to an "overlay" plane, such that every object in the overlay plane is in front of all objects left behind in the background plane. The selected objects maintain priority with respect to each other. Objects in the overlay plane are drawn as quickly as possible (e.g. with Imager fast-path operations) to make the animation appear smooth. Objects in the background plane are drawn properly. The image is updated as often as possible. When dragging completes, the image is repainted showing the selected objects in proper priority order. Rubberbanding while deforming. If some, but not all of the control points of an object are dragged, the object changes shape. As for dragging, the rubber-banding objects are moved to the overlay plane and are redrawn as quickly as possible, using fast-path operations. Intelligent Dragging and Alignment Tools The Idea -- Getting Away From Grids (and Constraints) A number of graphical editing operations require precision. These include: 1) Making two lines be parallel. 2) Making two line segments be the same length. 3) Making a line segment be horizontal or vertical. 4) Placing a line endpoint on a curve. 5) Placing two curves to be touching. 6) Scaling by a known factor. 7) Translating a known amount. 8) Rotating by a known angle. 9) Making an object have mirror symmetry. Griffin responded to the need for precision by using a grid. Grids in Griffin have a number of disadvantages including: 1) Once an object has been moved without the grid, it cannot be realigned with the grid. 2) Careful counting of grid positions is required to make lines parallel; to make lines the same length; to scale, translate, or rotate by a precise quantity, and to make an object have a certain symmetry (precise angles are especially hard). 3) The user is limited to lengths which are multiples of the grid unit (or must spend time changing the grid unit -- not possible in Griffin). 4) The problem of making objects touch is not really solved for curved objects. Juno responded to the need for precision by allowing the user to specify relationships with constraints. This has the advantage that the user's intent is made explicit, but has a number of disadvantages: 1) Decisions about which point to move to satisfy a constraint are made by the constraint solver. When the shape is only partially constrained, these decisions usually produce a different shape from the desired one. 2) Since shapes must be perfectly constrained to protect the user from the "whim of the constraint" solver (see point 1), the user must constrain every last degree of freedom. (For Greg Nelson's block letter A, this meant 20 constraints!). This can be quite time consuming. 3) No provision was made for Scaling, Translating, or Rotating by a know factor. Symmetry required a relatively large number of constraints. (For the block A example, a single "mirror symmetry" constraint would eliminate the need for 10 of the 20 primitive constraints.) 4) Objects could be made to touch at their control points, but not at an arbitrary location. The idea in Gargoyle, is to satisfy the need for precision without the limitations of grids or the unwieldiness of constraints. This involves two (related) mechanisms -- intelligent dragging, and alignment tools. We will briefly describe these mechanisms and then discuss how they satisfy the need for precision. Intelligent Dragging Intelligent Dragging is the provision of extra feedback during smooth dragging and rotating operations. The mechanism is as follows: As the user drags (or rotates) a collection of objects across the screen, Gargoyle notifies the user when certain alignments occur. For example, we might draw special overlay lines on the screen when lines on the moving object become colinear with lines on a nearby stationary object: /Cyan/Imaging/gargoyle/boxAlign.press leftMargin: 0.5 in, topMargin: 0.5 in, width: 6.25 in, height: 4.75 in Similarly, when we are dragging a single control point around to reshape a trajectory, we may be interested in times when that control point is colinear with some line in the scene: /Cyan/Imaging/gargoyle/pointAlign.press leftMargin: 1.0 in, topMargin: 0.75 in, width: 4.0 in, height: 5.75 in We will discuss other types of alignments in the Intelligent Dragging section below. Alignment Tools One of the advantages of a constraint-based system is that the system automatically propogates a change in one point to many other points. Unfortunately, as noted above, the constraint solver often repositions the points in an undesired fashion. Alignment tools in Gargoyle provide a deterministic (real-time) motion of many points when one or more representative points are translated or rotated. This mechanism cannot be as general as a full-blown constraint mechanism, but may turn out to be as useful. Potentially, there will be an alignment tool for each of the symmetry groups of the plane: 1) The cyclic groups Cn (shapes invariant under rotations of 2 2) The dihedral groups Dn (shapes which are Cn and invariant to reflections about lines at the angles i All of the regular polygons are in this group as well. 3) The frieze symmetry groups (shapes invariant to translation parallel to a line, reflection about the line, and/or reflections about a perpendicular to the line). Picket fences and centipede bodies are in this category. 4) The translational symmetry groups. Brick walls, grids, and hexagonal bathroom tile are in this category. The figure below shows a hexagon being designed with a C6 symmetry tool. From this description, it should be clear that alignment tool operations require that a dependency relationship be known for some points in the scene. For instance, if an alignment tool is going to help us design the hexagon, it must know that all six points are related to each other by a rotation about the center of the hexagon. Hence, an alignment tool is used in two stages. First, correspondences are established between various parts of the scene. Then, the tool maintains those correspondences as long as it is active. Restrictions will be placed on which tools can be active simultaneously so that every tool can always maintain the correspondences for which it is responsible. Providing Precision Intelligent Dragging There are a large number of alignments which the user might be interested in, and we cannot possibly anticipate all of them. However, we hope to build in the most used alignments and to provide a way for the user to construct his own. To begin with, we note that translational alignments are of the following three types: 1) A point is on another point. 2) A point is on a curve. 3) A curve is tangent to another curve. Rotational alignment can involve any of the translational alignments plus: 4) A straight line is parallel to another straight line. By curves, we mean curves which can be derived from the scene as well as curves which are drawn in it. For instance, we might wish a point to be colinear with a certain line segment in the scene, but not necessarily on that segment. In this case, we say the point is to be on the infinite line which includes that segment. This can be used when dragging a point into place to make a trajectory: It is also common to use centers of symmetry as alignment points, and symmetry reflection lines as alignment curves. We intend to compute and maintain information about the symmetries of various Gargoyle scene objects to test the usefulness of this idea. It may be that simply providing attachment points at line segment midpoints (as is done in Star) would provide most of the advantages we can hope to gain. We suspest otherwise. Symmetry lines provide a way of aligning objects even if they don't need to touch, and allow for alignment of rotation as well as translation. One of our experiments in Gargoyle will be to provide an interactive rotation, where objects rotate freely and Gargoyle announces interesting rotational alignments. Symmetry Tools