<> BOWMAN AND FLEGAL IDIOMATIC ILLUSTRATORS SIGGRAPH '87 TUTORIAL COURSE NOTES DOCUMENTATION GRAPHICS === length: 1 in Idiomatic Illustrators Idiomatic Illustrators William Bowman and Robert Flegal User Sciences Group Xerox Palo Alto Research Center November 1975 Introduction This report describes an effort to design and implement a set of computer-based graphic tools that enable people, unskilled in either Graphic Arts or Computer Science, to easily illustrate technical ideas and information. The basic notion explored was: is it possible to break down the world of technical graphics into `idioms' (constrained environments) such that the computer could provide both mechanical and aesthetic aid to the non-professional user. In order to test this concept, we divided technical graphics into four basic environments: 1. quantitative 2. ideographic 3. isomorphic 4. volumetric. Each of these basic environments was then further subdivided into graphic `idioms'. For example, piecharts and barcharts are quantitative idioms while exploded views and cutaways are examples of volumetric idioms. From the wide spectrum of possible idioms we chose to examine three of them: a typographic idiom, block diagrams, and piecharts. This report is primarily devoted to a description of the `idiomatic' approach to computer graphics as we experienced it within the context of working with these three idioms. 1. Idiomatic Illustrators Objectives The aim of this project was to provide Alto-based graphics tools that would enable people unskilled in either computer science or the graphic arts to easily construct articulate graphic statements. This was a six-month project, begun in February 1975 and concluded in August 1975. Method of Approach We conceived a research plan for creating a series of special-purpose subsystems, called illustrators, to deal with graphic problems on a specific rather than a general level. The design of these special-purpose illustrators was driven by an attempt to conform to conventional notions about graphic `idioms' which are commonly understood and used in the working world. To establish a comprehensive frame of reference for this approach we reviewed a wide variety of illustrations, and constructed a graphic mural (reproduced on the following page) which represented four basic graphic environments: 1. quantitative 2. ideographic 3. isomorphic 4. volumetric. Quantitative figures dealt with visual translations of numerical data. Ideographic figures symbolized conceptual information. Isomorphic figures communicated through abbreviated versions of real forms. Volumetric figures represented objects as they appear or might appear. In each environment we subdivided illustration types in terms of communicative aim, and displayed various particular occasions of each aim. Each of these specific aims, along with its associated occasions, we called an `idiom'; and it is on this basis that we built the idiomatic illustrator project. The reason for choosing the idiomatic approach is that one does not need to have the whole world of graphic language at one's command to create a bar chart (a graphic `idiom'); all one needs is some bars, a scale, and some labels. By the same token, if one would rather make a pie chart (another graphic `idiom') one doesn't need bars and scales; one needs a circle and some dividing lines. Applying this approach, the barchart program would only draw bars and the piechart program would only draw pies. Too constrained? Not for the unskilled user who simply wants a bar chart now without having to master the illustrator's bag of tricks, both technical and aesthetic. For the unskilled user, constraint means support: the user enters the graphic world at an idiomatic level, and so can deal with his/her ideas using the specific secondary forms which represent them (scales, bars, pies, etc.) rather than the more general primary form vocabulary (line, shape, texture, etc.) of the professional illustrator. ==================== ==================== Scope From the range of possible idiomatic illustrators we chose to work with three: SIGNA typographic program for simulating letraset type in making headlines, poster-notices, view-graphics, etc. (This idiom was not represented in the graphic mural.) BLOCKAn illustrator program for making block diagrams, organization charts, process charts, etc. PIE A program for visualizing tabular data automatically in the form of a pie chart. SIGN was chosen because of its simplicity and because it was needed by the PARC video communications group to make titles for their videotapes. This meant a set of real users with whom we could try out our ideas. BLOCK was chosen because of its potential value to PARC as a communication tool, and because it offered us an opportunity to deal with the basic graphical problem of form and space interaction. PIE was selected so that we could get some experience with an automatic table-driven illustrator. All of these programs were written in SMALLTALK, with much help from people in LRG. The following three sections of this report describe in detail the basic features of SIGN, BLOCK, and PIE. The last section presents research conclusions drawn from this project. 2. The Sign Program SIGN is a modest typographic program originally designed to produce hard copy text titles for use in PARC's videotape projects, but it is equally useful for creating bulletin-board notices, small posters, identification labels, view-graphs, and other kinds of `social-style' office communications. SIGN distinguishes itself from other text systems in that it is environmental: that is to say, it can be used to create word `pictures' that catch the eye in the physical world of competing visual objects, such as the PARC office scene. The basic design criteria for SIGN were: 1. A minimum 24 point font size, bold, and sans serif to insure readability in the video medium. A 24 point helvetica bold face was chosen. 2. Exact compositional control on the Alto screen and identical hard copy by SLOTso that what you see is what you get. 3. A simple operating procedure that enables people not skilled in computer science or the graphic arts to create professional headline text a-la-letraset. SIGN is also a step toward solving the graphical problems associated with text headings. Currently, it lacks a coherent scheme for dealing with margin justification, color, changeable leading, inter-character spacing, etc. Much interesting design remains to be done in this area. Two details about SIGN deserve mention: the spatial gridding and the ease with which a user can obtain hard copy. Vertical gridding is always enforced between lines. There is a grid of line of text is first specified. This aids centering along a vertical guideline. After the initial placement of a line of text, the horizontal gridding is relaxed. This allows for subsequent margin justification. The output is obtained through the use of command files (lots of crocks) which eventually send a press-format file of the screen image to LPT. The important point about output is that the program owes much of its popularity to the ease with which one can obtain it . . . with the `push of a button'. The command language for SIGN is menu-driven and `modeless'. The menu itself looks like a sign so as to relate aesthetically with the text on the screen. The SIGN program is the most complete of the illustrators discussed in this report; the design criteria were met, and the program has found much use. SIGN is a particularly interesting idiom in that it lies graphically between `run-on' text and illustration. Simply stated, SIGN deals with text pictorially. This is a very common procedure in the graphic design world in the production of headline materials for magazines, books, brochures, etc. The following examples illustrate some possible uses for SIGN. ==================== REMINDER SSL SHOWING OF XIP VIDEOTAPE WEDNESDAY APRIL 23 1:30 PM SECOND FLOOR COMMONS ROOM ==================== ==================== LRG STUDENT SCHEDULE DAY TIME NUMBER MON 9:00 - 11:30 5 TUE 3:00 - 4:30 3 WED   THU 1:30 - 3:00 10 FRI 9:00 - 11:30 5 ==================== ==================== SEMINAR BY: ROBERT KHAN TITLE: "PACKET RADIO A MICROPROCESSOR BROADCASTING NETWORK" TIME: 11:00 AM PLACE: CSL COMMONS ROOM ==================== SIGN: Summary Evaluation 1. Videotape title applications are very successful, and the program is now used regularly for that purpose by PARC's video communication group. 2. Totally inexperienced users in the video group were able to operate the program immediately, as were secretaries, researchers, and others in the PARC community. 3. The volume of general (non-video) office applications has been much larger than we expected, and has proved the program to be a useful multi-purpose workhorse. A dribble-file associated with the program has recorded this volume of use. 4. SIGN's single-font (caps only) capability is far too limited for most practical applications. Currently, we have no easy answer for this deficiency. 5. The program is essentially an elegant hack, and consequently some users have experienced frustrating breakdowns. 6. The move function is still crude, and offers inadequate support for the variety of alignment and spacing situations which commonly occur in graphic design. 7. Conceptually, SIGN offers considerable promise as a headlining device for graphic design work, particularly in the areas of magazine, book, and brochure production. The main reason for this is that it treats word forms as graphical objects, and consequently relates to the graphic designer's methodology. 3. The Block Program BLOCK is designed to deal with graphic problems in the idiom of block diagrams; including organization charts, process charts, and other rectilinear figures. This program is a specialist. It does not attempt to take on the whole world of graphic needs, although a few interesting by-products such as 3-D perspective are possible. BLOCK takes a view of graphic language that emphasizes design grammar (spatial dynamics, composition, etc.) rather than form vocabulary (gray scale, sophisticated detail, etc.). It is intended to help the ordinary (non-illustrator) user construct an articulate graphic figure without having to learn the illustrator's profession. Basic aesthetics as well as manual skills are supplied by the program. The design criteria for BLOCK were: 1. A basic form vocabulary of lines and rectangles for building the structural elements necessary to block diagrams. Secondary requirements included word and arrow forms. 2. A spatial grammar for composing form elements on the Alto picture plane with respect to aesthetics of planar design (visual relationship and differentiation). 3. A capability for visual editing, including move and copy functions. Later, an area move/copy function was added to the criteria. 4. A set of graphic processing utilities, including such functions as clean (refresh), file (save and get), print (XGP), and reset. BLOCK, like SIGN, has been developed to the point that it is a usable SMALLTALK subsystem for making illustrations. The essence of the BLOCK program lies in its gridding scheme which spatially organizes its graphical forms (box, line, arrow, text) in an aesthetically related manner. During the design of BLOCK it became clear that no existing font was suitable for diagrammatic purposes, so we designed and executed a new font. The design criteria for the font (BLOCKFONT) were: 1. That it be a condensed font to maximize horizontal space on the Alto screen, which is a major constraint in making diagrams. 2. That it have the smallest bold (2-bit thick) face possible on the Alto screen, and still remain readable. 3. That the font relate aesthetically to the rectilinear forms generated with the BLOCK program. First, an Alto font satisfying these criteria was designed. Its dimensions are 6 x 10. Subsequently, a coordinated spline outline version was constructed. This font should find wide usage in PARC terminal displays where horizontal space is at a premium. The command language for BLOCK is menu-driven and `modeless'. The thirteen commands are divided into four logical groups: 1. form vocabulary (box, line, arrow, text) 2. space control (grid module) 3. editing functions (area, move copy) 4. memory commands (print, save, get, reset, clean) The menu itself is presented in the form of a block diagram for (1) aesthetic relevance and (2) to enable the visual presence of a number of command options without creating a sense of visual confusion. Individual command functions for BLOCK are as follows: BOX: Draws boxes, any size or shape. The command requires two mouse inputs: upper left and lower right box coordinates. The box corners are positioned at the nearest points on a 32-unit grid. This aligns boxes automatically, provides consistent spacing, and allows the user to be rough in his/her manual command executions. LINE: Draws lines, any length or direction. The command requires two mouse inputs: beginning and ending points of the line. The line endpoints are positioned at the nearest points on a 16-unit grid. Because of the grid, lines will automatically split spaces between boxes, and provide centering and exact box contact when used as connecting links. In addition, lines built at right angles to each other automatically form a perfect corner. Again, the user may be somewhat rough in manual execution without problem. ARROW: Draws lines with arrowheads attached to the point designated by the second mouse input. Arrow lines may be any length, vertically or horizontally. In all other respects this command functions like line. TEXT: Prints a line of text as objects anywhere in the figure, an 8-unit grid. The text automatically centers itself within boxes. Inputs are typed sequences (terminated with carriage return); and mouse points (center location for text). MOVE: Moves any of the above objects anywhere in the image, in terms of its assigned grid. Move can also be used to dump unwanted objects into the garbage can at the bottom right of the screen, causing them to disappear. Two mouse inputs are required, corresponding to old and new locations. COPY: Copies objects anywhere in the image, in terms of assigned gridding. Like move, two mouse inputs are required, to indicate form selected and the desired position of its copy. AREA: Selects a form rather than an object, for moving or copying. As in box, two mouse inputs are required to indicate upper left and lower right corners of the rectangular area selected. In addition, third and fourth mouse inputs are required corresponding to old and new locations for the forms included within the rectangular area selected. The rectangular area selected will then be moved or copied in the new location. GRID: Permits the user to change the assigned grid spacing for any particular form. PRINT: Creates an XGP file for hard copy. Input is a filename (one word terminated with line-feed). The file created may then be transmitted to a NOVA with an XGP, and the command `XPLOT filename' given to the NOVA operating system. CLEAN: Refreshes the entire image, restoring forms damaged by moving, etc. SAVE: Allows images to be saved for future display, printing or modification. GET: Allows previously saved images to be recalled. RESET: Erases entire screen and restarts the BLOCK program. In addition to menu commands, line weight for any form may be controlled by the mouse button pushed: 1. top button: fine line 2. middle button: medium line 3. bottom button: heavy line. We have included a group of illustrations which describe BLOCK's range of capabilities, and suggest how the program might be used. ==================== ==================== ==================== ==================== BLOCK: Summary Evaluation 1. The most successful aspect of this program is its spatial control of form. The notion of `invisible' gridding as a strategy for the management of form/space interaction (design grammar) worked well, and has since been used with equal success by other programs at PARC (e.g., MARKUP). 2. The simplicity of BLOCK has enabled many (graphically) inexperienced users to construct effective block diagrams. However, it is also clear from the work done that BLOCK does not `do it all' as we had hoped, and that some elementary graphics skills are still required. 3. The BLOCKFONT worked well as a conserver of horizontal space, and competes well in the context of diagrammatic form. 4. Area move and copy functions are still difficult to control, and require too much visual editing. The displacement for all objects within the area is gridded according to the current grid setting for text objects (usually the smallest). 5. The concept of a fixed push-button graphic menu was, as in TAPE, felt to be an improvement over keyboard-oriented command systems. By the same token, it now appears that MARKUP's spatially-flexible menu system and TOOLBOX's keyset control system are much easier to operate than BLOCK's fixed menu. 6. BLOCK lets the user know where his/her cursor is in relation to the `invisible' grid spacing by moving the cursor to the nearest grid point (according to the form being created) when the mouse button is depressed. As long as the button remains depressed the cursor "hops" from grid point to grid point when the mouse is moved, and a point is specified when the mouse button is released. 7. It is a demonstrable fact that infinite variations on the `block diagram' theme can be created with relative ease using this program. However, exactly where BLOCK ends and FLOW, or PERT, etc., begin is not yet clear. Further exploration with other related idioms would help to answer this question. 8. Some fairly sophisticated illustrations can be created through line vocabulary alone. 9. Mouse buttons work well as a tactical means for controlling line weight. 10. In an illustrator context, it helps to be able to deal with words as graphic form objects (like lines or boxes). 4. The PIE Program PIE is an experimental effort to create an `automatic illustrator'; that is, a program that puts the `illustrator' entirely within the machine and thus allows the user to get a professional-level illustration without having to perform any graphical tasks. The graphic idiom of pie charts was chosen for this experiment because, as a data-based idiom, it lends itself naturally to mechanical graphic translation. The basic graphic design decisions in making a pie chart are quantitative: not only the spatial division of the pie into its component segments, but also the placement of labels in relation to the available space resulting from those segments. Therefore, all that PIE requires of the user is a table of items (labels) and their associated numerical values (segments). The program (1) makes the pie, (2) translates the numbers into percent values and cuts the pie into corresponding pieces, and (3) attaches item labels to the segments. User interaction takes place entirely within the context of creating and/or editing the tabular data, a familiar and ordinary office activity. Basic design criteria for PIE were: 1. A form vocabulary comprised of a single fixed-diameter circle, straight radial lines within that circle, and text labels. 2. A spatial grammar that translates a set of numbers into degree equivalents, and represents those equivalents as pie segments using radial lines. 3. Automatic/aesthetic label placement, with respect to shares and positions of pie segments. 4. A system for tabular data entry that permits interactive user editing. In satisfying the design criteria for an automatic piechart-maker the most difficult problem was that of label placement. The strategy for this part of the program was as follows: If a pie segment had adequate size and/or an advantageous position for (horizontal) text, then (1) the label would be placed internally and generally centered within the available space. If the segment was small and/or vertically oriented, then (2) the label would be placed externally, and related to its segment by a connecting link. The space available for a text label within a slice of the pie was computed as follows: (a) point p is chosen so that it lies on the bisector of angle ( ==================== ==================== (b) the four points S1, S2, S3, and S4 are computed by finding the intersections of the line y=x(Py+ the circle. (Note: h=font height) (c) next the four points t1, t2, t3, and t4 are computed by finding the 4 intersections of the line y=x(Py l2 and the circle. (d) finally, if si and sj (Sj,tl) is the space that text may occupy and still be inside the slice defined by It should be noted that the space for text obtained by the method just described does not yield the maximum width rectangle that can lie in a segment of the pie. Originally we completed the maximum width rectangle that can lie in a segment. Placing the text centered in this rectangle caused graphical interference between the text and the radial lines which divide the pie and the links used where text could not fit inside the slice. Hence we chose the algorithm which, in general, produced a smaller space for the text but yielded a more aesthetically pleasing result. The system for external pielabels sought to maximize the number of possible labels that could be automatically arrayed around the pie, and at the same time make the most economical use of available space on the Alto screen. It appeared that a parabolic arrangement of labels around the pie produced the most efficient and manageable external pielabel system, as illustrated by the following design drawing: ==================== ==================== The user interface for PIE is a simple table, into which the user types item names (for labels) and corresponding numerical quantities (for segments). As the user types in items and quantities the table expands downward. This table can be edited: items and quantities can be added, deleted, exchanged, or moved as desired. The order in which items are displayed corresponds to the order in which they are represented in the pie (starting at `noon' and advancing clockwise). Thus, the user has control over the segment arrangement in his piechart. It should be emphasized that unlike SIGN and BLOCK, PIE is still in an experimental state, and not yet ready for dependable work applications. However, we have tested the program against a variety of data situations, and have essentially succeeded in satisfying the original design criteria established for the idiom. We can offer the following illustration, executed with PIE, as an example of the program's current capability: ==================== ==================== PIE: Summary Evaluation 1. For certain kinds of illustrations (particularly quantitative) automatic illustrator programs are quite possible. Essentially, PIE can produce a good pie chart without any user participation in the graphic process. Based on our experience with PIE, we believe that bar charts and curve graphs can also be produced in a more or less automatic fashion. 2. Word and number labeling (because of its unpredictable length) is a serious problem for automatic illustrators, and as of now there appears to be no simple solution. 3. Graphic execution time saved in PIE-like illustrators is enormousmuch more than in SIGN or BLOCK. 4. Creating and editing tabular data is in itself a graphically idiomatic process (quite aside from its application) and from our experience with PIE looks like a pregnant area for future research. 5. Conclusions These conclusions are an attempt to summarize our research findings in relation to BLOCK, SIGN, and PIE. We hope these conclusions will be helpful to others involved in the design of interactive picture-making systems. On Methodology We did not adopt the more common approach of specifying and implementing a graphics system and then writing the application programs. We rather scrounged whatever graphics capability was available (SMALLTALK) and began by simulating the illustrator's habit of building up a `graphics language' as we worked. We were able to do this because SMALLTALK already contained a rich set of graphics primitives. We began our investigation with three of the simplest and most commonly used idioms. Our reasons for this decision were twofold: first, about a dozen simple, well-known conventional idioms account for the bulk of technical graphics used in the working world, and secondly, it allowed us to concentrate on user issues such as command languages rather than on system issues that arise when dealing with complex pictorial representation. This approach drove out two insights that we might have missed had we adopted the more conventional approach that involves the development of a graphics system and then the design and implementation of the application programs. The insights are: (1) a very simple set of programming tools is sufficient for the development of most graphical idioms for general office use and (2) the user requirements in applications where the presentation is 2-dimensional and dynamic are much more subtle and complex than we had imagined. We found ourselves designing form `processes' rather than form `products' through which to create pictures. Picture creation takes place in a human time continuum and the `rhythm' of visualization is as important as the availability of form options. On Resources As mentioned above, one does not need much in the way of a graphics system to write useful application programs. The SMALLTALK picture manipulation and drawing primitives are quite sufficient. These include and enable: 1. rectangles, points and gridsSPATIAL GRAMMAR 2. lines of up to seven thicknessesFORM VOCABULARY 3. text stringsLITERAL IDENTIFICATION 4. turtle delineationGRAPHIC STATEMENT For a complete description of this system, please refer to the SMALLTALK manual. On Project Results We feel that this project was a success in that we have demonstrated that it is possible to combine conventional graphic idioms and current computer technology to make it possible for ordinary (graphically unskilled) people to create articulate graphical statements. This has been demonstrated by various utilizations of the BLOCK program within the PARC community involving the creation of block diagrams. The simple compositional help that is offered by BLOCK greatly enhanced the aesthetic character of the user diagrams. The piechart program offers a powerful `machine tool' for the person who wants to represent tabular data in visual form without having to actively engage in the techniques of technical illustration, or in this case, decisions of label placement. Evidence of SIGN's utility can be found in PARC videotapes and on many PARC bulletin boards. On Future Research We have in the scope of this project only scratched the surface of the idiomatic illustrator concept. There remain many modifications to explore with BLOCK, PIE, and SIGN. For example, can one make the stages in picture specification like block-out and touch-up more explicit? This would offer the non-professional user even more help during the creation of his/her illustration. There also remain a host of other idioms to explore, such as barcharts, curve graphs, plans, maps, volumetric representations, etc. We believe an understanding of commonly understood graphic communication idioms in the context of a display-based interactive computing system will have large payoffs in office information systems of the future. For such systems (idiomatic illustrators) to be really useful they need to be integrated with a system that includes text. Research in this area is currently underway at PARC (Master-maker Project). Projecting even further into the future, text/graphics systems should allow for the personalization of graphic programs so that professionals in the fields of graphic design and illustration can incorporate the computer as an effective medium for visual communication. Bibliography Anderson, Donald. The Art of Written Forms. Holt, Rinehart, and Winston, New York, 1969. Bertin, Jacques. Semiologie Graphique. Mouton, Paris, 1967. Bowman, William. Graphic Communication. John Wiley and Sons, New York, 1968. Kepes, Gyorgy. Sign, Image, Symbol. George Braziller, New York, 1966. , Personal Dynamic Media. Learning Research Group, Xerox Corporation, Palo Alto Research Center, Palo Alto, CA, 1975.