<> <> <> <> <> STONE COLOR, GRAPHIC DESIGN, AND COMPUTER SYSTEMS SIGGRAPH'87 TUTORIAL COURSE NOTES DOCUMENTATION GRAPHICS Color, Graphic Design, and Computer Systems [Republished from Color Research and Application, June 1986, John Wiley and Sons, New York; Proceedings of the 1986 Interim Meeting of the Association Internationale de la Couluer (AIC), Color in Computer Generated Displays, June 19-20, Toronto, Canada. Unfortunately, the color figures cannot be reproduced in these notes. The interested reader is directed to the journal article.] Color, Graphic Design, and Computer Systems Maureen Stone Xerox Palo Alto Research Center The graphic designer of the future will work with a color display as a workbench and a powerful, interactive computer system as a toolbox. This prediction is based on the wide use of computers for text processing and production layout. Just as word processing systems have replaced typewriters, computer-based design systems will replace pen, knife and light-table for certain types of graphic designs. Included in this vision of the future are tools to manipulate color. This paper will examine some of the issues associated with color in such an environment. Introduction Our goal in the Imaging Research Group of the Computer Science Laboratory at the Xerox Palo Alto Research Center is to understand how to control color in our computing environment. This computing environment consists of powerful, personal workstations connected via a high-speed digital network to each other and to a variety of hardcopy output devices, specifically, digital printers and film recorders. Because there are many uses for color in this environment, no one technique will support all applications. We need to provide a framework which supports a variety of different methods for using color. The principal application discussed in this paper is computer aided graphic design. The designer uses interactive tools to form a picture on the workstation display, then sends a printable representation of the design to one of the output devices. The monitor never gives exactly the same appearance as the printer, but a good approximation minimizes the number of passes through the design/print/look/adjust cycle. Some designs are not directed towards a specific output device but are intended for use throughout the environment. Then the workstation monitor is no longer a ``proof copy'' of a specific printer but one of a family of media, each of which has its own capabilities, such as color gamut and resolution. For this problem, it is important to define methods that make it possible to represent and render a design effectively across a range of devices, a concept we call device independence. Let us take some examples of the kind of designs that are done here and what the color issues are. ==================== Figure 1: Lilacs, Figure 2: Xerox Spiral ==================== Figures 1 and 2 show designs containing regions of flat, filled color. The lilac picture was designed with an interactive illustration program; the spiral by mathematical transformations of the Xerox logo font. The initial colors were selected by name from a color palette. Once a test print was run, the colors were adjusted using a color order system. If the color names, color selection and proofing systems model the printer accurately, this type of design is straightforward. If they do not, it may take many adjustments to achieve a satisfactory result. At the time these images were designed, we did not have a good model of the color printer. Both the pale yellow-green for the background and the dark colors in the Xerox spiral were difficult to control. The foreground pattern in figure 3, the dragon head, flame and border, was designed with an interactive illustrator. Using a simple program, this pattern was repeated on a diagonal grid to form a watermark. Each element of this design is colored with a smoothly varying hue. The dragon's flame and border color is a gradation from dark to medium reddish-brown. The watermark color interpolates from green to blue, and the background is similar to the foreground. The designer achieved these affects by working with the three color separations directly. He built a pattern that generated an intensity gradation from light to dark with variable extremes and rates of change. Overlapping these patterns in different separations creates a wide range of interesting effects. The foreground pattern, for example, involved gradations in the magenta and yellow separations. The watermark colors involve the cyan and yellow separations. To choose his colors, the designer created a set of samples showing different pairs of toners at different screen densities similar to the Pantonẻ book for offset color printing. This gave him samples of points along the gradation. He used this to set the initial values for the gradations and did the final adjustments by trial and error. ==================== Figure 3: Dragon poster ==================== This design illustrates the issue of specifying and reproducing a continuously changing color. The specification of the color gradations here was tied directly to the mechanics of the color separation process. However, an extension of this technique would allow a designer to specify a controlled change in any color, leaving the mechanics of color separation to the computer system. Using a computer model of a color space, we can vary a wide range of parameters to achieve interesting effects. ==================== << [Artwork node; type 'ArtworkInterpress on' to command tool] >> Figure 4: a) Flow diagram in color << [Artwork node; type 'ArtworkInterpress on' to command tool] >> Figure 4: b) Flow diagram with grays. << [Artwork node; type 'ArtworkInterpress on' to command tool] >> Figure 4: c) Flow diagram with black and outline. ==================== Figure 4 shows a different set of design considerations relating to color. Here the color is used to code two different paths through the flow diagram as shown in figure 4a. The specific red and blue are not too important except that they should have similar impact; that is, the two paths should appear equally important. They also need to be dark enough that the shape of the arrows are clearly distinguishable. Figures 4b and 4c show the same diagram rendered for a monochrome device. In figure 4b, each color is represented by a gray value based on the brightness of the original color. While this approximation works well for black and white images of scenes, as in black and white photographs, in this illustration the difference between the two paths is lost. Furthermore, gray has already been used as a design element in the borders and arrows in another section of the picture. Figure 4c shows the illustration redesigned for a monochrome rendering, using black and outlined arrows to distinguish the two paths. To solve this type of problem, the designer needs to specify the intended purpose of the color at a more abstract level than the actual hues. Our approach to this problem is graphical style, which we define as representing the rendering characteristics in such an illustration as a set of names and rules for rendering [1]. In the previous example, the ``color'' of the two paths would be names such as ``HighlightColor-1'' and ``HighlightColor-2''. The rules for rendering these colors would be different for the monochrome device and the color device. The collection of all rules for rendering an image is called its style and is given a particular name. A set of related figures would use the same style to ensure consistency. Our intent is to capture the design decisions needed to provide a good rendering independent of the content of the illustration. This paper describes work in progress towards solving some of the problems identified in these examples. The remainder of the paper is in roughly two parts. First is a description of the characteristics of our color monitors and printers, including a discussion on calibration and standards. Following that is a description of the tools and capabilities for color selection that can be offered on a computer driven workstation. The conclusion will present a framework for addressing the problems of color specification and reproduction in this environment. Types of color devices A color monitor produces colors when the electron beam stimulates red, green and blue phosphor dots to produce a pattern of red, green and blue luminous spots. The brightness of the dots is a function of the power supplied by the electron beam. The color system described is additive and the three primaries are independent [5]. A color for a monitor, therefore, is specified as a red, green, blue triple, hereafter called an RGB value. A monitor must be properly adjusted so that the baseline and gain for the three guns are balanced to give a neutral gray scale for equal values of red, green and blue throughout the range of black to white. The natural relationship between input and luminance is not linear, but for the purposes of this paper we will assume that the colors have been compensated to remove this non-linearity [4]. A typical monitor can generate on the order of 16 million colors, and a good graphic design system would give the user control over the entire gamut. Due to cost or other limitations in the computer hardware, however, the specific palette of colors available to a design system may be limited to 256 or less. Such a system typically uses a table called a colormap to map a number between 0 and 255 to an RGB value. Therefore, while only 256 colors are available, they can be selected from the complete gamut of 16 million. This limitation means color approximation may be an issue for a color design system. The usual solution to this problem is a technique called dithering, which simulates colors with color patterns [6, 9]. Our experience has shown that most of the color gamut can be adequately simulated with patterns of 128 colors, leaving 128 for the cases where the approximation is inadequate. A digital color printer produces color by overlaying spots of magenta, cyan, yellow and (optionally) black inks on paper. The spots are all the same size as defined by the resolution of the printer. Printer resolutions range from 120 spots per inch for low cost ink-jet to 1200 spi for phototypeset quality color separations. Intensity variation is achieved by the use of patterns called halftones, as in conventional offset printing [13]. The technique of halftoning trades spatial resolution for intensity levels as shown in figure 5. Commercial printing typically uses halftone frequencies of 80 to 150 lines per inch. To simulate this on a digital printer we need printer resolutions of approximately 600 to 1200 lines per inch. Many color digital printers do not have sufficient spatial resolution to simulate even an 80 line screen. The designer must balance, therefore, the problems of seeing contours in shaded regions against the fuzziness and texturing caused by using a coarse screen. ==================== << [Artwork node; type 'ArtworkInterpress on' to command tool] >> Figure 5: Digital simulation of halftone patterns. ==================== A typical printer is neither linear nor independent with respect to its primary colors. A designer may compensate for this by using a set of color swatches. The number of samples needed to adequately cover the color space, however, is very large. One benefit of a computer controlled environment is that it is relatively easy to algorithmically define a set of test patterns which explore the region of the color space of interest. A better approach is to develop methods for converting colors specified in the well behaved RGB color space to a set of halftone patterns on the printer. The results of this work are elsewhere [12]. Standards and Calibration As indicated in the introduction, it is important to be able to create designs that can be effectively displayed or printed on all of the monitors and printers in the system. To do so implies developing a standard representation of color and calibrating all the devices to it. Our goal is to develop this standard, calibrated, if possible, to an international standard such as the system for computing tristimulus values recommended by the CIE (Commission Internationale de l'Eclairage) . Given a set of parameters that describe the phosphors and the voltage/intensity transfer curves, it is relatively straightforward to calibrate a color monitor with respect to the CIE coordinate system such that is possible to convert an RGB triple to a tristimulus value, XYZ, and vice versa, within the gamut of the monitor [5]. It is also possible, given the illuminant, to measure the CIE tristimulus values for samples of inks and papers, and therefore to compute the tristimulus value of any region of paper covered by a simple halftone pattern. A simple halftone pattern is one involving only patches of opaque ink and bare paper. If two inks overlap, they must overlap completely and the tristimulus values for the combination must be measured independently. That is, these tristimulus values cannot be derived from the coordinates of the two inks. For more complex cases, and unfortunately, most useful cases are more complex, things are not so well defined. Again, the work of converting from a well behaved system like RGB to the printer is crucial. The work reported by Starkweather [12] shows that it may be possible to develop a color standard based on red, green, blue primaries calibrated with respect to the CIE standard. Conversion of a point in color space between any two monitors would be an algebraic transformation. Each printer could be modeled and calibrated with respect to the standard. An additional feature would that there is yet another linear transformation that extracts the luminance information from the model; that is, the calculation for the tristimulus value Y. As in the NTSC broadcast television standard [7], Y can be used to produce an acceptable gray approximation of a color on a monochrome device, such as a grayscale monitor or a black and white laser printer. For any color specified in this standard, the actual rendering on a device will be an approximation of that color. There are a number of reasons why this is so. The lighting conditions typically will not be precisely controlled. Physical devices have limited gamuts and the colors within the gamuts are quantized, sometimes dramatically. Tradeoffs must be made between speed and accuracy. To be effective, the color specification must include some information from the designer which makes it possible to produce the best approximation of that color for that design. It also seems clear that any form of device independence, meaning a single specification of a color that defines it for all devices, will only work within a range of color values and quality levels. Only as different applications of color in design environments mature will it be possible to see how the definition of device independence evolves. Color selection The novel aspect of the computer driven color display, compared to other graphic arts media, is that it allows a designer to adjust and view colors in real time. This is different from any other color media currently used in the graphics arts industry. Color can be adjusted with buttons, sliders or knobs which control parameters in a color space. The computer also provides a mechanism for specifying colors and sets of colors in a manner that captures the designer's purpose for choosing a color. It allows the designer to specify interrelationships between colors in a design, using the power of the computer to maintain the relationships while the designer adjusts different aspects of the design. This section will first explore the models and tools available for selecting a single color, and then will examine some of the issues related to coordinating all the colors in a design. Color models The tools for selecting a single color consist of a color model and the controls defined and provided by the model. We can classify color models as device oriented or perceptually oriented, calibrated or uncalibrated. Device dependent models An example of a device dependent model is the cube defined by the red, green, and blue primaries of a color monitor. A tool based on this model provides the designer with three interactive controls that indicate the amount of each primary to add to the color. A color patch displays the color currently defined by the controls. Such a tool allows a designer to systematically explore the color gamut for a monitor and select the colors desired. Such a tool is simple to implement and is available on most computer systems that support color displays. While the user interface often presents a continuous set of values for each primary, most implementations are quantized to 256 values. Assuming that the monitor has been correctly compensated, the values are assumed to be linearly spaced in brightness. The three subtractive primaries, cyan, magenta and yellow, define a cube that is the inverse of the RGB system described above. That is, the cyan value equals (1 minus the red value, magenta equals (1 minus green) and yellow equals (1 minus blue). Most designers familiar with print media are familiar with this system and usually specify the colors as percentages of screened ink in each color separation. Since the values are specified considering the mechanics of halftoning, they are assumed to be linearly spaced in reflectance. If a black separation is available, control must be supplied for it also. The black control is interesting in that it is not necessarily independent of the three primaries because equal amounts of cyan, magenta and yellow also produce black. Undercolor removal is a technique for replacing some fraction of the ``black'' in dark colors with black ink. If the designer specified 10% undercolor removal, for example, 10% of the value of MIN[C,M,Y] should automatically be added to the black value and subtracted from the primaries. Perceptually oriented models While the base representations for our hardware are RGB and CMY, using these systems for color control requires significant training. There have been several efforts in the field of computer graphics to develop models that 1) require less training and 2) can be mapped efficiently onto the RGB gamut. Two are common in computer systems today. They are called HSV, for hue, saturation and value, and HSL, for hue, saturation and lightness [11, 6]. The names of the primaries in these systems are rather confusing, especially since there are alternate definitions of these terms in the color literature. Hue is the least ambiguous and is theoretically represented by a circle with the spectral colors arranged in order around it. To map these models efficiently to the RGB color cube, these systems are defined as having hexagonal cross sections with red, yellow, green, cyan, blue and magenta positioned at each of the vertices. The HSV model is shaped like a hexagonal cone with white at the center of the base and black at the tip. The HSL model is shaped like a double ended hexagonal cone with white at one tip and black at the other. In both models, the fully powered primaries and secondaries are on the edge of the widest part of the cone and the axis of the cone is achromatic. In both models, the saturation coordinate defines the distance towards the achromatic axis. The value or lightness coordinate defines the distance along the axis of the cone. The HSL model is similar to the Munsell [10] system. Lightness corresponds to Munsell value, saturation to Munsell chroma. The principal difference is that the colors are not perceptually spaced but spaced to map easily to the RGB color cube. The brightest yellow and the brightest blue, for example, have the same HSL lightness coordinate although they have very different Munsell values. The HSV model is easiest to use by thinking of the saturation and value coordinates as controlling the amount of white and the amount of black in a color. The ``black'' control is stronger than the white; adding white (decreasing the saturation) will only make the color white if the value is 1.0, otherwise it makes the color gray, whereas adding black (decreasing the value) always makes the color black. Another approach to color selection has been to standardize a set of color names similar to the NBS Universal Color Language [8, 2, 3]. Our implementation consists of seven color names: red, orange, yellow, green, blue, purple and brown with 3 steps of interpolation between them. i.e., green, bluish-green, green-blue, greenish-blue and blue. There are five lightness levels, from light to dark, and four saturation levels, from grayish to vivid. The user can specify a name using buttons, or simply type in the color and adjectives. While not strictly speaking a color model, the description is included here because the implementation currently maps the names directly to the coordinates of the HSL color model. These systems have been combined into an interactive tool that allows the user to adjust the color of a patch by changing values in any of a number of ways, thereby developing an intuition for the relationship between them. All of these models, even CMY, are defined with respect to the RGB color gamut of the specific monitor that is running the tool. The systems are uncalibrated, so different monitors produce different colors. Specific colors, however, can be calibrated with respect to an external standard if the monitor is calibrated. Calibrated models A tool for color selection in calibrated systems must provide the user with some feedback about the gamut limitations for a particular device. There are a number of design situations where this is an issue. The most common is in using the monitor to select colors for a specific printer. Given an adequate model of the color characteristics of the target device, the problem reduces to constraining the color selection tools such that it is impossible to specify colors outside the gamut of the target device. If the gamut of the target device does not lie entirely inside the gamut of the monitor some sort of feedback must be provided to inform the user that the color displayed does not match the color specified. The user still has a choice of using a model that is derived from the color space of the device, such as CMY for a printer, or using any other system that can be mapped to both the monitor and the printer models. Note that to provide ``real time'' interaction, it is necessary to have an efficient conversion algorithm, one that can produce color value for the monitor in less than a quarter of a second. We have adequate models for this type of interaction for many of our color printers, all our different color monitors, and the NTSC broadcast standard used for videotapes. It is important to remember that in this discussion we are considering only the issue of gamut and quantization, not other color restrictions such as the those on the rate of change of color between adjacent pixels in a videotape or the texturing caused by halftone patterns. We have also experimented with the inverse problem; that is, allowing a user to explore a model like the space formed by the XYZ tristimulus values. These values do not define a gamut in the context of a particular device such as a color monitor. The tristimulus values XYZ are mapped into x,y,Y to provide a more familiar representation. The familiar chromaticity diagram is used as the tool for controlling the values of x and y. The monitor's gamut is defined as a triangle in the x,y plane and only the region inside the triangle is active, automatically constraining the values of x and y. Luminance is controlled separately. Color selection is restricted to the gamut of the monitor by constraining luminance. The user can explore the surface of the gamut by setting Y to track the maximum value for each point x,y. The user can explore the interior of the gamut by disabling the tracking feature. In this mode, our current implementation adjusts Y only if it is necessary to stay inside the gamut. An interesting extension would be to allow Y to constrain x,y, making it easy to trace regions of equal Y. Color design A designer sits uses the monitor like a sheet of paper or a light table. The colortools described provide a mechanism for mixing a color and copying it to parts of the design. Since color perception is affected dramatically by surrounding colors, it is crucial to actually integrate the color selection mechanisms with the design system so that colors can be adjusted in place. An interesting extension of this is to create a tool that makes it possible to adjust a set of colors simultaneously. For any of the color models described, it would be straightforward to take a set of colors, extract the red, saturation, Y, or whatever components, and change them with a slider or knob. The question is, what color models do we really want to use to adjust a set of colors? There are a set of design issues having to do with color control precision. Unless constraints are applied, it is easy to use the color tool to generate a set of similar colors that are actually intended to be the same. We have found that a level of indirection, a color palette, is a useful method of organizing colors for a design or a set of designs. The designer uses the color tool to select a set of colors and name them. In the context of a specific design, it is possible either to adjust the value assigned to the color name, in which case all the areas colored by that name change, or it is possible to adjust a single region and assign a new name. An interesting question arises about how to represent the value of the new color. It can be defined as a completely independent color value or it can be defined in terms of the modification from the old value. For example, imagine a color named ``dark red'' and define it as 30% of a red primary. Create a new color called ``darker red''. Should it be defined as 25% of the red primary or as 5% darker than ``dark red?'' Graphical Style The example given in the introduction defined graphical style as a method for representing the rendering characteristics in an illustration as a set of names and rules. In the example colors were named by their function, such as ``HighlightColor-1.'' There is a wealth of interesting research issues surrounding this type of specification. The first step has been hinted at in the example above of ``dark red'' and ``darker red.'' This suggests that a designer should be able to specify colors by their relationship to other colors in the design. It is possible to imagine writing rules involving such perceptual terms as contrast, hue, lightness, etc., that would define a set of colors that could be adjusted in a manner that could really be called ``style''. For example, a design could be rendered in cool tones or warm ones, conservative colors or trendy ones. Graphical style is crucial when specifying designs to be used on a range of devices and for a range of purposes. Style rules can be used to encode a designer's choice for constraining a design to the limitations of a particular device. By extending the definition of color to include textures, even devices that have no color as conventionally defined can be accommodated. Graphical style can also be used to encode quality levels and to define the best compromises in terms of different costs. Conclusions This paper has described what is believed to be a prototype of future design environments. The designer works with a computer system as a workbench and must consider targeting a design towards multiple devices. A framework was presented for addressing the problems of color specification and reproduction in such an environment. It is a crucial first step to understand and calibrate all the devices in the system. Even if it isn't important to adhere to a standard as precise as the CIE system, it is important to be able to model the gamut, transfer functions and quantization effects for each device. This will enable making the monitor ``look like'' a particular device. The designer can then manipulate colors for the device in a device dependent manner and also use the monitor to proof the results for a particular device. As it typically takes much longer to print an image than display it, a quick proofing capability can save a great deal of time. The device dependent model is not sufficient for designs that are intended to be used across multiple devices. For many purposes, however, a device independent set of color names will provide an adequate interchange format. Each device can implement its ``best effort'' towards reproducing the named colors. As we experiment with this approach, it will be interesting to see if it is more important that all the named colors match a standard color or that all the named colors on a particular device maintain a standard relationship. The limitation of color names is that for certain designs the named colors do not contain the correct colors. Some designers will always want to adjust them. If the design is targeted for a particular device, a device dependent set of controls will be adequate. If the design is intended to be device independent, it is necessary to develop a device independent model for adjusting colors. The standard RGB outlined in the section on standards is one example of such a model. Again, it will be interesting to discover what types of parameters are most important to someone adjusting the colors in a design. The real solution to device independence lies in developing better ways of capturing the designer's purpose for using a color, as outlined in the section on graphical style. Acknowledgments This paper describes an environment inhabited by many people, specifically the Imaging Research Group which spreads across the Computer Science and Exploratory Development laboratories of the Xerox Palo Alto Research Center. We all have goals and visions with respect to color. I would like to thank William Cowan and Dusty Rhodes for answering my questions about color perception. Bibliography [1] Beach, R. J. and Stone, M. C. ``Graphical StyleTowards High Quality Illlustrations.'' Computer Graphics, 17, 3, 127-139, (uly 1983. [2] Berk, T., Brownston, L. & Kaufman, A. ``A New Color-Naming System for Graphics Languages.'' IEEE Computer Graphics and Applications, 2, 3, May 1982. [3] Berk, T., Brownston, L. & Kaufman, A. ``A Human Factors Study of Color Notation Systems for Computer Graphics.'' Comm. ACM, 25, 8, 547-550, August 1982. [4] Catmull, E. ``A Tutorial on Compensation Tables.'' Computer Graphics, 13, 2, 1-7, August 1979. [5] Cowan, W. B. ``An Inexpensive Scheme for Calibration of a Colour Monitor in Terms of CIE Standard Coordinates,'' Computer Graphics, 17, 3, 315-321, July 1983. [6] Foley, J. D. & van Dam, A. Fundamentals of Interactive Computer Graphics. Addison-Wesley, Reading, PA, 1982. [7] Hunt, R. W. G. The Reproduction of Colour, Fountain Press, London, 1975. [8] Kelly, K. L. and Judd, D. B. Color, Universal Language and Dictionary of Names. National Bureau of Standards (U.S.), Special Publication 440, Washington, December 1976. [9] Mulligan, J. B. ``Minimizing Quantization Errors in Digitally Controlled CRT Displays,'' Color Research and Application, 11, S42-S46, June 1986. [10] Munsell,A. H. Munsell Book of Color, Glossy Finish Collection with Additional 80 Hues. Macbeth Division of the Kollmorgen Corp., Baltimore, MD, 1976. [11] Smith,A. R. ``Color Gamut Transform Pairs.'' Computer Graphics, 12, 3, 12-19, August 1978. [12] Starkweather, G. K. ``A Color Correction Scheme for Color Electronic Printers,'' Color Research and Application, 11, S62-S66, June 1986. [13] Yule, J. A. C, Principles of Color Reproduction, John Wiley & Sons, New York, 1967.