A document communicates information both textually and pictorially. An author collects, organizes, and presents the information in a document. The author's purpose for the document may be to inform, to persuade or argue, or perhaps simply to entertain. This thesis deals with the problems of producing and reproducing documents for effective communication of their information.

Historically, documents were handwritten manuscripts reproduced labouriously by scribes. Many of these early documents were handsomely illustrated with hand-drawn sketches or pressings from engraved images. Early printing technology using movable type elements reproduced the handwritten documents with less labour while still striving for the quality of the scribes.

The traditional skills of reproducing modern documents are called the graphic arts. These crafts represent a high standard of reproduction quality and readability. The artistry of letter forms, the inclusion of photographs and line drawings, and the uniformity of the printing process are all testaments to the quality of graphic arts techniques. The traditional process is labour intensive and often time consuming. While electronic tools are being introduced into these processes, the standards and the skills remain largely traditional.

Authors assemble the text, reference material, and illustrations of a document and compile a manuscript. The graphic arts staff transform manuscripts into books, reports, pamphlets, posters, newsletters, and so on. The process is designed for the writer to guide the reader in the exploration of the information. Traditional printed documents are fixed, static, and unchanging with new versions being provided when the information changes. Devices like tables of contents, indices, and cross references are necessary to provide readers with some control over their reading path through a large document.

While electronic tools are being applied to the traditional graphic arts process, the exciting prospects lie not in producing the same document faster but rather in producing a different document. Electronic documents contain a corpus of knowledge communicated by words and images but examined by a reader who guides his own exploration. Such documents are read in electronic form on video displays or in printed form produced on demand from laser printers. Both forms are now striving to meet the reader's expectations of the same high level of graphic arts skills. These documents can become active rather than passive objects and respond to queries from the reader. The vision of Engelbart's NLS and Augment [Engelbart] and Nelson and van Dam's Hypertext [Nelson][van Dam] from the late 1960's represent seminal experiments with these ideas.

Modern documents contain a wide variety of material, some more easily presented than others. Documents now contain more visual material, perhaps due to a variety of societal influences including television [reference needed?]. Technical and scholarly documents include textual material, possibly in foreign languages, notation such as mathematical or chemical formulae, tabular presentations of data, photographs, line drawings, possibly with shaded and coloured elements. This thesis concentrates on techniques and tools for producing electronic documents, with mechanisms that simplify presenting this wide variety of information while ensuring a full gamut of choices.

To understand the process of composing documents better, this thesis surveys the traditional graphic arts and electronic techniques for producing documents. Such a survey helps to explain the complexities of achieving our expectations of quality and to highlight where difficulties and solutions remain to be found.

The author of this thesis has considerable experience producing scholarly books and journal articles to meet traditional graphic arts standards. This involves typesetting scholarly works for traditional publishers but utilizing electronic composition tools. Production difficulties with these electronic composition tools has helped to focus this research. Despite the benefits of such tools, many problems and difficulties arise which have to be circumvented due to deadline and economic pressures.

Some significant benefits to document composition problems accrue from the use of these electronic tools and techniques.

Large volumes of textual material are easily accommodated on computer file systems, especially systems with archiving facilities. The management of such large documents becomes easier with electronic aids. Subsequent editions and versions of manuscripts can be created from these archived files. An introductory text on computing that used the WATFIV-S programming language [WATFIV-S] was prepared and typeset from computer files. Later, the book was substantially revised to use two other programming languages, first PASCAL [PASCAL] and then FORTRAN77 [FORTRAN77]. The computer files of one manuscript were methodically reviewed and mechanical changes made to produce the preliminary version of the next book.

Editorial consistency can be achieved more easily using the facilities of word processing or text editing programs than by manual proofreading. The organization of documents into chapters, sections, paragraphs, tables, figure captions, and reference citations can be regulated more easily by coding tags or macro calls into the computer files. Families of documents, such as the three books mentioned previously or articles in a journal, can all share a similar design and layout by using the same composition program and design parameters. With sufficient care and access to the right equipment, good quality typography can be achieved that uses well designed typefaces and line breaking algorithms.

Some complex material can be accommodated within existing composition systems. Mathematical notation, tables, and simple illustrations can be included with these tools, although they may require a considerable time investment and may not produce everything you desire.

The accurate presentation of computer generated data and computer programs has been a significant improvement over traditional graphic arts techniques. Manual transcription of data, and misinterpretation of the unusual appearance of computer programs lead to inevitable errors. Typographers were experienced in transforming typewritten manuscripts into typeset books, but they did not appreciate that authors expected computer programs to appear in the published form as they appeared in the manuscript form.

Monospaced fonts for typewritten text were very rare on typesetting equipment, and the spacing rules, the use of quotation marks and apostrophes, and the alignment of text for computer programs and data were unabashedly ugly by traditional typographical standards. Yet the accurate treatment of these small details made possible by electronic document composition tools is crucial to documents in the computer science community. Nonetheless, several difficulties remain in the use of electronic composition tools.

Mathematical notation is still hard. Math typesetting tools like eqn and TEX provide abstract languages for an author to describe his math notation. However there are few interactive `what you see is what you get' (WYSIWYG) editing tools for mathematics, mainly commercial typesetting systems, and fewer still that understand any mathematical concepts. Accommodating new notation is a major limitation of these systems, since authors frequently invent new notations to serve their purposes and these new notations prove difficulty to incorporate.

Math notation is treated apart from normal text in most systems, thus limiting its use in all places of a document. For example, it is sometimes awkward to include math in chapter or section headings where the text is reused for running heads. Similarly, it is awkward to include math in figure captions where the size of type is different, in the table of contents where headings have been automatically saved, and in index entries where phrases have been automatically collected. A uniform document structure model that integrates document objects like text, math, and illustrations is necessary to alleviate this problem.

Tables of information are awkward to compose. Each table tends to be treated as a separate design problem. Table formatting tools are less well developed than text formatting tools, and as a result, many special table features are not provided. The content of tables may be restricted by the document structure so that math notation or illustrations may not be possible as table entries. Since tables are treated differently than text or math, it may be awkward to use the same macro packages for tables as for the textual parts of a document. Simple tables, especially spreadsheets or tables of mathematical data, are frequently formatted by special purpose programs making such tables difficult to include with the rest of a document.

Illustrations remain outside the main stream of document formatting. The current illustration packages available either produce crude results by graphic arts standards, or are limited in the range of illustrations possible. The wide range of illustration necessary includes sketches, engineering drawings, shaded or continuous tone images, graphical highlights on tables, math or textual material, and colour.

Pagination of large documents is another difficulty. Since the documents are large, automated techniques are desirable. Since the existing algorithms tend to produce unpleasant results in complicated situations, especially those involving figure placement and footnotes, special cases have to be handled manually.

A contributing factor in the pagination problem is the tendency to batch process the document formatting. Some systems are noninteractive and the processing is actually run a batch at a time, typically with a batch for each chapter. However even for interactive systems there is another implication, that of the `pregnant pause' where reproduction quality output is delayed until the moment when everything has been completely and finally formatted. Draft cycles in preparing a document often involve reprocessing much of what has not changed, thereby wasting resources and introducing delays. This reprocessing is necessary to get automatic numbering correctly sequenced, to correct cross references, to prepare index entries with the correct page references, and so on. The drafts are each of better quality than might be had in traditional systems, but there is no accumulation of partial results and there is a lack of confidence that the final output will eventually appear (something wary publishers worry about). This tendency to output everything at once at the final moment places considerable stress on the capacity of the formatting and output systems to handle the surge of load from the completed document. Yet last minute touch-ups are still necessary to correct overlooked mistakes or to include illustrations not produced with the system.

Electronic aids for document production have successfully contributed to document style. The notion of separating form from content in a document, formalized by Scribe [Reid] and other macro packages [-ms], is a crucial insight. This provides a classical split between logical versus physical representations of document content. Styles deal with the issues of appearance, aesthetics, and understandability in documents. For textual material, the practice is called typography:

Capturing a sufficiently large set of restrictions, or instructions for formatting documents, we can claim to have a document style. What to include in a style is simple to state:

The concept of document style will help to extend our control from the appearance of textual material to the appearance of illustrations and tables.

Illustrations may be viewed in a way that highlights their similarity to logical document structure. Illustrations are composed of graphical objects, such as captions, axes of a graph, curves on a graph, numeric and textual labels, dimension lines, shaded areas in bar charts, and so on. If these objects are identified and associated with style attributes, then the same mechanisms for document style can be extended to illustrations.

Tables also have a logical structure based mainly on the arrangement of boxes. This arrangement is aligned on a grid system, similar to grids used in graphic design for document layout. Grids provide a design discipline for creative expression, effectively bringing order out of chaos. The logical structure of table elements into rows and columns allows one to associate style attributes with some part of the structure, such as an element, a row, a set of rows, etc. This is similar to providing style attributes for the logical parts of a document.

The table formatting problem can serve as a concentrated focus for other document composition problems. When a table is viewed `in the small', then math notation alignment can be treated as a special table formatting problem with similar alignment requirements. When a table is viewed `in the large', then page layout can be treated as a large table formatting problem where material flows from one part of the table to the next. Hence tables represent two classes of difficulties, math formatting and pagination, and solutions suitable for table layout may be extendable to these areas also.