% This program is copyright (C) 1982 by D. E. Knuth; all rights are reserved. % Copying of this file is authorized only if (1) you are D. E. Knuth, or if % (2) you make absolutely no changes to your copy. (The WEB system provides % for alterations via an auxiliary file; the master file should stay intact.) % See Appendix H of the WEB manual for hints on how to install this program. % And see Appendix A of the TRIP manual for details about how to validate it. % Version 0 was released in September 1982 after it passed a variety of tests. % Version 1 was released in November 1983 after thorough testing. % Version 1.1 fixed ``disappearing font identifiers'' et alia (July 1984). % Version 1.2 allows `0' in response to an error, et alia (October 1984). % Version 1.3 made memory allocation more flexible and local (November 1984). % A reward of $5.12 will be paid to the first finder of any remaining bug. % Although considerable effort has been expended to make the TeX program % correct and reliable, no warranty is implied; the author disclaims any % obligation or liability for damages, including but not limited to % special, indirect, or consequential damages arising out of or in % connection with the use or performance of this software. This work has % been a ``labor of love'' and the author hopes that users enjoy it. % Here is TeX material that gets inserted after \input webmac \def\hang{\hangindent 3em\noindent\ignorespaces} \def\textindent#1{\hangindent2.5em\noindent\hbox to2.5em{\hss#1 }\ignorespaces} \font\ninerm=amr9 \let\mc=\ninerm % medium caps for names like PASCAL \def\PASCAL{{\mc PASCAL}} \def\ph{\hbox{\mc PASCAL-H}} \font\logo=manfnt % font used for the METAFONT logo \def\MF{{\logo META}\-{\logo FONT}} \def\<#1>{$\langle#1\rangle$} \def\section{\mathhexbox278} \def\(#1){} % this is used to make section names sort themselves better \def\9#1{} % this is used for sort keys in the index via @@:sort key}{entry@@> \def\drop{\kern-.1667em\lower.5ex\hbox{E}\kern-.125em} % middle of TeX \catcode`E=13 \uppercase{\def E{e}} \def\\#1{\hbox{\let E=\drop\it#1\/\kern.05em}} % italic type for identifiers \outer\def\N#1. \[#2]#3.{\MN#1.\vfil\eject % begin starred section \def\rhead{PART #2:\uppercase{#3}} % define running headline \message{*\modno} % progress report \edef\next{\write\cont{\Z{\?#2]#3}{\modno}{\the\pageno}}}\next \ifon\startsection{\bf\ignorespaces#3.\quad}\ignorespaces} \let\?=\relax % we want to be able to \write a \? \def\title{\TeX82} \def\topofcontents{\hsize 5.5in \vglue 0pt plus 1fil minus 1.5in \def\?##1]{\hbox to 1in{\hfil##1.\ }} } \def\botofcontents{\vskip 0pt plus 1fil minus 1.5in} \pageno=3 \def\glob{13} % this should be the section number of "<Global...>" \def\gglob{20, 26} % this should be the next two sections of "<Global...>" @* \[1] Introduction. This is \TeX, a document compiler intended to produce high-quality typesetting. The \PASCAL\ program that follows is the definition of \TeX82, a standard @:PASCAL}{\PASCAL@> @!@:TeX82}{\TeX82@> version of \TeX\ that is designed to be highly portable so that identical output will be obtainable on a great variety of different computers. The main purpose of the following program is to explain the algorithms of \TeX\ as clearly as possible. As a result, the program will not necessarily be very efficient when a particular \PASCAL\ compiler has translated it into a particular machine language. However, the program has been written so that it can be tuned to run efficiently in a wide variety of operating environments by making comparatively few changes. Such flexibility is possible because the documentation that follows is written in the \.{WEB} language, which is at a higher level than \PASCAL; the preprocessing step that converts \.{WEB} to \PASCAL\ is able to introduce most of the necessary refinements. Semi-automatic translation to other languages is also feasible, because the program below does not make extensive use of features that are peculiar to \PASCAL. A large piece of software like \TeX\ has inherent complexity that cannot be reduced below a certain level of difficulty, although each individual part is fairly simple by itself. The \.{WEB} language is intended to make the algorithms as readable as possible, by reflecting the way the individual program pieces fit together and by providing the cross-references that connect different parts. Detailed comments about what is going on, and about why things were done in certain ways, have been liberally sprinkled throughout the program. These comments explain features of the implementation, but they rarely attempt to explain the \TeX\ language itself, since the reader is supposed to be familiar with {\sl The \TeX book}. @.WEB@> @:TeXbook}{\sl The \TeX book@> @ The present implementation has a long ancestry, beginning in the summer of~1977, when Michael~F. Plass and Frank~M. Liang designed and coded a prototype @↑Plass, Michael Frederick@> @↑Liang, Franklin Mark@> @↑Knuth, Donald Ervin@> based on some specifications that the author had made in April of that year. This original proto\TeX\ included macro definitions and elementary manipulations on boxes and glue, but it did not have line-breaking, page-breaking, mathematical formulas, alignment routines, error recovery, or the present semantic nest; furthermore, it used character lists instead of token lists, so that a control sequence like \.{\\halign} was represented by a list of seven characters. A complete version of \TeX\ was designed and coded by the author in late 1977 and early 1978; that program, like its prototype, was written in the {\mc SAIL} language, for which an excellent debugging system was available. Preliminary plans to convert the {\mc SAIL} code into a form somewhat like the present ``web'' were developed by Luis Trabb~Pardo and the author at the beginning of 1979, and a complete implementation was created by Ignacio~A. Zabala in 1979 and 1980. The \TeX82 program, which @↑Zabala Salelles, Ignacio Andres@> was written by the author during the latter part of 1981 and the early part of 1982, also incorporates ideas from the 1979 implementation of @↑Guibas, Leonidas Ioannis@> @↑Sedgewick, Robert@> @↑Wyatt, Douglas Kirk@> \TeX\ in {\mc MESA} that was written by Leonidas Guibas, Robert Sedgewick, and Douglas Wyatt at the Xerox Palo Alto Research Center. Several hundred refinements were introduced into \TeX82 based on the experiences gained with the original implementations, so that essentially every part of the system has been substantially improved. After the appearance of ``Version 0'' in September, 1982, this program benefited greatly from the comments of many other people, notably David R. Fuchs and Howard W. Trickey. @↑Fuchs, David Raymond@> @↑Trickey, Howard Wellington@> No doubt there still is plenty of room for improvement, but the author is firmly committed to keeping \TeX82 ``frozen'' from now on; stability and reliability are to be its main virtues. On the other hand, the \.{WEB} description can be extended without changing the core of \TeX82 itself, and the program has been designed so that such extensions are not extremely difficult to make. The |banner| string defined here should be changed whenever \TeX\ undergoes any modifications, so that it will be clear which version of \TeX\ might be the guilty party when a problem arises. @↑extensions to \TeX@> @↑system dependencies@> @d banner=='This is TeX, Version 1.3' {printed when \TeX\ starts} @ Different \PASCAL s have slightly different conventions, and the present \def\ph{{\mc PASCAL-H}}% @!@:PASCAL H}{\ph@> program expresses \TeX\ in terms of the \PASCAL\ that was available to the author in 1982. The methods used here to work with this particular compiler, which we shall call \ph, should help the reader to see how to make an appropriate interface for other systems if necessary. (\ph\ is Charles Hedrick's modification of a compiler @↑Hedrick, Charles Locke@> for the DECsystem-10 that was originally developed at the University of Hamburg; cf.\ {\sl SOFTWARE---Practice \AM\ Experience \bf6} (1976), 29--42. The \TeX\ program below is intended to be adaptable, without extensive changes, to most other versions of \PASCAL, so it does not fully use the admirable features of \ph. Indeed, a conscious effort has been made here to avoid using several idiosyncratic features of standard \PASCAL\ itself, so that most of the code can be translated mechanically into other high-level languages. For example, the `\&{with}' and `\\{new}' features are not used, nor are pointer types, set types, or enumerated scalar types; there are no `\&{var}' parameters, except in the case of files; there are no tag fields on variant records; there are no assignments |real:=integer|; no procedures are declared local to other procedures.) The portions of this program that involve system-dependent code, where changes might be necessary because of differences between \PASCAL\ compilers and/or differences between operating systems, can be identified by looking at the sections whose numbers are listed under `system dependencies' in the index. Furthermore, the index entries for `dirty \PASCAL' list all places where the restrictions of \PASCAL\ have not been followed perfectly, for one reason or another. @!@↑system dependencies@> @!@↑dirty \PASCAL@> @ The program begins with a normal \PASCAL\ program heading, whose components will be filled in later, using the conventions of \.{WEB}. @.WEB@> For example, the portion of the program called `\X\glob:Global variables\X' here will be replaced by a sequence of variable declarations that starts in $\section\glob$ of this documentation. In this way, we are able to define each individual global variable when we are prepared to understand what it means; we do not have to define all of the globals at once. Cross references in $\section\glob$, where it says ``See also sections \gglob, \dots,'' also make it possible to look at the set of all global variables, if desired. Similar remarks apply to the other portions of the program heading. Actually the heading shown here is not quite normal: The |program| line does not mention any |output| file, because \ph\ would ask the \TeX\ user to specify a file name if |output| were specified here. @↑system dependencies@> @d mtype==t@&y@&p@&e {this is a \.{WEB} coding trick:} @f mtype==type {`\&{mtype}' will be equivalent to `\&{type}'} @f type==true {but `|type|' will not be treated as a reserved word} @p @t\4@>@<Compiler directives@>@/ program TEX; {all file names are defined dynamically} label @<Labels in the outer block@>@/ const @<Constants in the outer block@>@/ mtype @<Types in the outer block@>@/ var @<Global variables@>@/ @# procedure initialize; {this procedure gets things started properly} var @<Local variables for initialization@>@/ begin @<Initialize whatever \TeX\ might access@>@; end;@# @t\4@>@<Basic printing procedures@>@/ @t\4@>@<Error handling procedures@>@/ @ The overall \TeX\ program begins with the heading just shown, after which comes a bunch of procedure declarations and function declarations. Finally we will get to the main program, which begins with the comment `|start←here|'. If you want to skip down to the main program now, you can look up `|start←here|' in the index. But the author suggests that the best way to understand this program is to follow pretty much the order of \TeX's components as they appear in the \.{WEB} description you are now reading, since the present ordering is intended to combine the advantages of the ``bottom up'' and ``top down'' approaches to the problem of understanding a somewhat complicated system. @ Three labels must be declared in the main program, so we give them symbolic names. @d start←of←TEX=1 {go here when \TeX's variables are initialized} @d end←of←TEX=9998 {go here to close files and terminate gracefully} @d final←end=9999 {this label marks the ending of the program} @<Labels in the out...@>= start←of←TEX@t\hskip-2pt@>, end←of←TEX@t\hskip-2pt@>,@,final←end; {key control points} @ Some of the code below is intended to be used only when diagnosing the strange behavior that sometimes occurs when \TeX\ is being installed or when system wizards are fooling around with \TeX\ without quite knowing what they are doing. Such code will not normally be compiled; it is delimited by the codewords `$|debug|\ldots|gubed|$', with apologies to people who wish to preserve the purity of English. Similarly, there is some conditional code delimited by `$|stat|\ldots|tats|$' that is intended only for use when statistics are to be kept about \TeX's memory usage. @↑debugging@> @d debug==@{ {change this to `$\\{debug}\equiv\null$' when debugging} @d gubed==@t@>@} {change this to `$\\{gubed}\equiv\null$' when debugging} @f debug==begin @f gubed==end @# @d stat==@{ {change this to `$\\{stat}\equiv\null$' when gathering usage statistics} @d tats==@t@>@} {change this to `$\\{tats}\equiv\null$' when gathering usage statistics} @f stat==begin @f tats==end @ This program has two important variations: (1) There is a long and slow version called \.{INITEX}, which does the extra calculations needed to @.INITEX@> initialize \TeX's internal tables; and (2)~there is a shorter and faster production version, which cuts the initialization to a bare minimum. Parts of the program that are needed in (1) but not in (2) are delimited by the codewords `$|init|\ldots|tini|$'. @d init== {change this to `$\\{init}\equiv\.{@@\{}$' in the production version} @d tini== {change this to `$\\{tini}\equiv\.{@@\}}$' in the production version} @f init==begin @f tini==end @<Initialize whatever...@>= @<Set initial values of key variables@>@/ @!init @<Initialize table entries (done by \.{INITEX} only)@>@;@+tini @ If the first character of a \PASCAL\ comment is a dollar sign, \ph\ treats the comment as a list of ``compiler directives'' that will affect the translation of this program into machine language. The directives shown below specify full checking and inclusion of the \PASCAL\ debugger when \TeX\ is being debugged, but they cause range checking and other redundant code to be eliminated when the production system is being generated. Arithmetic overflow will be detected in all cases. @↑system dependencies@> @↑Overflow in arithmetic@> @<Compiler directives@>= @{@&$C-,A+,D-@} {no range check, catch arithmetic overflow, no debug overhead} @!debug @{@&$C+,D+@}@+ gubed {but turn everything on when debugging} @ This \TeX\ implementation conforms to the rules of the {\sl PASCAL User @:PASCAL}{\PASCAL@> @↑system dependencies@> Manual} published by Jensen and Wirth in 1975, except where system-dependent @↑Wirth, Niklaus@> @↑Jensen, Kathleen@> code is necessary to make a useful system program, and except in another respect where such conformity would unnecessarily obscure the meaning and clutter up the code: We assume that |case| statements may include a default case that applies if no matching label is found. Thus, we shall use constructions like $$\vbox{\halign{\ignorespaces#\hfil\cr |case x of|\cr 1: $\langle\,$code for $x=1\,\rangle$;\cr 3: $\langle\,$code for $x=3\,\rangle$;\cr |othercases| $\langle\,$code for |x<>1| and |x<>3|$\,\rangle$\cr |endcases|\cr}}$$ since most \PASCAL\ compilers have plugged this hole in the language by incorporating some sort of default mechanism. For example, the \ph\ compiler allows `|others|:' as a default label, and other \PASCAL s allow syntaxes like `\&{else}' or `\&{otherwise}' or `\\{otherwise}:', etc. The definitions of |othercases| and |endcases| should be changed to agree with local conventions. Note that no semicolon appears before |endcases| in this program, so the definition of |endcases| should include a semicolon if the compiler wants one. (Of course, if no default mechanism is available, the |case| statements of \TeX\ will have to be laboriously extended by listing all remaining cases. People who are stuck with such \PASCAL s have in fact done this, successfully but not happily!) @d othercases == others: {default for cases not listed explicitly} @d endcases == @+end {follows the default case in an extended |case| statement} @f othercases == else @f endcases == end @ The following parameters can be changed at compile time to extend or reduce \TeX's capacity. They may have different values in \.{INITEX} and in production versions of \TeX. @.INITEX@> @↑system dependencies@> @<Constants...@>= @!mem←max=30000; {greatest index in \TeX's internal |mem| array; must be strictly less than |max←halfword|; must be equal to |mem←top| in \.{INITEX}, otherwise |>=mem←top|} @!mem←min=0; {smallest index in \TeX's internal |mem| array; must be |min←halfword| or more; must be equal to |mem←bot| in \.{INITEX}, otherwise |<=mem←bot|} @!buf←size=500; {maximum number of characters simultaneously present in current lines of open files and in control sequences between \.{\\csname} and \.{\\endcsname}; must not exceed |max←halfword|} @!error←line=72; {width of context lines on terminal error messages} @!half←error←line=42; {width of first lines of contexts in terminal error messages; should be between 30 and |error←line-15|} @!max←print←line=79; {width of longest text lines output; should be at least 60} @!stack←size=200; {maximum number of simultaneous input sources} @!max←in←open=6; {maximum number of input files and error insertions that can be going on simultaneously} @!font←max=75; {maximum internal font number; must not exceed |max←quarterword| and must be at most |font←base+256|} @!font←mem←size=20000; {number of words of |font←info| for all fonts} @!param←size=60; {maximum number of simultaneous macro parameters} @!nest←size=40; {maximum number of semantic levels simultaneously active} @!max←strings=3000; {maximum number of strings; must not exceed |max←halfword|} @!string←vacancies=8000; {the minimum number of characters that should be available for the user's control sequences and font names, after \TeX's own error messages are stored} @!pool←size=32000; {maximum number of characters in strings, including all error messages and help texts, and the names of all fonts and control sequences; must exceed |string←vacancies| by the total length of \TeX's own strings, which is currently about 23000} @!save←size=600; {space for saving values outside of current group; must be at most |max←halfword|} @!trie←size=8000; {space for hyphenation patterns; should be larger for \.{INITEX} than it is in production versions of \TeX} @!dvi←buf←size=800; {size of the output buffer; must be a multiple of 8} @!file←name←size=40; {file names shouldn't be longer than this} @!pool←name='TeXformats:TEX.POOL '; {string of length |file←name←size|; tells where the string pool appears} @.TeXformats@> @ Like the preceding parameters, the following quantities can be changed at compile time to extend or reduce \TeX's capacity. But if they are changed, it is necessary to rerun the initialization program \.{INITEX} @.INITEX@> to generate new tables for the production \TeX\ program. One can't simply make helter-skelter changes to the following constants, since certain rather complex initialization numbers are computed from them. They are defined here using \.{WEB} macros, instead of being put into \PASCAL's |const| list, in order to emphasize this distinction. @d mem←bot=0 {smallest index in the |mem| array dumped by \.{INITEX}; must not be less than |mem←min|} @d mem←top==30000 {largest index in the |mem| array dumped by \.{INITEX}; must be substantially larger than |mem←bot| and not greater than |mem←max|} @d font←base=0 {smallest internal font number; must not be less than |min←quarterword|} @d hash←size=2100 {maximum number of control sequences; it should be at most about |(mem←max-mem←min)/10|, but 2100 is already quite generous} @d hash←prime=1777 {a prime number equal to about 85\% of |hash←size|} @d hyph←size=307 {another prime; the number of \.{\\hyphenation} exceptions} @↑system dependencies@> @ In case somebody has inadvertently made bad settings of the ``constants,'' \TeX\ checks them using a global variable called |bad|. This is the first of many sections of \TeX\ where global variables are defined. @<Glob...@>= @!bad:integer; {is some ``constant'' wrong?} @ Later on we will say `\ignorespaces|if mem←max>=max←halfword then bad:=10|', or something similar. (We can't do that until |max←halfword| has been defined.) @<Check the ``constant'' values for consistency@>= bad:=0; if (half←error←line<30)or(half←error←line>error←line-15) then bad:=1; if max←print←line<60 then bad:=2; if dvi←buf←size mod 8<>0 then bad:=3; if mem←bot+1100>mem←top then bad:=4; if hash←prime>hash←size then bad:=5; @ Labels are given symbolic names by the following definitions, so that occasional |goto| statements will be meaningful. We insert the label `|exit|:' just before the `\ignorespaces|end|\unskip' of a procedure in which we have used the `|return|' statement defined below; the label `|restart|' is occasionally used at the very beginning of a procedure; and the label `|reswitch|' is occasionally used just prior to a |case| statement in which some cases change the conditions and we wish to branch to the newly applicable case. Loops that are set up with the |loop| construction defined below are commonly exited by going to `|done|' or to `|found|' or to `|not←found|', and they are sometimes repeated by going to `|continue|'. If two or more parts of a subroutine start differently but end up the same, the shared code may be gathered together at `|common←ending|'. Incidentally, this program never declares a label that isn't actually used, because some fussy \PASCAL\ compilers will complain about redundant labels. @d exit=10 {go here to leave a procedure} @d restart=20 {go here to start a procedure again} @d reswitch=21 {go here to start a case statement again} @d continue=22 {go here to resume a loop} @d done=30 {go here to exit a loop} @d done1=31 {like |done|, when there is more than one loop} @d done2=32 {for exiting the second loop in a long block} @d done3=33 {for exiting the third loop in a very long block} @d done4=34 {for exiting the fourth loop in an extremely long block} @d done5=35 {for exiting the fifth loop in an immense block} @d done6=36 {for exiting the sixth loop in a block} @d found=40 {go here when you've found it} @d found1=41 {like |found|, when there's more than one per routine} @d found2=42 {like |found|, when there's more than two per routine} @d not←found=45 {go here when you've found nothing} @d common←ending=50 {go here when you want to merge with another branch} @ Here are some macros for common programming idioms. @d incr(#) == #:=#+1 {increase a variable by unity} @d decr(#) == #:=#-1 {decrease a variable by unity} @d negate(#) == #:=-# {change the sign of a variable} @d loop == @+ while true do@+ {repeat over and over until a |goto| happens} @f loop == xclause {\.{WEB}'s |xclause| acts like `\ignorespaces|while true do|\unskip'} @d do←nothing == {empty statement} @d return == goto exit {terminate a procedure call} @f return == nil @d empty=0 {symbolic name for a null constant} @* \[2] The character set. In order to make \TeX\ readily portable between a wide variety of computers, all of its input text is converted to an internal seven-bit code that is essentially standard ASCII, the ``American Standard Code for Information Interchange.'' This conversion is done immediately when each character is read in. Conversely, characters are converted from ASCII to the user's external representation just before they are output to a text file. Such an internal code is relevant to users of \TeX\ primarily because it governs the positions of characters in the fonts. For example, the character `\.A' has ASCII code $65=@'101$, and when \TeX\ typesets this letter it specifies character number 65 in the current font. If that font actually has `\.A' in a different position, \TeX\ doesn't know what the real position is; the program that does the actual printing from \TeX's device-independent files is responsible for converting from ASCII to a particular font encoding. @↑ASCII code@> \TeX's internal code is relevant also with respect to constants that begin with a reverse apostrophe; and it provides an index to the \.{\\catcode}, \.{\\mathcode}, \.{\\uccode}, \.{\\lccode}, and \.{\\delcode} tables. @ Characters of text that have been converted to \TeX's internal form are said to be of type |ASCII←code|, which is a subrange of the integers. @<Types...@>= @!ASCII←code=0..127; {seven-bit numbers} @ The original \PASCAL\ compiler was designed in the late 60s, when six-bit character sets were common, so it did not make provision for lowercase letters. Nowadays, of course, we need to deal with both capital and small letters in a convenient way, especially in a program for typesetting; so the present specification of \TeX\ has been written under the assumption that the \PASCAL\ compiler and run-time system permit the use of text files with more than 64 distinguishable characters. More precisely, we assume that the character set contains at least the letters and symbols associated with ASCII codes @'40 through @'176; all of these characters are now available on most computer terminals. Since we are dealing with more characters than were present in the first \PASCAL\ compilers, we have to decide what to call the associated data type. Some \PASCAL s use the original name |char| for the characters in text files, even though there now are more than 64 such characters, while other \PASCAL s consider |char| to be a 64-element subrange of a larger data type that has some other name. In order to accommodate this difference, we shall use the name |text←char| to stand for the data type of the characters that are converted to and from |ASCII←code| when they are input and output. We shall also assume that |text←char| consists of the elements |chr(first←text←char)| through |chr(last←text←char)|, inclusive. The following definitions should be adjusted if necessary. @↑system dependencies@> @d text←char == char {the data type of characters in text files} @d first←text←char=0 {ordinal number of the smallest element of |text←char|} @d last←text←char=127 {ordinal number of the largest element of |text←char|} @<Local variables for init...@>= i:0..last←text←char; @ The \TeX\ processor converts between ASCII code and the user's external character set by means of arrays |xord| and |xchr| that are analogous to \PASCAL's |ord| and |chr| functions. @<Glob...@>= @!xord: array [text←char] of ASCII←code; {specifies conversion of input characters} @!xchr: array [ASCII←code] of text←char; {specifies conversion of output characters} @ Since we are assuming that our \PASCAL\ system is able to read and write the visible characters of standard ASCII (although not necessarily using the ASCII codes to represent them), the following assignment statements initialize most of the |xchr| array properly, without needing any system-dependent changes. On the other hand, it is possible to implement \TeX\ with less complete character sets, and in such cases it will be necessary to change something here. @↑system dependencies@> @<Set init...@>= xchr[@'40]:=' '; xchr[@'41]:='!'; xchr[@'42]:='"'; xchr[@'43]:='#'; xchr[@'44]:='$'; xchr[@'45]:='%'; xchr[@'46]:='&'; xchr[@'47]:='''';@/ xchr[@'50]:='('; xchr[@'51]:=')'; xchr[@'52]:='*'; xchr[@'53]:='+'; xchr[@'54]:=','; xchr[@'55]:='-'; xchr[@'56]:='.'; xchr[@'57]:='/';@/ xchr[@'60]:='0'; xchr[@'61]:='1'; xchr[@'62]:='2'; xchr[@'63]:='3'; xchr[@'64]:='4'; xchr[@'65]:='5'; xchr[@'66]:='6'; xchr[@'67]:='7';@/ xchr[@'70]:='8'; xchr[@'71]:='9'; xchr[@'72]:=':'; xchr[@'73]:=';'; xchr[@'74]:='<'; xchr[@'75]:='='; xchr[@'76]:='>'; xchr[@'77]:='?';@/ xchr[@'100]:='@@'; xchr[@'101]:='A'; xchr[@'102]:='B'; xchr[@'103]:='C'; xchr[@'104]:='D'; xchr[@'105]:='E'; xchr[@'106]:='F'; xchr[@'107]:='G';@/ xchr[@'110]:='H'; xchr[@'111]:='I'; xchr[@'112]:='J'; xchr[@'113]:='K'; xchr[@'114]:='L'; xchr[@'115]:='M'; xchr[@'116]:='N'; xchr[@'117]:='O';@/ xchr[@'120]:='P'; xchr[@'121]:='Q'; xchr[@'122]:='R'; xchr[@'123]:='S'; xchr[@'124]:='T'; xchr[@'125]:='U'; xchr[@'126]:='V'; xchr[@'127]:='W';@/ xchr[@'130]:='X'; xchr[@'131]:='Y'; xchr[@'132]:='Z'; xchr[@'133]:='['; xchr[@'134]:='\'; xchr[@'135]:=']'; xchr[@'136]:='↑'; xchr[@'137]:='←';@/ xchr[@'140]:='`'; xchr[@'141]:='a'; xchr[@'142]:='b'; xchr[@'143]:='c'; xchr[@'144]:='d'; xchr[@'145]:='e'; xchr[@'146]:='f'; xchr[@'147]:='g';@/ xchr[@'150]:='h'; xchr[@'151]:='i'; xchr[@'152]:='j'; xchr[@'153]:='k'; xchr[@'154]:='l'; xchr[@'155]:='m'; xchr[@'156]:='n'; xchr[@'157]:='o';@/ xchr[@'160]:='p'; xchr[@'161]:='q'; xchr[@'162]:='r'; xchr[@'163]:='s'; xchr[@'164]:='t'; xchr[@'165]:='u'; xchr[@'166]:='v'; xchr[@'167]:='w';@/ xchr[@'170]:='x'; xchr[@'171]:='y'; xchr[@'172]:='z'; xchr[@'173]:='{'; xchr[@'174]:='|'; xchr[@'175]:='}'; xchr[@'176]:='~';@/ xchr[0]:=' '; xchr[@'177]:=' '; {ASCII codes 0 and |@'177| do not appear in text} @ Some of the ASCII codes without visible characters have been given symbolic names in this program because they are used with a special meaning. @d null←code=@'0 {ASCII code that might disappear} @d carriage←return=@'15 {ASCII code used at end of line} @d invalid←code=@'177 {ASCII code that should not appear} @ The ASCII code is ``standard'' only to a certain extent, since many computer installations have found it advantageous to have ready access to more than 94 printing characters. Appendix~C of {\sl The \TeX book\/} gives a complete specification of the intended correspondence between characters and \TeX's internal representation. @:TeXbook}{\sl The \TeX book@> If \TeX\ is being used on a garden-variety \PASCAL\ for which only standard ASCII codes will appear in the input and output files, it doesn't really matter what codes are specified in |xchr[1..@'37]|, but the safest policy is to blank everything out by using the code shown below. However, other settings of |xchr| will make \TeX\ more friendly on computers that have an extended character set, so that users can type things like `\.↑↑Z' instead of `\.{\\ne}'. At MIT, for example, it would be more appropriate to substitute the code $$\hbox{|for i:=1 to @'37 do xchr[i]:=chr(i);|}$$ \TeX's character set is essentially the same as MIT's, even with respect to characters less than~@'40. People with extended character sets can assign codes arbitrarily, giving an |xchr| equivalent to whatever characters the users of \TeX\ are allowed to have in their input files. It is best to make the codes correspond to the intended interpretations as shown in Appendix~C whenever possible; but this is not necessary. For example, in countries with an alphabet of more than 26 letters, it is usually best to map the additional letters into codes less than~@'40. @↑character set dependencies@> @↑system dependencies@> @<Set init...@>= for i:=1 to @'37 do xchr[i]:=' '; @ The following system-independent code makes the |xord| array contain a suitable inverse to the information in |xchr|. Note that if |xchr[i]=xchr[j]| where |i<j<@'177|, the value of |xord[xchr[i]]| will turn out to be |j| or more; hence, standard ASCII code numbers will be used instead of codes below @'40 in case there is a coincidence. @<Set init...@>= for i:=first←text←char to last←text←char do xord[chr(i)]:=invalid←code; for i:=1 to @'176 do xord[xchr[i]]:=i; @* \[3] Input and output. The bane of portability is the fact that different operating systems treat input and output quite differently, perhaps because computer scientists have not given sufficient attention to this problem. People have felt somehow that input and output are not a part of ``real'' programming. Well, it is true that some kinds of programming are more fun than others. With existing input/output conventions being so diverse and so messy, the only sources of joy in such parts of the code are the rare occasions when one can find a way to make the program a little less bad than it might have been. We have two choices: either to attack I/O now and get it over with, or to postpone it until near the end. Neither prospect is very attractive, so let's get it over with. The basic operations we need to do are (1)~inputting and outputting of text, to or from a file or the user's terminal; (2)~inputting and outputting of eight-bit bytes, to or from a file; (3)~instructing the operating system to initiate (``open'') or to terminate (``close'') input or output from a specified file; (4)~testing whether the end of an input file has been reached. \TeX\ needs to deal with two kinds of files. We shall use the term |alpha←file| for a file that contains textual data, and the term |byte←file| for a file that contains eight-bit binary information. These two types turn out to be the same on many computers, but sometimes there is a significant distinction, so we shall be careful to distinguish between them. Standard protocols for transferring such files from computer to computer, via high-speed networks, are now becoming available to more and more communities of users. The program actually makes use also of a third kind of file, called a |word←file|, when dumping and reloading base information for its own initialization. We shall define a word file later; but it will be possible for us to specify simple operations on word files before they are defined. @<Types...@>= @!eight←bits=0..255; {unsigned one-byte quantity} @!alpha←file=packed file of text←char; {files that contain textual data} @!byte←file=packed file of eight←bits; {files that contain binary data} @ Most of what we need to do with respect to input and output can be handled by the I/O facilities that are standard in \PASCAL, i.e., the routines called |get|, |put|, |eof|, and so on. But standard \PASCAL\ does not allow file variables to be associated with file names that are determined at run time, so it cannot be used to implement \TeX; some sort of extension to \PASCAL's ordinary |reset| and |rewrite| is crucial for our purposes. We shall assume that |name←of←file| is a variable of an appropriate type such that the \PASCAL\ run-time system being used to implement \TeX\ can open a file whose external name is specified by |name←of←file|. @↑system dependencies@> @<Glob...@>= @!name←of←file:packed array[1..file←name←size] of char;@;@/ {on some systems this may be a \&{record} variable} @!name←length:0..file←name←size;@/{this many characters are actually relevant in |name←of←file| (the rest are blank)} @ The \ph\ compiler with which the present version of \TeX\ was prepared has extended the rules of \PASCAL\ in a very convenient way. To open file~|f|, we can write $$\vbox{\halign{#\hfil\qquad&#\hfil\cr |reset(f,@t\\{name}@>,'/O')|&for input;\cr |rewrite(f,@t\\{name}@>,'/O')|&for output.\cr}}$$ The `\\{name}' parameter, which is of type `\ignorespaces|packed array[@t\<\\{any}>@>] of text←char|', stands for the name of the external file that is being opened for input or output. Blank spaces that might appear in \\{name} are ignored. The `\.{/O}' parameter tells the operating system not to issue its own error messages if something goes wrong. If a file of the specified name cannot be found, or if such a file cannot be opened for some other reason (e.g., someone may already be trying to write the same file), we will have |@!erstat(f)<>0| after an unsuccessful |reset| or |rewrite|. This allows \TeX\ to undertake appropriate corrective action. @:PASCAL H}{\ph@> @↑system dependencies@> \TeX's file-opening procedures return |false| if no file identified by |name←of←file| could be opened. @d reset←OK(#)==erstat(#)=0 @d rewrite←OK(#)==erstat(#)=0 @p function a←open←in(var f:alpha←file):boolean; {open a text file for input} begin reset(f,name←of←file,'/O'); a←open←in:=reset←OK(f); end; @# function a←open←out(var f:alpha←file):boolean; {open a text file for output} begin rewrite(f,name←of←file,'/O'); a←open←out:=rewrite←OK(f); end; @# function b←open←in(var f:byte←file):boolean; {open a binary file for input} begin reset(f,name←of←file,'/O'); b←open←in:=reset←OK(f); end; @# function b←open←out(var f:byte←file):boolean; {open a binary file for output} begin rewrite(f,name←of←file,'/O'); b←open←out:=rewrite←OK(f); end; @# function w←open←in(var f:word←file):boolean; {open a word file for input} begin reset(f,name←of←file,'/O'); w←open←in:=reset←OK(f); end; @# function w←open←out(var f:word←file):boolean; {open a word file for output} begin rewrite(f,name←of←file,'/O'); w←open←out:=rewrite←OK(f); end; @ Files can be closed with the \ph\ routine `|close(f)|', which @↑system dependencies@> should be used when all input or output with respect to |f| has been completed. This makes |f| available to be opened again, if desired; and if |f| was used for output, the |close| operation makes the corresponding external file appear on the user's area, ready to be read. These procedures should not generate error messages if a file is being closed before it has been successfully opened. @p procedure a←close(var f:alpha←file); {close a text file} begin close(f); end; @# procedure b←close(var f:byte←file); {close a binary file} begin close(f); end; @# procedure w←close(var f:word←file); {close a word file} begin close(f); end; @ Binary input and output are done with \PASCAL's ordinary |get| and |put| procedures, so we don't have to make any other special arrangements for binary~I/O. Text output is also easy to do with standard \PASCAL\ routines. The treatment of text input is more difficult, however, because of the necessary translation to |ASCII←code| values, and because \TeX's conventions should be efficient and they should blend nicely with the user's operating environment. @ Input from text files is read one line at a time, using a routine called |input←ln|. This function is defined in terms of global variables called |buffer|, |first|, and |last| that will be described in detail later; for now, it suffices for us to know that |buffer| is an array of |ASCII←code| values, and that |first| and |last| are indices into this array representing the beginning and ending of a line of text. @<Glob...@>= @!buffer:array[0..buf←size] of ASCII←code; {lines of characters being read} @!first:0..buf←size; {the first unused position in |buffer|} @!last:0..buf←size; {end of the line just input to |buffer|} @!max←buf←stack:0..buf←size; {largest index used in |buffer|} @ The |input←ln| function brings the next line of input from the specified field into available positions of the buffer array and returns the value |true|, unless the file has already been entirely read, in which case it returns |false| and sets |last:=first|. In general, the |ASCII←code| numbers that represent the next line of the file are input into |buffer[first]|, |buffer[first+1]|, \dots, |buffer[last-1]|; and the global variable |last| is set equal to |first| plus the length of the line. Trailing blanks are removed from the line; thus, either |last=first| (in which case the line was entirely blank) or |buffer[last-1]<>" "|. An overflow error is given, however, if the normal actions of |input←ln| would make |last>=buf←size|; this is done so that other parts of \TeX\ can safely look at the contents of |buffer[last+1]| without overstepping the bounds of the |buffer| array. Upon entry to |input←ln|, the condition |first<buf←size| will always hold, so that there is always room for an ``empty'' line. The variable |max←buf←stack|, which is used to keep track of how large the |buf←size| parameter must be to accommodate the present job, is also kept up to date by |input←ln|. If the |bypass←eoln| parameter is |true|, |input←ln| will do a |get| before looking at the first character of the line; this skips over an |eoln| that was in |f↑|. The procedure does not do a |get| when it reaches the end of the line; therefore it can be used to acquire input from the user's terminal as well as from ordinary text files. Standard \PASCAL\ says that a file should have |eoln| immediately before |eof|, but \TeX\ needs only a weaker restriction: If |eof| occurs in the middle of a line, the system function |eoln| should return a |true| result (even though |f↑| will be undefined). Since the inner loop of |input←ln| is part of \TeX's ``inner loop''---each character of input comes in at this place---it is wise to reduce system overhead by making use of special routines that read in an entire array of characters at once, if such routines are available. The following code uses standard \PASCAL\ to illustrate what needs to be done, but finer tuning is often possible at well-developed \PASCAL\ sites. @↑inner loop@> @p function input←ln(var f:alpha←file;@!bypass←eoln:boolean):boolean; {inputs the next line or returns |false|} var last←nonblank:0..buf←size; {|last| with trailing blanks removed} begin if bypass←eoln then if not eof(f) then get(f); {input the first character of the line into |f↑|} last:=first; {cf.\ Matthew 19\thinspace:\thinspace30} if eof(f) then input←ln:=false else begin last←nonblank:=first; while not eoln(f) do begin if last>=max←buf←stack then begin max←buf←stack:=last+1; if max←buf←stack=buf←size then overflow("buffer size",buf←size); @:TeX capacity exceeded buffer size}{\quad buffer size@> end; buffer[last]:=xord[f↑]; get(f); incr(last); if buffer[last-1]<>" " then last←nonblank:=last; end; last:=last←nonblank; input←ln:=true; end; end; @ The user's terminal acts essentially like other files of text, except that it is used both for input and for output. When the terminal is considered an input file, the file variable is called |term←in|, and when it is considered an output file the file variable is |term←out|. @↑system dependencies@> @<Glob...@>= @!term←in:alpha←file; {the terminal as an input file} @!term←out:alpha←file; {the terminal as an output file} @ Here is how to open the terminal files in \ph. The `\.{/I}' switch suppresses the first |get|. @↑system dependencies@> @d t←open←in==reset(term←in,'TTY:','/O/I') {open the terminal for text input} @d t←open←out==rewrite(term←out,'TTY:','/O') {open the terminal for text output} @ Sometimes it is necessary to synchronize the input/output mixture that happens on the user's terminal, and three procedures are used for this purpose. The first of these, |update←terminal|, is called when we want to make sure that everything we have output to the terminal so far has actually left the computer's internal buffers and been sent. The second, |clear←terminal|, is called when we wish to cancel any input that the user may have typed ahead (since we are about to issue an unexpected error message). The third, |wake←up←terminal|, is supposed to revive the terminal if the user has disabled it by some instruction to the operating system. The following macros show how these operations can be specified in \ph: @↑system dependencies@> @d update←terminal == break(term←out) {empty the terminal output buffer} @d clear←terminal == break←in(term←in,true) {clear the terminal input buffer} @d wake←up←terminal == do←nothing {cancel the user's cancellation of output} @ We need a special routine to read the first line of \TeX\ input from the user's terminal. This line is different because it is read before we have opened the transcript file; there is sort of a ``chicken and egg'' problem here. If the user types `\.{\\input paper}' on the first line, or if some macro invoked by that line does such an \.{\\input}, the transcript file will be named `\.{paper.log}'; but if no \.{\\input} commands are performed during the first line of terminal input, the transcript file will acquire its default name `\.{texput.log}'. (The transcript file will not contain error messages generated by the first line before the first \.{\\input} command.) @.texput@> The first line is even more special if we are lucky enough to have an operating system that treats \TeX\ differently from a run-of-the-mill \PASCAL\ object program. It's nice to let the user start running a \TeX\ job by typing a command line like `\.{tex paper}'; in such a case, \TeX\ will operate as if the first line of input were `\.{paper}', i.e., the first line will consist of the remainder of the command line, after the part that invoked \TeX. @ Different systems have different ways to get started. But regardless of what conventions are adopted, the routine that initializes the terminal should satisfy the following specifications: \yskip\textindent{1)}It should open file |term←in| for input from the terminal. (The file |term←out| will already be open for output to the terminal.) \textindent{2)}If the user has given a command line, this line should be considered the first line of terminal input. Otherwise the user should be prompted with `\.{**}', and the first line of input should be whatever is typed in response. \textindent{3)}The first line of input, which might or might not be a command line, should appear in locations |first| to |last-1| of the |buffer| array. \textindent{4)}The global variable |loc| should be set so that the character that \TeX\ reads next is in |buffer[loc]|. This character should not be blank, and we should have |loc<last|. \yskip\noindent(It may be necessary to prompt the user several times before a non-blank line comes in. The prompt is `\.{**}' instead of the later `\.*' because the meaning is slightly different: `\.{\\input}' need not be typed immediately after~`\.{**}'.) @d loc==cur←input.loc←field {location of first unread character in |buffer|} @ The following program does the required initialization without retrieving a possible command line. It should be clear how to modify this routine to deal with command lines, if the system permits them. @↑system dependencies@> @p function init←terminal:boolean; {gets the terminal input started} label exit; begin t←open←in; loop@+begin wake←up←terminal; write(term←out,'**'); update←terminal; @.**@> if not input←ln(term←in,true) then {this shouldn't happen} begin write←ln(term←out); write(term←out,'! End of file on the terminal... why?'); @.End of file on the terminal@> init←terminal:=false; return; end; loc:=first; while (loc<last)and(buffer[loc]=" ") do incr(loc); if loc<last then begin init←terminal:=true; return; {return unless the line was all blank} end; write←ln(term←out,'Please type the name of your input file.'); end; exit:end; @* \[4] String handling. Control sequence names and diagnostic messages are variable-length strings of seven-bit characters. Since \PASCAL\ does not have a well-developed string mechanism, \TeX\ does all of its string processing by homegrown methods. Elaborate facilities for dynamic strings are not needed, so all of the necessary operations can be handled with a simple data structure. The array |str←pool| contains all of the (seven-bit) ASCII codes in all of the strings, and the array |str←start| contains indices of the starting points of each string. Strings are referred to by integer numbers, so that string number |s| comprises the characters |str←pool[j]| for |str←start[s]<=j<str←start[s+1]|. Additional integer variables |pool←ptr| and |str←ptr| indicate the number of entries used so far in |str←pool| and |str←start|, respectively; locations |str←pool[pool←ptr]| and |str←start[str←ptr]| are ready for the next string to be allocated. String numbers 0 to 127 are reserved for strings that correspond to single ASCII characters. This is in accordance with the conventions of \.{WEB}, @.WEB@> which converts single-character strings into the ASCII code number of the single character involved, while it converts other strings into integers and builds a string pool file. Thus, when the string constant \.{"."} appears in the program below, \.{WEB} converts it into the integer 46, which is the ASCII code for a period, while \.{WEB} will convert a string like \.{"hello"} into some integer greater than~127. String number 46 will presumably be the single character `\..'; but some ASCII codes have no standard visible representation, and \TeX\ sometimes needs to be able to print an arbitrary ASCII character, so the first 128 strings are used to specify exactly what should be printed for each of the 128 possibilities. Elements of the |str←pool| array must be ASCII codes that can actually be printed; i.e., they must have an |xchr| equivalent in the local character set. (However, the names of control sequences need not meet this restriction, when they appear in |str←pool|.) @<Types...@>= @!pool←pointer = 0..pool←size; {for variables that point into |str←pool|} @!str←number = 0..max←strings; {for variables that point into |str←start|} @ @<Glob...@>= @!str←pool:packed array[pool←pointer] of ASCII←code; {the characters} @!str←start : array[str←number] of pool←pointer; {the starting pointers} @!pool←ptr : pool←pointer; {first unused position in |str←pool|} @!str←ptr : str←number; {start of the current string being created} @!init←pool←ptr : pool←pointer; {the starting value of |pool←ptr|} @!init←str←ptr : str←number; {the starting value of |str←ptr|} @ Several of the elementary string operations are performed using \.{WEB} macros instead of using \PASCAL\ procedures, because many of the operations are done quite frequently and we want to avoid the overhead of procedure calls. For example, here is a simple macro that computes the length of a string. @.WEB@> @d length(#)==(str←start[#+1]-str←start[#]) {the number of characters in string number \#} @ The length of the current string is called |cur←length|: @d cur←length == (pool←ptr - str←start[str←ptr]) @ Strings are created by appending character codes to |str←pool|. The macro called |append←char|, defined here, does not check to see if the value of |pool←ptr| has gotten too high; this test is supposed to be made before |append←char| is used. There is also a |flush←char| macro, which erases the last character appended. To test if there is room to append |l| more characters to |str←pool|, we shall write |str←room(l)|, which aborts \TeX\ and gives an apologetic error message if there isn't enough room. @d append←char(#) == {put |ASCII←code| \# at the end of |str←pool|} begin str←pool[pool←ptr]:=#; incr(pool←ptr); end @d flush←char == decr(pool←ptr) {forget the last character in the pool} @d str←room(#) == {make sure that the pool hasn't overflowed} begin if pool←ptr+# > pool←size then overflow("pool size",pool←size-init←pool←ptr); @:TeX capacity exceeded pool size}{\quad pool size@> end @ Once a sequence of characters has been appended to |str←pool|, it officially becomes a string when the function |make←string| is called. This function returns the identification number of the new string as its value. @p function make←string : str←number; {current string enters the pool} begin if str←ptr=max←strings then overflow("number of strings",max←strings-init←str←ptr); @:TeX capacity exceeded number of strings}{\quad number of strings@> incr(str←ptr); str←start[str←ptr]:=pool←ptr; make←string:=str←ptr-1; end; @ To destroy the most recently made string, we say |flush←string|. @d flush←string==begin decr(str←ptr); pool←ptr:=str←start[str←ptr]; end @ The following subroutine compares string |s| with another string of the same length that appears in |buffer| starting at position |k|; the result is |true| if and only if the strings are equal. Empirical tests indicate that |str←eq←buf| is used in such a way that it tends to return |true| about 80 percent of the time. @p function str←eq←buf(@!s:str←number;@!k:integer):boolean; {test equality of strings} label not←found; {loop exit} var j: pool←pointer; {running index} @!result: boolean; {result of comparison} begin j:=str←start[s]; while j<str←start[s+1] do begin if str←pool[j]<>buffer[k] then begin result:=false; goto not←found; end; incr(j); incr(k); end; result:=true; not←found: str←eq←buf:=result; end; @ Here is a similar routine, but it compares two strings in the string pool, and it does not assume that they have the same length. @p function str←eq←str(@!s,@!t:str←number):boolean; {test equality of strings} label not←found; {loop exit} var j,@!k: pool←pointer; {running indices} @!result: boolean; {result of comparison} begin result:=false; if length(s)<>length(t) then goto not←found; j:=str←start[s]; k:=str←start[t]; while j<str←start[s+1] do begin if str←pool[j]<>str←pool[k] then goto not←found; incr(j); incr(k); end; result:=true; not←found: str←eq←str:=result; end; @ The initial values of |str←pool|, |str←start|, |pool←ptr|, and |str←ptr| are computed by the \.{INITEX} program, based in part on the information that \.{WEB} has output while processing \TeX. @.INITEX@> @↑string pool@> @p @!init function get←strings←started:boolean; {initializes the string pool, but returns |false| if something goes wrong} label done,exit; var k,@!l:0..127; {small indices or counters} @!m,@!n:text←char; {characters input from |pool←file|} @!g:str←number; {garbage} @!a:integer; {accumulator for check sum} @!c:boolean; {check sum has checked} begin pool←ptr:=0; str←ptr:=0; str←start[0]:=0; @<Make the first 128 strings@>; @<Read the other strings from the \.{TEX.POOL} file and return |true|, or give an error message and return |false|@>; exit:end; tini @ @<Make the first 128...@>= for k:=0 to 127 do begin if (@<Character |k| cannot be printed@>) then begin append←char("↑"); append←char("↑"); if k<@'100 then append←char(k+@'100) else append←char(k-@'100); end else append←char(k); g:=make←string; end @ The first 128 strings will contain 95 standard ASCII characters, and the other 33 characters will be printed in three-symbol form like `\.{\↑\↑A}' unless a system-dependent change is made here. Installations that have an extended character set, where for example |xchr[@'32]=@t\.{\'↑↑Z\'}@>|, would like string @'32 to be the single character @'32 instead of the three characters @'136, @'136, @'132 (\.{\↑\↑Z}). On the other hand, even people with an extended character set will want to represent string @'15 by \.{\↑\↑M}, since @'15 is |carriage←return|; the idea is to produce visible strings instead of tabs or line-feeds or carriage-returns or bell-rings or characters that are treated anomalously in text files. The boolean expression defined here should be |true| unless \TeX\ internal code number~|k| corresponds to a non-troublesome visible symbol in the local character set. At MIT, for example, the appropriate formula would be `|k in [0,@'10..@'12,@'14,@'15,@'33,@'177]|'. If character |k| cannot be printed, then character |k+@'100| or |k-@'100| must be printable; thus, at least 64 printable characters are needed. @↑character set dependencies@> @↑system dependencies@> @<Character |k| cannot be printed@>= (k<" ")or(k>"~") @ When the \.{WEB} system program called \.{TANGLE} processes the \.{TEX.WEB} description that you are now reading, it outputs the \PASCAL\ program \.{TEX.PAS} and also a string pool file called \.{TEX.POOL}. The \.{INITEX} @.WEB@>@.INITEX@> program reads the latter file, where each string appears as a two-digit decimal length followed by the string itself, and the information is recorded in \TeX's string memory. @<Glob...@>= @!init @!pool←file:alpha←file; {the string-pool file output by \.{TANGLE}} tini @ @d bad←pool(#)==begin wake←up←terminal; write←ln(term←out,#); a←close(pool←file); get←strings←started:=false; return; end @<Read the other strings...@>= name←of←file:=pool←name; {we needn't set |name←length|} if a←open←in(pool←file) then begin c:=false; repeat @<Read one string, but return |false| if the string memory space is getting too tight for comfort@>; until c; a←close(pool←file); get←strings←started:=true; end else bad←pool('! I can''t read TEX.POOL.') @.I can't read TEX.POOL@> @ @<Read one string...@>= begin if eof(pool←file) then bad←pool('! TEX.POOL has no check sum.'); @.TEX.POOL has no check sum@> read(pool←file,m,n); {read two digits of string length} if m='*' then @<Check the pool check sum@> else begin if (xord[m]<"0")or(xord[m]>"9")or@| (xord[n]<"0")or(xord[n]>"9") then bad←pool('! TEX.POOL line doesn''t begin with two digits.'); @.TEX.POOL line doesn't...@> l:=xord[m]*10+xord[n]-"0"*11; {compute the length} if pool←ptr+l+string←vacancies>pool←size then bad←pool('! You have to increase POOLSIZE.'); @.You have to increase POOLSIZE@> for k:=1 to l do begin if eoln(pool←file) then m:=' '@+else read(pool←file,m); append←char(xord[m]); end; read←ln(pool←file); g:=make←string; end; end @ The \.{WEB} operation \.{@@\$} denotes the value that should be at the end of this \.{TEX.POOL} file; any other value means that the wrong pool file has been loaded. @↑check sum@> @<Check the pool check sum@>= begin a:=0; k:=1; loop@+ begin if (xord[n]<"0")or(xord[n]>"9") then bad←pool('! TEX.POOL check sum doesn''t have nine digits.'); @.TEX.POOL check sum...@> a:=10*a+xord[n]-"0"; if k=9 then goto done; incr(k); read(pool←file,n); end; done: if a<>@$ then bad←pool('! TEX.POOL doesn''t match; TANGLE me again.'); @.TEX.POOL doesn't match@> c:=true; end @* \[5] On-line and off-line printing. Messages that are sent to a user's terminal and to the transcript-log file are produced by several `|print|' procedures. These procedures will direct their output to a variety of places, based on the setting of the global variable |selector|, which has the following possible values: \yskip \hang |term←and←log|, the normal setting, prints on the terminal and on the transcript file. \hang |log←only|, prints only on the transcript file. \hang |term←only|, prints only on the terminal. \hang |no←print|, doesn't print at all. This is used only in rare cases before the transcript file is open. \hang |pseudo|, puts output into a cyclic buffer that is used by the |show←context| routine; see that routine for the reasoning behind this curious mode. \hang |new←string|, appends the output to the current string in the string pool. \hang 0 to 15, prints on one of the sixteen files for \.{\\write} output. \yskip \noindent The symbolic names `|term←and←log|', etc., have been assigned numeric codes that satisfy the convenient relations |no←print+1=term←only|, |no←print+2=log←only|, |term←only+2=log←only+1=term←and←log|. Three additional global variables, |tally| and |term←offset| and |file←offset|, record the number of characters that have been printed since they were most recently cleared to zero. We use |tally| to record the length of (possibly very long) stretches of printing; |term←offset| and |file←offset|, on the other hand, keep track of how many characters have appeared so far on the current line that has been output to the terminal or to the transcript file, respectively. @d no←print=16 {|selector| setting that makes data disappear} @d term←only=17 {printing is destined for the terminal only} @d log←only=18 {printing is destined for the transcript file only} @d term←and←log=19 {normal |selector| setting} @d pseudo=20 {special |selector| setting for |show←context|} @d new←string=21 {printing is deflected to the string pool} @d max←selector=21 {highest selector setting} @<Glob...@>= @!log←file : alpha←file; {transcript of \TeX\ session} @!selector : 0..max←selector; {where to print a message} @!dig : array[0..22] of 0..15; {digits in a number being output} @!tally : integer; {the number of characters recently printed} @!term←offset : 0..max←print←line; {the number of characters on the current terminal line} @!file←offset : 0..max←print←line; {the number of characters on the current file line} @!trick←buf:array[0..error←line] of ASCII←code; {circular buffer for pseudoprinting} @!trick←count: integer; {threshold for pseudoprinting, explained later} @!first←count: integer; {another variable for pseudoprinting} @ @<Initialize the output routines@>= selector:=term←only; tally:=0; term←offset:=0; file←offset:=0; @ Macro abbreviations for output to the terminal and to the log file are defined here for convenience. Some systems need special conventions for terminal output, and it is possible to adhere to those conventions by changing |wterm|, |wterm←ln|, and |wterm←cr| in this section. @↑system dependencies@> @d wterm(#)==write(term←out,#) @d wterm←ln(#)==write←ln(term←out,#) @d wterm←cr==write←ln(term←out) @d wlog(#)==write(log←file,#) @d wlog←ln(#)==write←ln(log←file,#) @d wlog←cr==write←ln(log←file) @ To end a line of text output, we call |print←ln|. @<Basic print...@>= procedure print←ln; {prints an end-of-line} begin case selector of term←and←log: begin wterm←cr; wlog←cr; term←offset:=0; file←offset:=0; end; log←only: begin wlog←cr; file←offset:=0; end; term←only: begin wterm←cr; term←offset:=0; end; no←print,pseudo,new←string: do←nothing; othercases write←ln(write←file[selector]) endcases;@/ end; {|tally| is not affected} @ The |print←char| procedure sends one character to the desired destination, using the |xchr| array to map it into an external character compatible with |input←ln|. All printing comes through |print←ln| or |print←char|. @<Basic printing...@>= procedure print←char(@!s:ASCII←code); {prints a single character} label exit; begin if @<Character |s| is the current new-line character@> then if selector<pseudo then begin print←ln; return; end; case selector of term←and←log: begin wterm(xchr[s]); wlog(xchr[s]); incr(term←offset); incr(file←offset); if term←offset=max←print←line then begin wterm←cr; term←offset:=0; end; if file←offset=max←print←line then begin wlog←cr; file←offset:=0; end; end; log←only: begin wlog(xchr[s]); incr(file←offset); if file←offset=max←print←line then print←ln; end; term←only: begin wterm(xchr[s]); incr(term←offset); if term←offset=max←print←line then print←ln; end; no←print: do←nothing; pseudo: if tally<trick←count then trick←buf[tally mod error←line]:=s; new←string: begin if pool←ptr<pool←size then append←char(s); end; {we drop characters if the string space is full} othercases write(write←file[selector],xchr[s]) endcases;@/ incr(tally); exit:end; @ An entire string is output by calling |print|. Note that if we are outputting the single standard ASCII character \.c, we could call |print("c")|, since |"c"=99| is the number of a single-character string, as explained above. But |print←char("c")| is quicker, so \TeX\ goes directly to the |print←char| routine when it knows that this is safe. (The present implementation assumes that it is always safe to print a visible ASCII character.) @↑system dependencies@> @<Basic print...@>= procedure print(@!s:integer); {prints string |s|} label exit; var j:pool←pointer; {current character code position} begin if s>=str←ptr then s:="???" {this can't happen} @.???@> else if s<128 then if s<0 then s:="???" {can't happen} else if (@<Character |s| is the current new-line character@>) then if selector<pseudo then begin print←ln; return; end; j:=str←start[s]; while j<str←start[s+1] do begin print←char(str←pool[j]); incr(j); end; exit:end; @ Control sequence names might contain |ASCII←code| values that can't be printed using |print←char|. Therefore we use |slow←print| for them: @<Basic print...@>= procedure slow←print(@!s:integer); {prints string |s|} label exit; var j:pool←pointer; {current character code position} begin if s>=str←ptr then s:="???" {this can't happen} @.???@> else if s<128 then if s<0 then s:="???" {can't happen} else if (@<Character |s| is the current new-line character@>) then if selector<pseudo then begin print←ln; return; end; j:=str←start[s]; while j<str←start[s+1] do begin print(str←pool[j]); incr(j); end; exit:end; @ Here is the very first thing that \TeX\ prints: a headline that identifies the version number and format package. The |term←offset| variable is temporarily incorrect, but the discrepancy is not serious since we assume that the banner and format identifier together will occupy at most |max←print←line| character positions. @<Initialize the output...@>= wterm(banner); if format←ident=0 then wterm←ln(' (no format preloaded)') else begin print(format←ident); print←ln; end; @ The procedure |print←nl| is like |print|, but it makes sure that the string appears at the beginning of a new line. @<Basic print...@>= procedure print←nl(@!s:str←number); {prints string |s| at beginning of line} begin if ((term←offset>0)and(odd(selector)))or@| ((file←offset>0)and(selector>=log←only)) then print←ln; print(s); end; @ The procedure |print←esc| prints a string that is preceded by the user's escape character (which is usually a backslash). @<Basic print...@>= procedure print←esc(@!s:str←number); {prints escape character, then |s|} var c:integer; {the escape character code} begin @<Set variable |c| to the current escape character@>; if c>=0 then if c<128 then print(c); print(s); end; @ An array of digits in the range |0..15| is printed by |print←the←digs|. @<Basic print...@>= procedure print←digs(@!k:eight←bits); {prints |dig[k-1]|$\,\ldots\,$|dig[0]|} begin while k>0 do begin decr(k); if dig[k]<10 then print←char("0"+dig[k]) else print←char("A"-10+dig[k]); end; end; @ The following procedure, which prints out the decimal representation of a given integer |n|, has been written carefully so that it works properly if |n=0| or if |(-n)| would cause overflow. It does not apply |mod| or |div| to negative arguments, since such operations are not implemented consistently by all \PASCAL\ compilers. @<Basic print...@>= procedure print←int(@!n:integer); {prints an integer in decimal form} var k:0..23; {index to current digit; we assume that $|n|<10↑{23}$} @!m:integer; {used to negate |n| in possibly dangerous cases} begin k:=0; if n<0 then begin print←char("-"); if n>-100000000 then negate(n) else begin m:=-1-n; n:=m div 10; m:=(m mod 10)+1; k:=1; if m<10 then dig[0]:=m else begin dig[0]:=0; incr(n); end; end; end; repeat dig[k]:=n mod 10; n:=n div 10; incr(k); until n=0; print←digs(k); end; @ Here is a trivial procedure to print two digits; it is usually called with a parameter in the range |0<=n<=99|. @p procedure print←two(@!n:integer); {prints two least significant digits} begin n:=abs(n) mod 100; print←char("0"+(n div 10)); print←char("0"+(n mod 10)); end; @ Hexadecimal printing of nonnegative integers is accomplished by |print←hex|. @p procedure print←hex(@!n:integer); {prints a positive integer in hexadecimal form} var k:0..22; {index to current digit; we assume that $0\L n<16↑{22}$} begin k:=0; print←char(""""); repeat dig[k]:=n mod 16; n:=n div 16; incr(k); until n=0; print←digs(k); end; @ In certain situations, \TeX\ prints either a standard visible ASCII character or its hexadecimal ASCII code. @p procedure print←ASCII(@!c:integer); {prints a character or its code} begin if (c>=0) and (c<=127) then print(c) else begin print←char("["); if c<0 then print←int(c)@+else print←hex(c); print←char("]"); end; end; @ Roman numerals are produced by the |print←roman←int| routine. Readers who like puzzles might enjoy trying to figure out how this tricky code works; therefore no explanation will be given. @p procedure print←roman←int(@!n:integer); label exit; var j,@!k: pool←pointer; {mysterious indices into |str←pool|} @!u,@!v: nonnegative←integer; {mysterious numbers} begin j:=str←start["m2d5c2l5x2v5i"]; v:=1000; loop@+ begin while n>=v do begin print←char(str←pool[j]); n:=n-v; end; if n<=0 then return; {nonpositive input produces no output} k:=j+2; u:=v div (str←pool[k-1]-"0"); if str←pool[k-1]="2" then begin k:=k+2; u:=u div (str←pool[k-1]-"0"); end; if n+u>=v then begin print←char(str←pool[k]); n:=n+u; end else begin j:=j+2; v:=v div (str←pool[j-1]-"0"); end; end; exit:end; @ The |print| subroutine will not print a string that is still being created. The following procedure will. @p procedure print←current←string; {prints a yet-unmade string} var j:pool←pointer; {points to current character code} begin j:=str←start[str←ptr]; while j<pool←ptr do begin print←char(str←pool[j]); incr(j); end; end; @ Here is a procedure that asks the user to type a line of input, assuming that the |selector| setting is either |term←only| or |term←and←log|. The input is placed into locations |first| through |last-1| of the |buffer| array, and echoed on the transcript file if appropriate. This procedure is never called when |interaction<scroll←mode|. @d prompt←input(#)==begin wake←up←terminal; print(#); term←input; end {prints a string and gets a line of input} @p procedure term←input; {gets a line from the terminal} var k:0..buf←size; {index into |buffer|} begin update←terminal; {Now the user sees the prompt for sure} if not input←ln(term←in,true) then fatal←error("End of file on the terminal!"); @.End of file on the terminal@> term←offset:=0; {the user's line ended with \<\rm return>} decr(selector); {prepare to echo the input} if last<>first then for k:=first to last-1 do print(buffer[k]); print←ln; incr(selector); {restore previous status} end; @* \[6] Reporting errors. When something anomalous is detected, \TeX\ typically does something like this: $$\vbox{\halign{#\hfil\cr |print←err("Something anomalous has been detected");|\cr |help3("This is the first line of my offer to help.")|\cr |("This is the second line. I'm trying to")|\cr |("explain the best way for you to proceed.");|\cr |error;|\cr}}$$ A two-line help message would be given using |help2|, etc.; these informal helps should use simple vocabulary that complements the words used in the official error message that was printed. (Outside of the U.S.A., the help messages should preferably be translated into the local vernacular. Each line of help is at most 60 characters long, in the present implementation, so that |max←print←line| will not be exceeded.) The |print←err| procedure supplies a `\.!' before the official message, and makes sure that the terminal is awake if a stop is going to occur. The |error| procedure supplies a `\..' after the official message, then it shows the location of the error; and if |interaction=error←stop←mode|, it also enters into a dialog with the user, during which time the help message may be printed. @↑system dependencies@> @ The global variable |interaction| has four settings, representing increasing amounts of user interaction: @d batch←mode=0 {omits all stops and omits terminal output} @d nonstop←mode=1 {omits all stops} @d scroll←mode=2 {omits error stops} @d error←stop←mode=3 {stops at every opportunity to interact} @d print←err(#)==begin if interaction=error←stop←mode then wake←up←terminal; print←nl("! "); print(#); end @<Glob...@>= @!interaction:batch←mode..error←stop←mode; {current level of interaction} @ @<Set init...@>=interaction:=error←stop←mode; @ \TeX\ is careful not to call |error| when the print |selector| setting might be unusual. The only possible values of |selector| at the time of error messages are \yskip\hang|no←print| (when |interaction=batch←mode| and |log←file| not yet open); \hang|term←only| (when |interaction>batch←mode| and |log←file| not yet open); \hang|log←only| (when |interaction=batch←mode| and |log←file| is open); \hang|term←and←log| (when |interaction>batch←mode| and |log←file| is open). @<Initialize the print |selector| based on |interaction|@>= if interaction=batch←mode then selector:=no←print@+else selector:=term←only @ A global variable |deletions←allowed| is set |false| if the |get←next| routine is active when |error| is called; this ensures that |get←next| and related routines like |get←token| will never be called recursively. @↑recursion@> The global variable |history| records the worst level of error that has been detected. It has four possible values: |spotless|, |warning←issued|, |error←message←issued|, and |fatal←error←stop|. Another global variable, |error←count|, is increased by one when an |error| occurs without an interactive dialog, and it is reset to zero at the end of every paragraph. If |error←count| reaches 100, \TeX\ decides that there is no point in continuing further. @d spotless=0 {|history| value when nothing has been amiss yet} @d warning←issued=1 {|history| value when |begin←diagnostic| has been called} @d error←message←issued=2 {|history| value when |error| has been called} @d fatal←error←stop=3 {|history| value when termination was premature} @<Glob...@>= @!deletions←allowed:boolean; {is it safe for |error| to call |get←token|?} @!history:spotless..fatal←error←stop; {has the source input been clean so far?} @!error←count:-1..100; {the number of scrolled errors since the last paragraph ended} @ The value of |history| is initially |fatal←error←stop|, but it will be changed to |spotless| if \TeX\ survives the initialization process. @<Set init...@>= deletions←allowed:=true; error←count:=0; {|history| is initialized elsewhere} @ Since errors can be detected almost anywhere in \TeX, we want to declare the error procedures near the beginning of the program. But the error procedures in turn use some other procedures, which need to be declared |forward| before we get to |error| itself. It is possible for |error| to be called recursively if some error arises when |get←token| is being used to delete a token, or if some fatal error occurs while \TeX\ is trying to fix a non-fatal one. But such recursion @↑recursion@> is never more than one level deep. @<Error handling...@>= procedure@?normalize←selector; forward;@t\2@>@/ procedure@?get←token; forward;@t\2@>@/ procedure@?term←input; forward;@t\2@>@/ procedure@?show←context; forward;@t\2@>@/ procedure@?begin←file←reading; forward;@t\2@>@/ procedure@?open←log←file; forward;@t\2@>@/ procedure@?close←files←and←terminate; forward;@t\2@>@/ procedure@?clear←for←error←prompt; forward;@t\2@>@/ procedure@?give←err←help; forward;@t\2@>@/ @t\4\hskip-\fontdimen2\font@>@;@+@!debug@+procedure@?debug←help; forward;@;@+gubed @ Individual lines of help are recorded in the array |help←line|, which contains entries in positions |0..(help←ptr-1)|. They should be printed in reverse order, i.e., with |help←line[0]| last. @d hlp1(#)==help←line[0]:=#;@+end @d hlp2(#)==help←line[1]:=#; hlp1 @d hlp3(#)==help←line[2]:=#; hlp2 @d hlp4(#)==help←line[3]:=#; hlp3 @d hlp5(#)==help←line[4]:=#; hlp4 @d hlp6(#)==help←line[5]:=#; hlp5 @d help0==help←ptr:=0 {sometimes there might be no help} @d help1==@+begin help←ptr:=1; hlp1 {use this with one help line} @d help2==@+begin help←ptr:=2; hlp2 {use this with two help lines} @d help3==@+begin help←ptr:=3; hlp3 {use this with three help lines} @d help4==@+begin help←ptr:=4; hlp4 {use this with four help lines} @d help5==@+begin help←ptr:=5; hlp5 {use this with five help lines} @d help6==@+begin help←ptr:=6; hlp6 {use this with six help lines} @<Glob...@>= @!help←line:array[0..5] of str←number; {helps for the next |error|} @!help←ptr:0..6; {the number of help lines present} @!use←err←help:boolean; {should the |err←help| list be shown?} @ @<Set init...@>= help←ptr:=0; use←err←help:=false; @ The |jump←out| procedure just cuts across all active procedure levels and goes to |end←of←TEX|. This is the only nonlocal |@!goto| statement in the whole program. It is used when there is no recovery from a particular error. Some \PASCAL\ compilers do not implement non-local |goto| statements. @↑system dependencies@> In such cases the body of |jump←out| should simply be `|close←files←and←terminate|;\thinspace' followed by a call on some system procedure that quietly terminates the program. @<Error hand...@>= procedure jump←out; begin goto end←of←TEX; end; @ Here now is the general |error| routine. @<Error hand...@>= procedure error; {completes the job of error reporting} label continue,exit; var c:ASCII←code; {what the user types} @!s1,@!s2,@!s3,@!s4:integer; {used to save global variables when deleting tokens} begin if history<error←message←issued then history:=error←message←issued; print←char("."); show←context; if interaction=error←stop←mode then @<Get user's advice and |return|@>; incr(error←count); if error←count=100 then begin print←nl("(That makes 100 errors; please try again.)"); @.That makes 100 errors...@> history:=fatal←error←stop; jump←out; end; @<Put help message on the transcript file@>; exit:end; @ @<Get user's advice...@>= loop@+begin continue: clear←for←error←prompt; prompt←input("? "); @.?@> if last=first then return; c:=buffer[first]; if c>="a" then c:=c+"A"-"a"; {convert to uppercase} @<Interpret code |c| and |return| if done@>; end @ It is desirable to provide an `\.E' option here that gives the user an easy way to return from \TeX\ to the system editor, with the offending line ready to be edited. But such an extension requires some system wizardry, so the present implementation simply types out what file should be edited and the relevant line number. @↑system dependencies@> There is a secret `\.D' option available when the debugging routines have not been commented out. @↑debugging@> @<Interpret code |c| and |return| if done@>= case c of "0","1","2","3","4","5","6","7","8","9": if deletions←allowed then @<Delete |c-"0"| tokens and |goto continue|@>; @t\4\4@>@;@+@!debug "D": begin debug←help; goto continue;@+end;@+gubed@/ "E": if base←ptr>0 then begin print←nl("You want to edit file "); @.You want to edit file x@> print(input←stack[base←ptr].name←field); print(" at line "); print←int(line); interaction:=scroll←mode; jump←out; end; "H": @<Print the help information and |goto continue|@>; "I":@<Introduce new material from the terminal and |return|@>; "Q","R","S":@<Change the interaction level and |return|@>; "X":begin interaction:=scroll←mode; jump←out; end; othercases do←nothing endcases;@/ @<Print the menu of available options@> @ @<Print the menu...@>= begin print("Type <return> to proceed, S to scroll future error messages,");@/ @.Type <return> to proceed...@> print←nl("R to run without stopping, Q to run quietly,");@/ print←nl("I to insert something, "); if base←ptr>0 then print("E to edit your file,"); if deletions←allowed then print←nl("1 or ... or 9 to ignore the next 1 to 9 tokens of input,"); print←nl("H for help, X to quit."); end @ Here the author of \TeX\ apologizes for making use of the numerical relation between |"Q"|, |"R"|, |"S"|, and the desired interaction settings |batch←mode|, |nonstop←mode|, |scroll←mode|. @↑Knuth, Donald Ervin@> @<Change the interaction...@>= begin error←count:=0; interaction:=batch←mode+c-"Q"; print("OK, entering "); case c of "Q":begin print←esc("batchmode"); decr(selector); end; "R":print←esc("nonstopmode"); "S":print←esc("scrollmode"); end; {there are no other cases} print("..."); print←ln; update←terminal; return; end @ When the following code is executed, |buffer[(first+1)..(last-1)]| may contain the material inserted by the user; otherwise another prompt will be given. In order to understand this part of the program fully, you need to be familiar with \TeX's input stacks. @<Introduce new material...@>= begin begin←file←reading; {enter a new syntactic level for terminal input} {now |state=mid←line|, so an initial blank space will count as a blank} if last>first+1 then begin loc:=first+1; buffer[first]:=" "; end else begin prompt←input("insert>"); loc:=first; @.insert>@> end; first:=last; cur←input.limit←field:=last-1; {no |end←line←char| ends this line} return; end @ We allow deletion of up to 99 tokens at a time. @<Delete |c-"0"| tokens...@>= begin s1:=cur←tok; s2:=cur←cmd; s3:=cur←chr; s4:=align←state; align←state:=1000000; OK←to←interrupt:=false; if (last>first+1) and (buffer[first+1]>="0")and(buffer[first+1]<="9") then c:=c*10+buffer[first+1]-"0"*11 else c:=c-"0"; while c>0 do begin get←token; {one-level recursive call of |error| is possible} decr(c); end; cur←tok:=s1; cur←cmd:=s2; cur←chr:=s3; align←state:=s4; OK←to←interrupt:=true; help2("I have just deleted some text, as you asked.")@/ ("You can now delete more, or insert, or whatever."); show←context; goto continue; end @ @<Print the help info...@>= begin if use←err←help then begin give←err←help; use←err←help:=false; end else begin if help←ptr=0 then help2("Sorry, I don't know how to help in this situation.")@/ @t\kern1em@>("Maybe you should try asking a human?"); repeat decr(help←ptr); print(help←line[help←ptr]); print←ln; until help←ptr=0; end; help4("Sorry, I already gave what help I could...")@/ ("Maybe you should try asking a human?")@/ ("An error might have occurred before I noticed any problems.")@/ ("``If all else fails, read the instructions.''");@/ goto continue; end @ @<Put help message on the transcript file@>= if interaction>batch←mode then decr(selector); {avoid terminal output} if use←err←help then begin print←ln; give←err←help; end else while help←ptr>0 do begin decr(help←ptr); print←nl(help←line[help←ptr]); end; print←ln; if interaction>batch←mode then incr(selector); {re-enable terminal output} print←ln @ A dozen or so error messages end with a parenthesized integer, so we save a teeny bit of program space by declaring the following procedure: @p procedure int←error(@!n:integer); begin print(" ("); print←int(n); print←char(")"); error; end; @ In anomalous cases, the print selector might be in an unknown state; the following subroutine is called to fix things just enough to keep running a bit longer. @p procedure normalize←selector; begin if job←name>0 then selector:=term←and←log else selector:=term←only; if interaction=batch←mode then decr(selector); if job←name=0 then open←log←file; end; @ The following procedure prints \TeX's last words before dying. @d succumb==begin if interaction=error←stop←mode then interaction:=scroll←mode; {no more interaction} error; @!debug if interaction>batch←mode then debug←help;@+gubed@;@/ history:=fatal←error←stop; jump←out; {irrecoverable error} end @<Error hand...@>= procedure fatal←error(@!s:str←number); {prints |s|, and that's it} begin normalize←selector;@/ print←err("Emergency stop"); help1(s); succumb; @.Emergency stop@> end; @ Here is the most dreaded error message. @<Error hand...@>= procedure overflow(@!s:str←number;@!n:integer); {stop due to finiteness} begin normalize←selector; print←err("TeX capacity exceeded, sorry ["); @.TeX capacity exceeded ...@> print(s); print←char("="); print←int(n); print←char("]"); help2("If you really absolutely need more capacity,")@/ ("you can ask a wizard to enlarge me."); succumb; end; @ The program might sometime run completely amok, at which point there is no choice but to stop. If no previous error has been detected, that's bad news; a message is printed that is really intended for the \TeX\ maintenance person instead of the user (unless the user has been particularly diabolical). The index entries for `this can't happen' may help to pinpoint the problem. @↑dry rot@> @<Error hand...@>= procedure confusion(@!s:str←number); {consistency check violated; |s| tells where} begin normalize←selector; if history<error←message←issued then begin print←err("This can't happen ("); print(s); print←char(")"); @.This can't happen@> help1("I'm broken. Please show this to someone who can fix can fix"); end else begin print←err("I can't go on meeting you like this"); @.I can't go on...@> help2("One of your earlier faux pas has wounded me deeply,")@/ ("so I'm barely conscious. Please fix it and try again."); end; succumb; end; @ Users occasionally want to interrupt \TeX\ while it's running. If the \PASCAL\ runtime system allows this, one can implement a routine that sets the global variable |interrupt| to some nonzero value when such an interrupt is signalled. Otherwise there is probably at least a way to make |interrupt| nonzero using the \PASCAL\ debugger. @↑system dependencies@> @↑debugging@> @d check←interrupt==begin if interrupt<>0 then pause←for←instructions; end @<Global...@>= @!interrupt:integer; {should \TeX\ pause for instructions?} @!OK←to←interrupt:boolean; {should interrupts be observed?} @ @<Set init...@>= interrupt:=0; OK←to←interrupt:=true; @ When an interrupt has been detected, the program goes into its highest interaction level and lets the user have the full flexibility of the |error| routine. \TeX\ checks for interrupts only at times when it is safe to do this. @p procedure pause←for←instructions; begin if OK←to←interrupt then begin interaction:=error←stop←mode; if (selector=log←only)or(selector=no←print) then incr(selector); print←err("Interruption"); @.Interruption@> help3("You rang?")@/ ("Try to insert some instructions for me (e.g.,`I\showlists'),")@/ ("unless you just want to quit by typing `X'."); deletions←allowed:=false; error; deletions←allowed:=true; interrupt:=0; end; end; @* \[7] Arithmetic with scaled dimensions. The principal computations performed by \TeX\ are done entirely in terms of integers less than $2↑{31}$ in magnitude; and divisions are done only when both dividend and divisor are nonnegative. Thus, the arithmetic specified in this program can be carried out in exactly the same way on a wide variety of computers, including some small ones. Why? Because the arithmetic calculations need to be spelled out precisely in order to guarantee that \TeX\ will produce identical output on different machines. If some quantities were rounded differently in different implementations, we would find that line breaks and even page breaks might occur in different places. Hence the arithmetic of \TeX\ has been designed with care, and systems that claim to be implementations of \TeX82 should follow precisely the @:TeX82}{\TeX82@> calculations as they appear in the present program. (Actually there are three places where \TeX\ uses |div| with a possibly negative numerator. These are harmless; see |div| in the index. Also if the user sets the \.{\\time} or the \.{\\year} to a negative value, some diagnostic information will involve negative-numerator division. The same remarks apply for |mod| as well as for |div|.) @ Here is a routine that calculates half of an integer, using an unambiguous convention with respect to signed odd numbers. @p function half(@!x:integer):integer; begin if odd(x) then half:=(x+1) div 2 else half:=x @!div 2; end; @ Fixed-point arithmetic is done on {\sl scaled integers\/} that are multiples of $2↑{-16}$. In other words, a binary point is assumed to be sixteen bit positions from the right end of a binary computer word. @d unity == @'200000 {$2↑{16}$, represents 1.00000} @d two == @'400000 {$2↑{17}$, represents 2.00000} @<Types...@>= @!scaled = integer; {this type is used for scaled integers} @!nonnegative←integer=0..@'17777777777; {$0\L x<2↑{31}$} @!small←number=0..63; {this type is self-explanatory} @ The following function is used to create a scaled integer from a decimal fraction $(.d←0d←1\ldots d←{k-1})$, where |0<=k<=17|. The digit $d←i$ is given in |dig[i]|, and the calculation produces a correctly rounded result. @p function round←decimals(@!k:small←number) : scaled; {converts a decimal fraction} var a:integer; {the accumulator} begin a:=0; while k>0 do begin decr(k); a:=(a+dig[k]*two) div 10; end; round←decimals:=(a+1) div 2; end; @ Conversely, here is a procedure analogous to |print←int|. If the output of this procedure is subsequently read by \TeX\ and converted by the |round←decimals| routine above, it turns out that the original value will be reproduced exactly; the ``simplest'' such decimal number is output, but there is always at least one digit following the decimal point. The invariant relation in the \&{repeat} loop is that a sequence of decimal digits yet to be printed will yield the original number if and only if they form a fraction~$f$ in the range $s-\delta\L10\cdot2↑{16}f<s$. We can stop if and only if $f=0$ satisfies this condition; the loop will terminate before $s$ can possibly become zero. @p procedure print←scaled(@!s:scaled); {prints scaled real, rounded to five digits} var delta:scaled; {amount of allowable inaccuracy} begin if s<0 then begin print←char("-"); negate(s); {print the sign, if negative} end; print←int(s div unity); {print the integer part} print←char("."); s:=10*(s mod unity)+5; delta:=10; repeat if delta>unity then s:=s+@'100000-(delta div 2); {round the last digit} print←char("0"+(s div unity)); s:=10*(s mod unity); delta:=delta*10; until s<=delta; end; @ Physical sizes that a \TeX\ user specifies for portions of documents are represented internally as scaled points. Thus, if we define an `sp' (scaled @↑sp@> point) as a unit equal to $2↑{-16}$ printer's points, every dimension inside of \TeX\ is an integer number of sp. There are exactly 4,736,286.72 sp per inch. Users are not allowed to specify dimensions larger than $2↑{30}-1$ sp, which is a distance of about 18.892 feet (5.7583 meters); two such quantities can be added without overflow on a 32-bit computer. The present implementation of \TeX\ does not check for overflow when @↑Overflow in arithmetic@> dimensions are added or subtracted. This could be done by inserting a few dozen tests of the form `\ignorespaces|if x>=@'10000000000 then @t\\{report\←overflow}@>|', but the chance of overflow is so remote that such tests do not seem worthwhile. \TeX\ needs to do only a few arithmetic operations on scaled quantities, other than addition and subtraction, and the following subroutines do most of the work. A single computation might use several subroutine calls, and it is desirable to avoid producing multiple error messages in case of arithmetic overflow; so the routines set the global variable |arith←error| to |true| instead of reporting errors directly to the user. Another global variable, |remainder|, holds the remainder after a division. @<Glob...@>= @!arith←error:boolean; {has arithmetic overflow occurred recently?} @!remainder:scaled; {amount subtracted to get an exact division} @ The first arithmetical subroutine we need computes $nx+y$, where |x| and~|y| are |scaled| and |n| is an integer. @p function nx←plus←y(@!n:integer;@!x,@!y:scaled):scaled; begin if n<0 then begin negate(x); negate(n); end; if n=0 then nx←plus←y:=y else if ((x<=(@'7777777777-y) div n)and(-x<=(@'7777777777+y) div n)) then nx←plus←y:=n*x+y else begin arith←error:=true; nx←plus←y:=0; end; end; @ We also need to divide scaled dimensions by integers. @p function x←over←n(@!x:scaled;@!n:integer):scaled; var negative:boolean; {should |remainder| be negated?} begin negative:=false; if n=0 then begin arith←error:=true; x←over←n:=0; remainder:=x; end else begin if n<0 then begin negate(x); negate(n); negative:=true; end; if x>=0 then begin x←over←n:=x div n; remainder:=x mod n; end else begin x←over←n:=-((-x) div n); remainder:=-((-x) mod n); end; end; if negative then negate(remainder); end; @ Then comes the multiplication of a scaled number by a fraction |n/d|, where |n| and |d| are nonnegative integers |<=@t$2↑{16}$@>| and |d| is positive. It would be too dangerous to multiply by~|n| and then divide by~|d|, in separate operations, since overflow might well occur; and it would be too inaccurate to divide by |d| and then multiply by |n|. Hence this subroutine simulates 1.5-precision arithmetic. @p function xn←over←d(@!x:scaled; @!n,@!d:integer):scaled; var positive:boolean; {was |x>=0|?} @!t,@!u,@!v:nonnegative←integer; {intermediate quantities} begin if x>=0 then positive:=true else begin negate(x); positive:=false; end; t:=(x mod @'100000)*n; u:=(x div @'100000)*n+(t div @'100000); v:=(u mod d)*@'100000 + (t mod @'100000); if u div d>=@'100000 then arith←error:=true else u:=@'100000*(u div d) + (v div d); if positive then begin xn←over←d:=u; remainder:=v mod d; end else begin xn←over←d:=-u; remainder:=-(v mod d); end; end; @ The next subroutine is used to compute the ``badness'' of glue, when a total~|t| is supposed to be made from amounts that sum to~|s|. According to {\sl The \TeX book}, the badness of this situation is $100(t/s)↑3$; however, badness is simply a heuristic, so we need not squeeze out the last drop of accuracy when computing it. All we really want is an approximation that has similar properties. @:TeXbook}{\sl The \TeX book@> The actual method used to compute the badness is easier to read from the program than to describe in words. It produces an integer value that is a reasonably close approximation to $100(t/s)↑3$, and all implementations of \TeX\ should use precisely this method. Any badness of $2↑{13}$ or more is treated as infinitely bad, and represented by 10000. It is not difficult to prove that $$\hbox{|badness(t+1,s)>=badness(t,s) >=badness(t,s+1)|}.$$ The badness function defined here is capable of computing at most 1095 distinct values, but that is plenty. @d inf←bad = 10000 {infinitely bad value} @p function badness(@!t,@!s:scaled):halfword; {compute badness, given |t>=0|} var r:integer; {approximation to $\alpha t/s$, where $\alpha↑3\approx 100\cdot2↑{18}$} begin if t=0 then badness:=0 else if s<=0 then badness:=inf←bad else begin if t<=7230584 then r:=(t*297) div s {$297↑3=99.94\times2↑{18}$} else if s>=1663497 then r:=t div (s div 297) else r:=t; if r>1290 then badness:=inf←bad {$1290↑3<2↑{31}<1291↑3$} else badness:=(r*r*r+@'400000) div @'1000000; end; {that was $r↑3/2↑{18}$, rounded to the nearest integer} end; @ When \TeX\ ``packages'' a list into a box, it needs to calculate the proportionality ratio by which the glue inside the box should stretch or shrink. This calculation does not affect \TeX's decision making, so the precise details of rounding, etc., in the glue calculation are not of critical importance for the consistency of results on different computers. We shall use the type |glue←ratio| for such proportionality ratios. A glue ratio should take the same amount of memory as an |integer| (usually 32 bits) if it is to blend smoothly with \TeX's other data structures. Thus |glue←ratio| should be equivalent to |short←real| in some implementations of \PASCAL. Alternatively, it is possible to deal with glue ratios using nothing but fixed-point arithmetic; see {\sl TUGboat \bf3},1 (February 1982), 10--27. (But the routines cited there must be modified to allow negative glue ratios.) @↑system dependencies@> @d set←glue←ratio←zero(#) == #:=0.0 {store the representation of zero ratio} @d set←glue←ratio←one(#) == #:=1.0 {store the representation of unit ratio} @d float(#) == # {convert from |glue←ratio| to type |real|} @d unfloat(#) == # {convert from |real| to type |glue←ratio|} @d float←constant(#) == #.0 {convert |integer| constant to |real|} @<Types...@>= @!glue←ratio=real; {one-word representation of a glue expansion factor} @* \[8] Packed data. In order to make efficient use of storage space, \TeX\ bases its major data structures on a |memory←word|, which contains either a (signed) integer, possibly scaled, or an (unsigned) |glue←ratio|, or a small number of fields that are one half or one quarter of the size used for storing integers. If |x| is a variable of type |memory←word|, it contains up to four fields that can be referred to as follows: $$\vbox{\halign{\hfil#&#\hfil&#\hfil\cr |x|&.|int|&(an |integer|)\cr |x|&.|sc|\qquad&(a |scaled| integer)\cr |x|&.|gr|&(a |glue←ratio|)\cr |x.hh.lh|, |x.hh|&.|rh|&(two halfword fields)\cr |x.hh.b0|, |x.hh.b1|, |x.hh|&.|rh|&(two quarterword fields, one halfword field)\cr |x.qqqq.b0|, |x.qqqq.b1|, |x.qqqq|&.|b2|, |x.qqqq.b3|\hskip-100pt &\qquad\qquad\qquad(four quarterword fields)\cr}}$$ This is somewhat cumbersome to write, and not very readable either, but macros will be used to make the notation shorter and more transparent. The \PASCAL\ code below gives a formal definition of |memory←word| and its subsidiary types, using packed variant records. \TeX\ makes no assumptions about the relative positions of the fields within a word. Since we are assuming 32-bit integers, a halfword must contain at least 16 bits, and a quarterword must contain at least 8 bits. @↑system dependencies@> But it doesn't hurt to have more bits; for example, with enough 36-bit words you might be able to have |mem←max| as large as 262142, which is eight times as much memory as anybody had during the first four years of \TeX's existence. N.B.: Valuable memory space will be dreadfully wasted unless \TeX\ is compiled by a \PASCAL\ that packs all of the |memory←word| variants into the space of a single integer. This means, for example, that |glue←ratio| words should be |short←real| instead of |real| on some computers. Some \PASCAL\ compilers will pack an integer whose subrange is `|0..255|' into an eight-bit field, but others insist on allocating space for an additional sign bit; on such systems you can get 256 values into a quarterword only if the subrange is `|-128..127|'. The present implementation tries to accommodate as many variations as possible, so it makes rather general assumptions. If integers having the subrange `|min←quarterword..max←quarterword|' can be packed into a quarterword, and if integers having the subrange `|min←halfword..max←halfword|' can be packed into a halfword, everything should work satisfactorily. It is usually most efficient to have |min←quarterword=min←halfword=0|, so one should try to achieve this unless it causes a severe problem. The values defined here are recommended for most 32-bit computers. @d min←quarterword=0 {smallest allowable value in a |quarterword|} @d max←quarterword=255 {largest allowable value in a |quarterword|} @d min←halfword==0 {smallest allowable value in a |halfword|} @d max←halfword==65535 {largest allowable value in a |halfword|} @ Here are the inequalities that the quarterword and halfword values must satisfy (or rather, the inequalities that they mustn't satisfy): @<Check the ``constant''...@>= init if (mem←min<>mem←bot)or(mem←max<>mem←top) then bad:=10;@+tini@;@/ if (mem←min>mem←bot)or(mem←max<mem←top) then bad:=10; if (min←quarterword>0)or(max←quarterword<127) then bad:=11; if (min←halfword>0)or(max←halfword<32767) then bad:=12; if (min←quarterword<min←halfword)or@| (max←quarterword>max←halfword) then bad:=13; if (mem←min<min←halfword)or(mem←max>=max←halfword) then bad:=14; if (font←base<min←quarterword)or(font←max>max←quarterword) then bad:=15; if font←max>font←base+256 then bad:=16; if (save←size>max←halfword)or(max←strings>max←halfword) then bad:=17; if buf←size>max←halfword then bad:=18; if max←quarterword-min←quarterword<255 then bad:=19; @ The operation of adding or subtracting |min←quarterword| occurs quite frequently in \TeX, so it is convenient to abbreviate this operation by using the macros |qi| and |qo| for input and output to and from quarterword format. The inner loop of \TeX\ will run faster with respect to compilers that don't optimize expressions like `|x+0|' and `|x-0|', if these macros are simplified in the obvious way when |min←quarterword=0|. @↑inner loop@>@↑system dependencies@> @d qi(#)==#+min←quarterword {to put an |eight←bits| item into a quarterword} @d qo(#)==#-min←quarterword {to take an |eight←bits| item out of a quarterword} @d hi(#)==#+min←halfword {to put a sixteen-bit item into a halfword} @d ho(#)==#-min←halfword {to take a sixteen-bit item from a halfword} @ The reader should study the following definitions closely: @↑system dependencies@> @d sc==int {|scaled| data is equivalent to |integer|} @<Types...@>= @!quarterword = min←quarterword..max←quarterword; {1/4 of a word} @!halfword=min←halfword..max←halfword; {1/2 of a word} @!two←choices = 1..2; {used when there are two variants in a record} @!four←choices = 1..4; {used when there are four variants in a record} @!two←halves = packed record@;@/ @!rh:halfword; case two←choices of 1: (@!lh:halfword); 2: (@!b0:quarterword; @!b1:quarterword); end; @!four←quarters = packed record@;@/ @!b0:quarterword; @!b1:quarterword; @!b2:quarterword; @!b3:quarterword; end; @!memory←word = record@;@/ case four←choices of 1: (@!int:integer); 2: (@!gr:glue←ratio); 3: (@!hh:two←halves); 4: (@!qqqq:four←quarters); end; @!word←file = file of memory←word; @ When debugging, we may want to print a |memory←word| without knowing what type it is; so we print it in all modes. @↑dirty \PASCAL@>@↑debugging@> @p @!debug procedure print←word(@!w:memory←word); {prints |w| in all ways} begin print←int(w.int); print←char(" ");@/ print←scaled(w.sc); print←char(" ");@/ print←scaled(round(unity*float(w.gr))); print←ln;@/ @↑real multiplication@> print←int(w.hh.lh); print←char("="); print←int(w.hh.b0); print←char(":"); print←int(w.hh.b1); print←char(";"); print←int(w.hh.rh); print←char(" ");@/ print←int(w.qqqq.b0); print←char(":"); print←int(w.qqqq.b1); print←char(":"); print←int(w.qqqq.b2); print←char(":"); print←int(w.qqqq.b3); end; gubed @* \[9] Dynamic memory allocation. The \TeX\ system does nearly all of its own memory allocation, so that it can readily be transported into environments that do not have automatic facilities for strings, garbage collection, etc., and so that it can be in control of what error messages the user receives. The dynamic storage requirements of \TeX\ are handled by providing a large array |mem| in which consecutive blocks of words are used as nodes by the \TeX\ routines. Pointer variables are indices into this array, or into another array called |eqtb| that will be explained later. A pointer variable might also be a special flag that lies outside the bounds of |mem|, so we allow pointers to assume any |halfword| value. The minimum halfword value represents a null pointer. \TeX\ does not assume that |mem[null]| exists. @d pointer==halfword {a flag or a location in |mem| or |eqtb|} @d null==min←halfword {the null pointer} @<Glob...@>= @!temp←ptr:pointer; {a pointer variable for occasional emergency use} @ The |mem| array is divided into two regions that are allocated separately, but the dividing line between these two regions is not fixed; they grow together until finding their ``natural'' size in a particular job. Locations less than or equal to |lo←mem←max| are used for storing variable-length records consisting of two or more words each. This region is maintained using an algorithm similar to the one described in exercise 2.5--19 of {\sl The Art of Computer Programming}. However, no size field appears in the allocated nodes; the program is responsible for knowing the relevant size when a node is freed. Locations greater than or equal to |hi←mem←min| are used for storing one-word records; a conventional \.{AVAIL} stack is used for allocation in this region. Locations of |mem| between |mem←bot| and |mem←top| may be dumped as part of preloaded format files, by the \.{INITEX} preprocessor. @.INITEX@> Production versions of \TeX\ may extend the memory at both ends in order to provide more space; locations between |mem←min| and |mem←bot| are always used for variable-size nodes, and locations between |mem←top| and |mem←max| are always used for single-word nodes. The key pointers that govern |mem| allocation have a prescribed order: $$\hbox{|null<=mem←min<=mem←bot<lo←mem←max< hi←mem←min<mem←top<=mem←end<=mem←max|.}$$ Empirical tests show that the present implementation of \TeX\ tends to spend about 9\% of its running time allocating nodes, and about 6\% deallocating them after their use. @<Glob...@>= @!mem : array[mem←min..mem←max] of memory←word; {the big dynamic storage area} @!lo←mem←max : pointer; {the largest location of variable-size memory in use} @!hi←mem←min : pointer; {the smallest location of one-word memory in use} @ In order to study the memory requirements of particular applications, it is possible to prepare a version of \TeX\ that keeps track of current and maximum memory usage. When code between the delimiters |@!stat| $\ldots$ |tats| is not ``commented out,'' \TeX\ will run a bit slower but it will report these statistics when |tracing←stats| is sufficiently large. The |stat| $\ldots$ |tats| code also implements diagnostic information for |tracing←paragraphs| and |tracing←pages|. @<Glob...@>= @!var←used, @!dyn←used : integer; {how much memory is in use} @ Let's consider the one-word memory region first, since it's the simplest. The pointer variable |mem←end| holds the highest-numbered location of |mem| that has ever been used. The free locations of |mem| that occur between |hi←mem←min| and |mem←end|, inclusive, are of type |two←halves|, and we write |info(p)| and |link(p)| for the |lh| and |rh| fields of |mem[p]| when it is of this type. The single-word free locations form a linked list $$|avail|,\;\hbox{|link(avail)|},\;\hbox{|link(link(avail))|},\;\ldots$$ terminated by |null|. @d link(#) == mem[#].hh.rh {the |link| field of a memory word} @d info(#) == mem[#].hh.lh {the |info| field of a memory word} @<Glob...@>= @!avail : pointer; {head of the list of available one-word nodes} @!mem←end : pointer; {the last one-word node used in |mem|} @ If memory is exhausted, it might mean that the user has forgotten a right brace. We will define some procedures later that try to help pinpoint the trouble. @p @<Declare the procedure called |show←token←list|@>@/ @<Declare the procedure called |runaway|@> @ The function |get←avail| returns a pointer to a new one-word node whose |link| field is null. However, \TeX\ will halt if there is no more room left. @↑inner loop@> If the available-space list is empty, i.e., if |avail=null|, we try first to increase |mem←end|. If that cannot be done, i.e., if |mem←end=mem←max|, we try to decrease |hi←mem←min|. If that cannot be done, i.e., if |hi←mem←min=low←mem←max+1|, we have to quit. @p function get←avail : pointer; {single-word node allocation} var p:pointer; {the new node being got} begin p:=avail; {get top location in the |avail| stack} if p<>null then avail:=link(avail) {and pop it off} else if mem←end<mem←max then {or go into virgin territory} begin incr(mem←end); p:=mem←end; end else begin decr(hi←mem←min); p:=hi←mem←min; if hi←mem←min<=lo←mem←max then begin runaway; {if memory is exhausted, display possible runaway text} overflow("main memory size",mem←max+1-mem←min); {quit; all one-word nodes are busy} @:TeX capacity exceeded main memory size}{\quad main memory size@> end; end; link(p):=null; {provide an oft-desired initialization of the new node} @!stat incr(dyn←used);@+tats@;{maintain statistics} get←avail:=p; end; @ Conversely, a one-word node is recycled by calling |free←avail|. This routine is part of \TeX's ``inner loop,'' so we want it to be fast. @↑inner loop@> @d free←avail(#)== {single-word node liberation} begin link(#):=avail; avail:=#; @!stat decr(dyn←used);@+tats@/ end @ There's also a |fast←get←avail| routine, which saves the procedure-call overhead at the expense of extra programming. This routine is used in the places that would otherwise account for the most calls of |get←avail|. @↑inner loop@> @d fast←get←avail(#)==@t@>@;@/ begin #:=avail; {avoid |get←avail| if possible, to save time} if #=null then #:=get←avail else begin avail:=link(#); link(#):=null; @!stat incr(dyn←used);@+tats@/ end; end @ The procedure |flush←list(p)| frees an entire linked list of one-word nodes that starts at position |p|. @↑inner loop@> @p procedure flush←list(@!p:pointer); {makes list of single-word nodes available} var q:pointer; {the successor of node |p|} begin while p<>null do begin q:=link(p); free←avail(p); p:=q; end; end; @ The available-space list that keeps track of the variable-size portion of |mem| is a nonempty, doubly-linked circular list of empty nodes, pointed to by the roving pointer |rover|. Each empty node has size 2 or more; the first word contains the special value |max←halfword| in its |link| field and the size in its |info| field; the second word contains the two pointers for double linking. Each nonempty node also has size 2 or more. Its first word is of type |two←halves|\kern-1pt, and its |link| field is never equal to |max←halfword|. Otherwise there is complete flexibility with respect to the contents of its other fields and its other words. (We require |mem←max<max←halfword| because terrible things can happen when |max←halfword| appears in the |link| field of a nonempty node.) @d empty←flag == max←halfword {the |link| of an empty variable-size node} @d is←empty(#) == (link(#)=empty←flag) {tests for empty node} @d node←size == info {the size field in empty variable-size nodes} @d llink(#) == info(#+1) {left link in doubly-linked list of empty nodes} @d rlink(#) == link(#+1) {right link in doubly-linked list of empty nodes} @<Glob...@>= @!rover : pointer; {points to some node in the list of empties} @ A call to |get←node| with argument |s| returns a pointer to a new node of size~|s|, which must be 2~or more. The |link| field of the first word of this new node is set to null. An overflow stop occurs if no suitable space exists. If |get←node| is called with $s=2↑{30}$, it simply merges adjacent free areas and returns the value |max←halfword|. @p function get←node(@!s:integer):pointer; {variable-size node liberation} label found,exit,restart; var p:pointer; {the node currently under inspection} @!q:pointer; {the node physically after node |p|} @!r:integer; {the newly allocated node, or a candidate for this honor} @!t:pointer; {temporary register} begin restart: p:=rover; {start at some free node in the ring} repeat @<Try to allocate within node |p| and its physical successors, and |goto found| if allocation was possible@>; @↑inner loop@> p:=rlink(p); {move to the next node in the ring} until p=rover; {repeat until the whole list has been traversed} if s=@'10000000000 then begin get←node:=max←halfword; return; end; if lo←mem←max+2<hi←mem←min then @<Grow more variable-size memory and |goto restart|@>; overflow("main memory size",mem←max+1-mem←min); {sorry, nothing satisfactory is left} @:TeX capacity exceeded main memory size}{\quad main memory size@> found: link(r):=null; {this node is now nonempty} @!stat var←used:=var←used+s; {maintain usage statistics} tats@;@/ get←node:=r; exit:end; @ The lower part of |mem| grows by 1000 words at a time, unless we are very close to going under. When it grows, we simply link a new node into the available-space list. This method of controlled growth helps to keep the |mem| usage consecutive when \TeX\ is implemented on ``virtual memory'' systems. @↑virtual memory@> @<Grow more variable-size memory and |goto restart|@>= begin if lo←mem←max+1000<hi←mem←min then t:=lo←mem←max+1000 else t:=(lo←mem←max+hi←mem←min+2) div 2; {|lo←mem←max+2<=t<hi←mem←min|} p:=llink(rover); q:=lo←mem←max; rlink(p):=q; llink(rover):=q;@/ rlink(q):=rover; llink(q):=p; link(q):=empty←flag; node←size(q):=t-lo←mem←max;@/ lo←mem←max:=t; link(lo←mem←max):=null; info(lo←mem←max):=null; rover:=q; goto restart; end @ Empirical tests show that the routine in this section performs a node-merging operation about 0.75 times per allocation, on the average, after which it finds that |r>p+1| about 95\% of the time. @<Try to allocate...@>= q:=p+node←size(p); {find the physical successor} @↑inner loop@> while is←empty(q) do {merge node |p| with node |q|} begin t:=rlink(q); if q=rover then rover:=t; llink(t):=llink(q); rlink(llink(q)):=t;@/ q:=q+node←size(q); end; r:=q-s; if r>p+1 then @<Allocate from the top of node |p| and |goto found|@>; if r=p then if ((rlink(p)<>rover) or (llink(p)<>rover)) then @<Allocate entire node |p| and |goto found|@>; node←size(p):=q-p {reset the size in case it grew} @ @<Allocate from the top...@>= begin node←size(p):=r-p; {store the remaining size} @↑inner loop@> rover:=p; {start searching here next time} goto found; end @ Here we delete node |p| from the ring, and let |rover| rove around. @<Allocate entire...@>= begin rover:=rlink(p); t:=llink(p); llink(rover):=t; rlink(t):=rover; goto found; end @ Conversely, when some variable-size node |p| of size |s| is no longer needed, the operation |free←node(p,s)| will make its words available, by inserting |p| as a new empty node just before where |rover| now points. @↑inner loop@> @p procedure free←node(@!p:pointer; @!s:halfword); {variable-size node liberation} var q:pointer; {|llink(rover)|} begin node←size(p):=s; link(p):=empty←flag; q:=llink(rover); llink(p):=q; rlink(p):=rover; {set both links} llink(rover):=p; rlink(q):=p; {insert |p| into the ring} @!stat var←used:=var←used-s;@+tats@;{maintain statistics} end; @ Just before \.{INITEX} writes out the memory, it sorts the doubly linked available space list. The list is probably very short at such times, so a simple insertion sort is used. The smallest available location will be pointed to by |rover|, the next-smallest by |rlink(rover)|, etc. @p @!init procedure sort←avail; {sorts the available variable-size nodes by location} var p,@!q,@!r: pointer; {indices into |mem|} @!old←rover:pointer; {initial |rover| setting} begin p:=get←node(@'10000000000); {merge adjacent free areas} p:=rlink(rover); rlink(rover):=max←halfword; old←rover:=rover; while p<>old←rover do @<Sort |p| into the list starting at |rover| and advance |p| to |rlink(p)|@>; p:=rover; while rlink(p)<>max←halfword do begin llink(rlink(p)):=p; p:=rlink(p); end; rlink(p):=rover; llink(rover):=p; end; tini @ The following |while| loop terminates, since the list that starts at |rover| ends with |max←halfword| during the sorting procedure. @<Sort |p|...@>= if p<rover then begin q:=p; p:=rlink(q); rlink(q):=rover; rover:=q; end else begin q:=rover; while rlink(q)<p do q:=rlink(q); r:=rlink(p); rlink(p):=rlink(q); rlink(q):=p; p:=r; end @* \[10] Data structures for boxes and their friends. From the computer's standpoint, \TeX's chief mission is to create horizontal and vertical lists. We shall now investigate how the elements of these lists are represented internally as nodes in the dynamic memory. A horizontal or vertical list is linked together by |link| fields in the first word of each node. Individual nodes represent boxes, glue, penalties, or special things like discretionary hyphens; because of this variety, some nodes are longer than others, and we must distinguish different kinds of nodes. We do this by putting a `|type|' field in the first word, together with the link and an optional `|subtype|'. @d type(#) == mem[#].hh.b0 {identifies what kind of node this is} @d subtype(#) == mem[#].hh.b1 {secondary identification in some cases} @ A |@!char←node|, which represents a single character, is the most important kind of node because it accounts for the vast majority of all boxes. Special precautions are therefore taken to ensure that a |char←node| does not take up much memory space. Every such node is one word long, and in fact it is identifiable by this property, since other kinds of nodes have at least two words, and they appear in |mem| locations less than |hi←mem←min|. This makes it possible to omit the |type| field in a |char←node|, leaving us room for two bytes that identify a |font| and a |character| within that font. Note that the format of a |char←node| allows for up to 256 different fonts and up to 256 characters per font; but most implementations will probably limit the total number of fonts to fewer than 75 per job, and most fonts will stick to characters less than 128 (since higher codes are accessed outside of math mode only via ligatures and the \.{\\char} operator). Extensions of \TeX\ intended for oriental languages will need even more than $256\times256$ possible characters, when we consider different sizes @↑oriental characters@>@↑Chinese characters@>@↑Japanese characters@> and styles of type. It is suggested that Chinese and Japanese fonts be handled by representing such characters in two consecutive |char←node| entries: The first of these would identify the font and the character dimensions, and it would also link to the second, where the full halfword |info| field would address the desired character. Such an extension of \TeX\ would not be difficult; further details are left to the reader. The saving feature about oriental characters is that most of them have the same box dimensions. In order to make sure that the |character| code fits in a quarterword, \TeX\ adds the quantity |min←quarterword| to the actual code. Character nodes appear only in horizontal lists, never in vertical lists. @d is←char←node(#) == (#>=hi←mem←min) {does the argument point to a |char←node|?} @d font == type {the font code in a |char←node|} @d character == subtype {the character code in a |char←node|} @ An |hlist←node| stands for a box that was made from a horizontal list. Each |hlist←node| is seven words long, and contains the following fields (in addition to the mandatory |type| and |link|, which we shall not mention explicitly when discussing the other node types): The |height| and |width| and |depth| are scaled integers denoting the dimensions of the box. There is also a |shift←amount| field, a scaled integer indicating how much this box should be lowered (if it appears in a horizontal list), or how much it should be moved to the right (if it appears in a vertical list). There is a |list←ptr| field, which points to the beginning of the list from which this box was fabricated; if |list←ptr| is |null|, the box is empty. Finally, there are three fields that represent the setting of the glue: |glue←set(p)| is a word of type |glue←ratio| that represents the proportionality constant for glue setting; |glue←sign(p)| is |stretching| or |shrinking| or |normal| depending on whether or not the glue should stretch or shrink or remain rigid; and |glue←order(p)| specifies the order of infinity to which glue setting applies (|normal|, |fil|, |fill|, or |filll|). The |subtype| field is not used. @d hlist←node=0 {|type| of hlist nodes} @d box←node←size=7 {number of words to allocate for a box node} @d width←offset=1 {position of |width| field in a box node} @d depth←offset=2 {position of |depth| field in a box node} @d height←offset=3 {position of |height| field in a box node} @d width(#) == mem[#+width←offset].sc {width of the box, in sp} @d depth(#) == mem[#+depth←offset].sc {depth of the box, in sp} @d height(#) == mem[#+height←offset].sc {height of the box, in sp} @d shift←amount(#) == mem[#+4].sc {repositioning distance, in sp} @d list←offset=5 {position of |list←ptr| field in a box node} @d list←ptr(#) == link(#+list←offset) {beginning of the list inside the box} @d glue←order(#) == subtype(#+list←offset) {applicable order of infinity} @d glue←sign(#) == type(#+list←offset) {stretching or shrinking} @d normal=0 {the most common case when several cases are named} @d stretching = 1 {glue setting applies to the stretch components} @d shrinking = 2 {glue setting applies to the shrink components} @d glue←offset = 6 {position of |glue←set| in a box node} @d glue←set(#) == mem[#+glue←offset].gr {a word of type |glue←ratio| for glue setting} @ The |new←null←box| function returns a pointer to an |hlist←node| in which all subfields have the values corresponding to `\.{\\hbox\{\}}'. The |subtype| field is set to |min←quarterword|, since that is the |span←count| value that is desired in case this |hlist←node| is changed to an |unset←node|. @p function new←null←box:pointer; {creates a new box node} var p:pointer; {the new node} begin p:=get←node(box←node←size); type(p):=hlist←node; subtype(p):=min←quarterword; width(p):=0; depth(p):=0; height(p):=0; shift←amount(p):=0; list←ptr(p):=null; glue←sign(p):=normal; glue←order(p):=normal; set←glue←ratio←zero(glue←set(p)); new←null←box:=p; end; @ A |vlist←node| is like an |hlist←node| in all respects except that it was made from a vertical list. @d vlist←node=1 {|type| of vlist nodes} @ A |rule←node| stands for a solid black rectangle; it has |width|, |depth|, and |height| fields just as in an |hlist←node|. However, if any of these dimensions is $-2↑{30}$, the actual value will be determined by running the rule up to the boundary of the innermost enclosing box. This is called a ``running dimension.'' The |width| is never running in an hlist; the |height| and |depth| are never running in a vlist. @d rule←node=2 {|type| of rule nodes} @d rule←node←size=4 {number of words to allocate for a rule node} @d null←flag==-@'10000000000 {$-2↑{30}$, signifies a missing item} @d is←running(#) == (#=null←flag) {tests for a running dimension} @ A new rule node is delivered by the |new←rule| function. It makes all the dimensions ``running,'' so you have to change the ones that are not allowed to run. @p function new←rule:pointer; var p:pointer; {the new node} begin p:=get←node(rule←node←size); type(p):=rule←node; subtype(p):=0; {the |subtype| is not used} width(p):=null←flag; depth(p):=null←flag; height(p):=null←flag; new←rule:=p; end; @ Insertions are represented by |ins←node| records, where the |subtype| indicates the corresponding box number. For example, `\.{\\insert 250}' leads to an |ins←node| whose |subtype| is |250+min←quarterword|. The |height| field of an |ins←node| is slightly misnamed; it actually holds the natural height plus depth of the vertical list being inserted. The |depth| field holds the |split←max←depth| to be used in case this insertion is split, and the |split←top←ptr| points to the corresponding |split←top←skip|. The |float←cost| field holds the |floating←penalty| that will be used if this insertion floats to a subsequent page after a split insertion of the same class. There is one more field, the |ins←ptr|, which points to the beginning of the vlist for the insertion. @d ins←node=3 {|type| of insertion nodes} @d ins←node←size=5 {number of words to allocate for an insertion} @d float←cost(#)==mem[#+1].int {the |floating←penalty| to be used} @d ins←ptr(#)==info(#+4) {the vertical list to be inserted} @d split←top←ptr(#)==link(#+4) {the |split←top←skip| to be used} @ A |mark←node| has a |mark←ptr| field that points to the reference count of a token list that contains the user's \.{\\mark} text. This field occupies a full word instead of a halfword, because there's nothing to put in the other halfword; it is easier in \PASCAL\ to use the full word than to risk leaving garbage in the unused half. @d mark←node=4 {|type| of a mark node} @d small←node←size=2 {number of words to allocate for most node types} @d mark←ptr(#)==mem[#+1].int {head of the token list for a mark} @ An |adjust←node|, which occurs only in horizontal lists, specifies material that will be moved out into the surrounding vertical list; i.e., it is used to implement \TeX's `\.{\\vadjust}' operation. The |adjust←ptr| field points to the vlist containing this material. @d adjust←node=5 {|type| of an adjust node} @d adjust←ptr==mark←ptr {vertical list to be moved out of horizontal list} @ A |ligature←node|, which occurs only in horizontal lists, specifies a composite character that was formed from two or more actual characters. The second word of the node, which is called the |lig←char| word, contains |font| and |character| fields just as in a |char←node|. The characters that generated the ligature have not been forgotten, since they are needed for diagnostic messages and for hyphenation; the |lig←ptr| field points to a linked list of character nodes for those characters. @d ligature←node=6 {|type| of a ligature node} @d lig←char(#)==#+1 {the word where the ligature is to be found} @d lig←ptr(#)==link(lig←char(#)) {the list of characters} @ The |new←ligature| function creates a ligature node having given contents of the |font|, |character|, and |lig←ptr| fields. @p function new←ligature(@!f,@!c:quarterword; @!q:pointer):pointer; var p:pointer; {the new node} begin p:=get←node(small←node←size); type(p):=ligature←node; subtype(p):=0; {the |subtype| is not used} font(lig←char(p)):=f; character(lig←char(p)):=c; lig←ptr(p):=q; new←ligature:=p; end; @ A |disc←node|, which occurs only in horizontal lists, specifies a ``dis\-cretion\-ary'' line break. If such a break occurs at node |p|, the text that starts at |pre←break(p)| will precede the break, the text that starts at |post←break(p)| will follow the break, and text that appears in the next |replace←count(p)| nodes will be ignored. For example, an ordinary discretionary hyphen, indicated by `\.{\\-}', yields a |disc←node| with |pre←break| pointing to a |char←node| containing a hyphen, |post←break=null|, and |replace←count=0|. All three of the discretionary texts must be lists that consist entirely of character, kern, box, rule, and ligature nodes. If |pre←break(p)=null|, the |ex←hyphen←penalty| will be charged for this break. Otherwise the |hyphen←penalty| will be charged. The texts will actually be substituted into the list by the line-breaking algorithm if it decides to make the break, and the discretionary node will disappear at that time; thus, the output routine sees only discretionaries that were not chosen. @d disc←node=7 {|type| of a discretionary node} @d replace←count==subtype {how many subsequent nodes to replace} @d pre←break==llink {text that precedes a discretionary break} @d post←break==rlink {text that follows a discretionary break} @p function new←disc:pointer; {creates an empty |disc←node|} var p:pointer; {the new node} begin p:=get←node(small←node←size); type(p):=disc←node; replace←count(p):=0; pre←break(p):=null; post←break(p):=null; new←disc:=p; end; @ A |whatsit←node| is a wild card reserved for extensions to \TeX. The |subtype| field in its first word says what `\\{whatsit}' it is, and implicitly determines the node size (which must be 2 or more) and the format of the remaining words. When a |whatsit←node| is encountered in a list, special actions are invoked; knowledgeable people who are careful not to mess up the rest of \TeX\ are able to make \TeX\ do new things by adding code at the end of the program. For example, there might be a `\TeX nicolor' extension to specify different colors of ink, @↑extensions to \TeX@> and the whatsit node might contain the desired parameters. The present implementation of \TeX\ treats the features associated with `\.{\\write}' and `\.{\\special}' as if they were extensions, in order to illustrate how such routines might be coded. We shall defer further discussion of extensions until the end of this program. @d whatsit←node=8 {|type| of special extension nodes} @ A |math←node|, which occurs only in horizontal lists, appears before and after mathematical formulas. The |subtype| field is |before| before the formula and |after| after it. There is a |width| field, which represents the amount of surrounding space inserted by \.{\\mathsurround}. @d math←node=9 {|type| of a math node} @d before=0 {|subtype| for math node that introduces a formula} @d after=1 {|subtype| for math node that winds up a formula} @p function new←math(@!w:scaled;@!s:small←number):pointer; var p:pointer; {the new node} begin p:=get←node(small←node←size); type(p):=math←node; subtype(p):=s; width(p):=w; new←math:=p; end; @ \TeX\ makes use of the fact that |hlist←node|, |vlist←node|, |rule←node|, |ins←node|, |mark←node|, |adjust←node|, |ligature←node|, |disc←node|, |whatsit←node|, and |math←node| are at the low end of the type codes, by permitting a break at glue in a list if and only if the |type| of the previous node is less than |math←node|. Furthermore, a node is discarded after a break if its type is |math←node| or more. @d precedes←break(#)==(type(#)<math←node) @d non←discardable(#)==(type(#)<math←node) @ A |glue←node| represents glue in a list. However, it is really only a pointer to a separate glue specification, since \TeX\ makes use of the fact that many essentially identical nodes of glue are usually present. If |p| points to a |glue←node|, |glue←ptr(p)| points to another packet of words that specify the stretch and shrink components, etc. Glue nodes also serve to represent leaders; the |subtype| is used to distinguish between ordinary glue (which is called |normal|) and the three kinds of leaders (which are called |a←leaders|, |c←leaders|, and |x←leaders|). The |leader←ptr| field points to a rule node or to a box node containing the leaders; it is set to |null| in ordinary glue nodes. Many kinds of glue are computed from \TeX's ``skip'' parameters, and it is helpful to know which parameter has led to a particular glue node. Therefore the |subtype| is set to indicate the source of glue, whenever it originated as a parameter. We will be defining symbolic names for the parameter numbers later (e.g., |line←skip←code=0|, |baseline←skip←code=1|, etc.); it suffices for now to say that the |subtype| of parametric glue will be the same as the parameter number, plus one. In math formulas there are two more possibilities for the |subtype| in a glue node: |mu←glue| denotes an \.{\\mskip} (where the units are scaled \.{mu} instead of scaled \.{pt}); and |cond←math←glue| denotes the `\.{\\nonscript}' feature that cancels the glue node immediately following if it appears in a subscript. @d glue←node=10 {|type| of node that points to a glue specification} @d cond←math←glue=98 {special |subtype| to suppress glue in the next node} @d mu←glue=99 {|subtype| for math glue} @d a←leaders=100 {|subtype| for aligned leaders} @d c←leaders=101 {|subtype| for centered leaders} @d x←leaders=102 {|subtype| for expanded leaders} @d glue←ptr==llink {pointer to a glue specification} @d leader←ptr==rlink {pointer to box or rule node for leaders} @ A glue specification has a halfword reference count in its first word, @↑reference counts@> representing |null| plus the number of glue nodes that point to it (less one). Note that the reference count appears in the same position as the |link| field in list nodes; this is the field that is initialized to |null| when a node is allocated, and it is also the field that is flagged by |empty←flag| in empty nodes. Glue specifications also contain three |scaled| fields, for the |width|, |stretch|, and |shrink| dimensions. Finally, there are two one-byte fields called |stretch←order| and |shrink←order|; these contain the orders of infinity (|normal|, |fil|, |fill|, or |filll|) corresponding to the stretch and shrink values. @d glue←spec←size=4 {number of words to allocate for a glue specification} @d glue←ref←count(#) == link(#) {reference count of a glue specification} @d stretch(#) == mem[#+2].sc {the stretchability of this glob of glue} @d shrink(#) == mem[#+3].sc {the shrinkability of this glob of glue} @d stretch←order == type {order of infinity for stretching} @d shrink←order == subtype {order of infinity for shrinking} @d fil=1 {first-order infinity} @d fill=2 {second-order infinity} @d filll=3 {third-order infinity} @<Types...@>= @!glue←ord=normal..filll; {infinity to the 0, 1, 2, or 3 power} @ Here is a function that returns a pointer to a copy of a glue spec. The reference count in the copy is |null|, because there is assumed to be exactly one reference to the new specification. @p function new←spec(@!p:pointer):pointer; {duplicates a glue specification} var q:pointer; {the new spec} begin q:=get←node(glue←spec←size);@/ mem[q]:=mem[p]; glue←ref←count(q):=null;@/ width(q):=width(p); stretch(q):=stretch(p); shrink(q):=shrink(p); new←spec:=q; end; @ And here's a function that creates a glue node for a given parameter identified by its code number; for example, |new←param←glue(line←skip←code)| returns a pointer to a glue node for the current \.{\\lineskip}. @p function new←param←glue(@!n:small←number):pointer; var p:pointer; {the new node} @!q:pointer; {the glue specification} begin p:=get←node(small←node←size); type(p):=glue←node; subtype(p):=n+1; leader←ptr(p):=null;@/ q:=@<Current |mem| equivalent of glue parameter number |n|@>@t@>; glue←ptr(p):=q; incr(glue←ref←count(q)); new←param←glue:=p; end; @ Glue nodes that are more or less anonymous are created by |new←glue|, whose argument points to a glue specification. @p function new←glue(@!q:pointer):pointer; var p:pointer; {the new node} begin p:=get←node(small←node←size); type(p):=glue←node; subtype(p):=normal; leader←ptr(p):=null; glue←ptr(p):=q; incr(glue←ref←count(q)); new←glue:=p; end; @ Still another subroutine is needed: this one is sort of a combination of |new←param←glue| and |new←glue|. It creates a glue node for one of the current glue parameters, but it makes a fresh copy of the glue specification, since that specification will probably be subject to change, while the parameter will stay put. The global variable |temp←ptr| is set to the address of the new spec. @p function new←skip←param(@!n:small←number):pointer; var p:pointer; {the new node} begin temp←ptr:=new←spec(@<Current |mem| equivalent of glue parameter...@>); p:=new←glue(temp←ptr); glue←ref←count(temp←ptr):=null; subtype(p):=n+1; new←skip←param:=p; end; @ A |kern←node| has a |width| field to specify a (normally negative) amount of spacing. This spacing correction appears in horizontal lists between letters like A and V when the font designer said that it looks better to move them closer together or further apart. A kern node can also appear in a vertical list, when its `|width|' denotes additional spacing in the vertical direction. The |subtype| is either |normal| (for kerns inserted from font information or math mode calculations) or |explicit| (for kerns inserted from \.{\\kern} and \.{\\/} commands or non-math accents) or |mu←glue| (for kerns inserted from \.{\\mkern} specifications in math formulas). @d kern←node=11 {|type| of a kern node} @d explicit=1 {|subtype| of kern nodes from \.{\\kern} and \.{\\/} and accents} @ The |new←kern| function creates a kern node having a given width. @p function new←kern(@!w:scaled):pointer; var p:pointer; {the new node} begin p:=get←node(small←node←size); type(p):=kern←node; subtype(p):=normal; width(p):=w; new←kern:=p; end; @ A |penalty←node| specifies the penalty associated with line or page breaking, in its |penalty| field. This field is a fullword integer, but the full range of integer values is not used: Any penalty |>=10000| is treated as infinity, and no break will be allowed for such high values. Similarly, any penalty |<=-10000| is treated as negative infinity, and a break will be forced. @d penalty←node=12 {|type| of a penalty node} @d inf←penalty=inf←bad {``infinite'' penalty value} @d eject←penalty=-inf←penalty {``negatively infinite'' penalty value} @d penalty(#) == mem[#+1].int {the added cost of breaking a list here} @ Anyone who has been reading the last few sections of the program will be able to guess what comes next. @p function new←penalty(@!m:integer):pointer; var p:pointer; {the new node} begin p:=get←node(small←node←size); type(p):=penalty←node; subtype(p):=0; {the |subtype| is not used} penalty(p):=m; new←penalty:=p; end; @ You might think that we have introduced enough node types by now. Well, almost, but there is one more: An |unset←node| has nearly the same format as an |hlist←node| or |vlist←node|; it is used for entries in \.{\\halign} or \.{\\valign} that are not yet in their final form, since the box dimensions are their ``natural'' sizes before any glue adjustment has been made. The |glue←set| word is not present; instead, we have a |glue←stretch| field, which contains the total stretch of order |glue←order| that is present in the hlist or vlist being boxed. Similarly, the |shift←amount| field is replaced by a |glue←shrink| field, containing the total shrink of order |glue←sign| that is present. The |subtype| field is called |span←count|; an unset box typically contains the data for |qo(span←count)+1| columns. Unset nodes will be changed to box nodes when alignment is completed. @d unset←node=13 {|type| for an unset node} @d glue←stretch(#)==mem[#+glue←offset].sc {total stretch in an unset node} @d glue←shrink==shift←amount {total shrink in an unset node} @d span←count==subtype {indicates the number of spanned columns} @ In fact, there are still more types coming. When we get to math formula processing we will see that a |style←node| has |type=14|; and a number of larger type codes will also be defined, for use in math mode only. @ Warning: If any changes are made to these data structure layouts, such as changing any of the node sizes or even reordering the words of nodes, the |copy←node←list| procedure and the memory initialization code below may have to be changed. Such potentially dangerous parts of the program are listed in the index under `data structure assumptions'. @!@↑data structure assumptions@> However, other references to the nodes are made symbolically in terms of the \.{WEB} macro definitions above, so that format changes will leave \TeX's other algorithms intact. @↑system dependencies@> @* \[11] Memory layout. Some areas of |mem| are dedicated to fixed usage, since static allocation is more efficient than dynamic allocation when we can get away with it. For example, locations |mem←bot| to |mem←bot+3| are always used to store the specification for glue that is `\.{0pt plus 0pt minus 0pt}'. The following macro definitions accomplish the static allocation by giving symbolic names to the fixed positions. Static variable-size nodes appear in locations |mem←bot| through |lo←mem←stat←max|, and static single-word nodes appear in locations |hi←mem←stat←min| through |mem←top|, inclusive. It is harmless to let |lig←trick|, |garbage|, and |backup←head| share the same location of |mem|. @d zero←glue==mem←bot {specification for \.{0pt plus 0pt minus 0pt}} @d fil←glue==zero←glue+glue←spec←size {\.{0pt plus 1fil minus 0pt}} @d fill←glue==fil←glue+glue←spec←size {\.{0pt plus 1fill minus 0pt}} @d ss←glue==fill←glue+glue←spec←size {\.{0pt plus 1fil minus 1fil}} @d fil←neg←glue==ss←glue+glue←spec←size {\.{0pt plus -1fil minus 0pt}} @d lo←mem←stat←max==fil←neg←glue+glue←spec←size-1 {largest statically allocated word in the variable-size |mem|} @# @d page←ins←head==mem←top {list of insertion data for current page} @d contrib←head==mem←top-1 {vlist of items not yet on current page} @d page←head==mem←top-2 {vlist for current page} @d temp←head==mem←top-3 {head of a temporary list of some kind} @d hold←head==mem←top-4 {head of a temporary list of another kind} @d adjust←head==mem←top-5 {head of adjustment list returned by |hpack|} @d active==mem←top-6 {head of active list in |line←break|, needs two words} @d align←head==mem←top-8 {head of preamble list for alignments} @d end←span==mem←top-9 {tail of spanned-width lists} @d omit←template==mem←top-10 {a constant token list} @d null←list==mem←top-11 {permanently empty list} @d lig←trick==mem←top-12 {a ligature masquerading as a |char←node|} @d garbage==mem←top-12 {used for scrap information} @d backup←head==mem←top-13 {head of token list built by |scan←keyword|} @d hi←mem←stat←min==mem←top-13 {smallest statically allocated word in the one-word |mem|} @ The following code gets |mem| off to a good start, when \TeX\ is initializing itself the slow way. @<Local variables for init...@>= @!k:integer; {index into |mem|, |eqtb|, etc.} @ @<Initialize table entries...@>= for k:=mem←bot+1 to lo←mem←stat←max do mem[k].sc:=0; {all glue dimensions are zeroed} @↑data structure assumptions@> k:=mem←bot;@+while k<=lo←mem←stat←max do {set first words of glue specifications} begin glue←ref←count(k):=null+1; stretch←order(k):=normal; shrink←order(k):=normal; k:=k+glue←spec←size; end; stretch(fil←glue):=unity; stretch←order(fil←glue):=fil;@/ stretch(fill←glue):=unity; stretch←order(fill←glue):=fill;@/ stretch(ss←glue):=unity; stretch←order(ss←glue):=fil;@/ shrink(ss←glue):=unity; shrink←order(ss←glue):=fil;@/ stretch(fil←neg←glue):=-unity; stretch←order(fil←neg←glue):=fil;@/ rover:=lo←mem←stat←max+1; link(rover):=empty←flag; {now initialize the dynamic memory} node←size(rover):=1000; {which is a 1000-word available node} llink(rover):=rover; rlink(rover):=rover;@/ lo←mem←max:=rover+1000; link(lo←mem←max):=null; info(lo←mem←max):=null;@/ for k:=hi←mem←stat←min to mem←top do mem[k]:=mem[lo←mem←max]; {clear list heads} @<Initialize the special list heads and constant nodes@>; avail:=null; mem←end:=mem←top; hi←mem←min:=hi←mem←stat←min; {initialize the one-word memory} var←used:=lo←mem←stat←max+1-mem←bot; dyn←used:=mem←top+1-hi←mem←stat←min; {initialize statistics} @ If \TeX\ is extended improperly, the |mem| array might get screwed up. For example, some pointers might be wrong, or some ``dead'' nodes might not have been freed when the last reference to them disappeared. Procedures |check←mem| and |search←mem| are available to help diagnose such problems. These procedures make use of two arrays called |free| and |was←free| that are present only if \TeX's debugging routines have been included. (You may want to decrease the size of |mem| while you @↑debugging@> are debugging.) @<Glob...@>= @!debug @!free: packed array [mem←min..mem←max] of boolean; {free cells} @t\hskip1em@>@!was←free: packed array [mem←min..mem←max] of boolean; {previously free cells} @t\hskip1em@>@!was←mem←end,@!was←lo←max,@!was←hi←min: pointer; {previous |mem←end|, |lo←mem←max|,and |hi←mem←min|} @t\hskip1em@>@!panicking:boolean; {do we want to check memory constantly?} gubed @ @<Set initial...@>= @!debug was←mem←end:=mem←min; {indicate that everything was previously free} was←lo←max:=mem←min; was←hi←min:=mem←max; panicking:=false; gubed @ Procedure |check←mem| makes sure that the available space lists of |mem| are well formed, and it optionally prints out all locations that are reserved now but were free the last time this procedure was called. @p @!debug procedure check←mem(@!print←locs : boolean); label done1,done2; {loop exits} var p,@!q:pointer; {current locations of interest in |mem|} @!clobbered:boolean; {is something amiss?} begin for p:=mem←min to lo←mem←max do free[p]:=false; {you can probably do this faster} for p:=hi←mem←min to mem←end do free[p]:=false; {ditto} @<Check single-word |avail| list@>; @<Check variable-size |avail| list@>; @<Check flags of unavailable nodes@>; if print←locs then @<Print newly busy locations@>; for p:=mem←min to lo←mem←max do was←free[p]:=free[p]; for p:=hi←mem←min to mem←end do was←free[p]:=free[p]; {|was←free:=free| might be faster} was←mem←end:=mem←end; was←lo←max:=lo←mem←max; was←hi←min:=hi←mem←min; end; gubed @ @<Check single-word...@>= p:=avail; q:=null; clobbered:=false; while p<>null do begin if (p>mem←end)or(p<hi←mem←min) then clobbered:=true else if free[p] then clobbered:=true; if clobbered then begin print←nl("AVAIL list clobbered at "); @.AVAIL list clobbered...@> print←int(q); goto done1; end; free[p]:=true; q:=p; p:=link(q); end; done1: @ @<Check variable-size...@>= p:=rover; q:=null; clobbered:=false; repeat if (p>=lo←mem←max)or(p<mem←min) then clobbered:=true else if (rlink(p)>=lo←mem←max)or(rlink(p)<mem←min) then clobbered:=true else if not(is←empty(p))or(node←size(p)<2)or@| (p+node←size(p)>lo←mem←max)or@| (llink(rlink(p))<>p) then clobbered:=true; if clobbered then begin print←nl("Double-AVAIL list clobbered at "); print←int(q); goto done2; end; for q:=p to p+node←size(p)-1 do {mark all locations free} begin if free[q] then begin print←nl("Doubly free location at "); @.Doubly free location...@> print←int(q); goto done2; end; free[q]:=true; end; q:=p; p:=rlink(p); until p=rover; done2: @ @<Check flags...@>= p:=mem←min; while p<=lo←mem←max do {node |p| should not be empty} begin if is←empty(p) then begin print←nl("Bad flag at "); print←int(p); @.Bad flag...@> end; while (p<=lo←mem←max) and not free[p] do incr(p); while (p<=lo←mem←max) and free[p] do incr(p); end @ @<Print newly busy...@>= begin print←nl("New busy locs:"); for p:=mem←min to lo←mem←max do if not free[p] and ((p>was←lo←max) or was←free[p]) then begin print←char(" "); print←int(p); end; for p:=hi←mem←min to mem←end do if not free[p] and ((p<was←hi←min) or (p>was←mem←end) or was←free[p]) then begin print←char(" "); print←int(p); end; end @ The |search←mem| procedure attempts to answer the question ``Who points to node~|p|?'' In doing so, it fetches |link| and |info| fields of |mem| that might not be of type |two←halves|. Strictly speaking, this is @↑dirty \PASCAL@> undefined in \PASCAL, and it can lead to ``false drops'' (words that seem to point to |p| purely by coincidence). But for debugging purposes, we want to rule out the places that do {\sl not\/} point to |p|, so a few false drops are tolerable. @p @!debug procedure search←mem(@!p:pointer); {look for pointers to |p|} var q:integer; {current position being searched} begin for q:=mem←min to lo←mem←max do begin if link(q)=p then begin print←nl("LINK("); print←int(q); print←char(")"); end; if info(q)=p then begin print←nl("INFO("); print←int(q); print←char(")"); end; end; for q:=hi←mem←min to mem←end do begin if link(q)=p then begin print←nl("LINK("); print←int(q); print←char(")"); end; if info(q)=p then begin print←nl("INFO("); print←int(q); print←char(")"); end; end; @<Search |eqtb| for equivalents equal to |p|@>; @<Search |save←stack| for equivalents that point to |p|@>; @<Search |hyph←list| for pointers to |p|@>; end; gubed @* \[12] Displaying boxes. We can reinforce our knowledge of the data structures just introduced by considering two procedures that display a list in symbolic form. The first of these, called |short←display|, is used in ``overfull box'' messages to give the top-level description of a list. The other one, called |show←node←list|, prints a detailed description of exactly what is in the data structure. The philosophy of |short←display| is to ignore the fine points about exactly what is inside boxes, except that ligatures and discretionary breaks are expanded. As a result, |short←display| is a recursive procedure, but the recursion is never more than one level deep. @↑recursion@> A global variable |font←in←short←display| keeps track of the font code that is assumed to be present when |short←display| begins; deviations from this font will be printed. @<Glob...@>= @!font←in←short←display:integer; {an internal font number} @ Boxes, rules, inserts, whatsits, marks, and things in general that are sort of ``complicated'' are indicated only by printing `\.{[]}'. @p procedure short←display(@!p:integer); {prints highlights of list |p|} var n:integer; {for replacement counts} begin while p>null do begin if is←char←node(p) then begin if p<=mem←end then begin if font(p)<>font←in←short←display then begin if (font(p)<font←base)or(font(p)>font←max) then print←char("*") else @<Print the font identifier for |font(p)|@>; print←char(" "); font←in←short←display:=font(p); end; print←ASCII(qo(character(p))); end; end else @<Print a short indication of the contents of node |p|@>; p:=link(p); end; end; @ @<Print a short indication of the contents of node |p|@>= case type(p) of hlist←node,vlist←node,ins←node,whatsit←node,mark←node,adjust←node, unset←node: print("[]"); rule←node: print←char("|"); glue←node: if glue←ptr(p)<>zero←glue then print←char(" "); math←node: print←char("$"); ligature←node: short←display(lig←ptr(p)); disc←node: begin short←display(pre←break(p)); short←display(post←break(p));@/ n:=replace←count(p); while n>0 do begin if link(p)<>null then p:=link(p); decr(n); end; end; othercases do←nothing endcases @ The |show←node←list| routine requires some auxiliary subroutines: one to print a font-and-character combination, one to print a token list without its reference count, and one to print a rule dimension. @p procedure print←font←and←char(@!p:integer); {prints |char←node| data} begin if p>mem←end then print←esc("CLOBBERED.") else begin if (font(p)<font←base)or(font(p)>font←max) then print←char("*") else @<Print the font identifier for |font(p)|@>; print←char(" "); print←ASCII(qo(character(p))); end; end; @# procedure print←mark(@!p:integer); {prints token list data in braces} begin print←char("{"); if (p<hi←mem←min)or(p>mem←end) then print←esc("CLOBBERED.") else show←token←list(link(p),null,max←print←line-10); print←char("}"); end; @# procedure print←rule←dimen(@!d:scaled); {prints dimension in rule node} begin if is←running(d) then print←char("*") else print←scaled(d); end; @ Then there is a subroutine that prints glue stretch and shrink, possibly followed by the name of finite units: @p procedure print←glue(@!d:scaled;@!order:integer;@!s:str←number); {prints a glue component} begin print←scaled(d); if (order<normal)or(order>filll) then print("foul") else if order>normal then begin print("fil"); while order>fil do begin print←char("l"); decr(order); end; end else if s<>0 then print(s); end; @ The next subroutine prints a whole glue specification. @p procedure print←spec(@!p:integer;@!s:str←number); {prints a glue specification} begin if (p<mem←min)or(p>=lo←mem←max) then print←char("*") else begin print←scaled(width(p)); if s<>0 then print(s); if stretch(p)<>0 then begin print(" plus "); print←glue(stretch(p),stretch←order(p),s); end; if shrink(p)<>0 then begin print(" minus "); print←glue(shrink(p),shrink←order(p),s); end; end; end; @ We also need to declare some procedures that appear later in this documentation. @p @<Declare procedures needed for displaying the elements of mlists@>@; @<Declare the procedure called |print←skip←param|@> @ Since boxes can be inside of boxes, |show←node←list| is inherently recursive, @↑recursion@> up to a given maximum number of levels. The history of nesting is indicated by the current string, which will be printed at the beginning of each line; the length of this string, namely |cur←length|, is the depth of nesting. Recursive calls on |show←node←list| therefore use the following pattern: @d node←list←display(#)== begin append←char("."); show←node←list(#); flush←char; end {|str←room| need not be checked; see |show←box| below} @ A global variable called |depth←threshold| is used to record the maximum depth of nesting for which |show←node←list| will show information. If we have |depth←threshold=0|, for example, only the top level information will be given and no sublists will be traversed. Another global variable, called |breadth←max|, tells the maximum number of items to show at each level; |breadth←max| had better be positive, or you won't see anything. @<Glob...@>= @!depth←threshold : integer; {maximum nesting depth in box displays} @!breadth←max : integer; {maximum number of items shown at the same list level} @ Now we are ready for |show←node←list| itself. This procedure has been written to be ``extra robust'' in the sense that it should not crash or get into a loop even if the data structures have been messed up by bugs in the rest of the program. You can safely call its parent routine |show←box(p)| for arbitrary values of |p| when you are debugging \TeX. However, in the presence of bad data, the procedure may @↑dirty \PASCAL@>@↑debugging@> fetch a |memory←word| whose variant is different from the way it was stored; for example, it might try to read |mem[p].hh| when |mem[p]| contains a scaled integer, if |p| is a pointer that has been clobbered or chosen at random. @p procedure show←node←list(@!p:pointer); {prints a node list symbolically} label exit; var n:integer; {the number of items already printed at this level} @!g:real; {a glue ratio, as a floating point number} begin if cur←length>depth←threshold then begin if p>null then print(" []"); {indicate that there's been some truncation} return; end; n:=0; while p>null do begin print←ln; print←current←string; {display the nesting history} if p>mem←end then {pointer out of range} begin print("Bad link, display aborted."); return; @.Bad link...@> end; incr(n); if n>breadth←max then {time to stop} begin print("etc."); return; @.etc@> end; @<Display node |p|@>; p:=link(p); end; exit: end; @ @<Display node |p|@>= if is←char←node(p) then print←font←and←char(p) else case type(p) of hlist←node,vlist←node,unset←node: @<Display box |p|@>; rule←node: @<Display rule |p|@>; ins←node: @<Display insertion |p|@>; whatsit←node: @<Display the whatsit node |p|@>; glue←node: @<Display glue |p|@>; kern←node: @<Display kern |p|@>; math←node: @<Display math node |p|@>; ligature←node: @<Display ligature |p|@>; penalty←node: @<Display penalty |p|@>; disc←node: @<Display discretionary |p|@>; mark←node: @<Display mark |p|@>; adjust←node: @<Display adjustment |p|@>; @t\4@>@<Cases of |show←node←list| that arise in mlists only@>@; othercases print("Unknown node type!") endcases @ @<Display box |p|@>= begin if type(p)=hlist←node then print←esc("h") else if type(p)=vlist←node then print←esc("v") else print←esc("unset"); print("box("); print←scaled(height(p)); print←char("+"); print←scaled(depth(p)); print(")x"); print←scaled(width(p)); if type(p)=unset←node then @<Display special fields of the unset node |p|@> else begin @<Display the value of |glue←set(p)|@>; if shift←amount(p)<>0 then begin print(", shifted "); print←scaled(shift←amount(p)); end; end; node←list←display(list←ptr(p)); {recursive call} end @ @<Display special fields of the unset node |p|@>= begin if span←count(p)<>min←quarterword then begin print(" ("); print←int(qo(span←count(p))+1); print(" columns)"); end; if glue←stretch(p)<>0 then begin print(", stretch "); print←glue(glue←stretch(p),glue←order(p),0); end; if glue←shrink(p)<>0 then begin print(", shrink "); print←glue(glue←shrink(p),glue←sign(p),0); end; end @ The code will have to change in this place if |glue←ratio| is a structured type instead of an ordinary |real|. Note that this routine should avoid arithmetic errors even if the |glue←set| field holds an arbitrary random value. The following code assumes that a properly formed nonzero |real| number has absolute value $2↑{20}$ or more when it is regarded as an integer; this precaution was adequate to prevent floating point underflow on the author's computer. @↑system dependencies@> @↑dirty \PASCAL@> @<Display the value of |glue←set(p)|@>= g:=float(glue←set(p)); if (g<>float←constant(0))and(glue←sign(p)<>normal) then begin print(", glue set "); if glue←sign(p)=shrinking then print("- "); if abs(mem[p+glue←offset].int)<@'4000000 then print("?.?") else if abs(g)>float←constant(20000) then begin if g>float←constant(0) then print←char(">") else print("< -"); print←glue(20000*unity,glue←order(p),0); end else print←glue(round(unity*g),glue←order(p),0); @↑real multiplication@> end @ @<Display rule |p|@>= begin print←esc("rule("); print←rule←dimen(height(p)); print←char("+"); print←rule←dimen(depth(p)); print(")x"); print←rule←dimen(width(p)); end @ @<Display insertion |p|@>= begin print←esc("insert"); print←int(qo(subtype(p))); print(", natural size "); print←scaled(height(p)); print("; split("); print←spec(split←top←ptr(p),0); print←char(","); print←scaled(depth(p)); print("); float cost "); print←int(float←cost(p)); node←list←display(ins←ptr(p)); {recursive call} end @ @<Display glue |p|@>= if subtype(p)>=a←leaders then @<Display leaders |p|@> else begin print←esc("glue"); if subtype(p)<>normal then begin print←char("("); if subtype(p)<cond←math←glue then print←skip←param(subtype(p)-1) else if subtype(p)=cond←math←glue then print←esc("nonscript") else print←esc("mskip"); print←char(")"); end; if subtype(p)<>cond←math←glue then begin print←char(" "); if subtype(p)<cond←math←glue then print←spec(glue←ptr(p),0) else print←spec(glue←ptr(p),"mu"); end; end @ @<Display leaders |p|@>= begin print←esc(""); if subtype(p)=c←leaders then print←char("c") else if subtype(p)=x←leaders then print←char("x"); print("leaders "); print←spec(glue←ptr(p),0); node←list←display(leader←ptr(p)); {recursive call} end @ An ``explicit'' kern value is indicated implicitly by an explicit space. @<Display kern |p|@>= if subtype(p)<=explicit then begin print←esc("kern"); if subtype(p)=explicit then print←char(" "); print←scaled(width(p)); end else begin print←esc("mkern"); print←scaled(width(p)); print("mu"); end @ @<Display math node |p|@>= begin print←esc("math"); if subtype(p)=before then print("on") else print("off"); if width(p)<>0 then begin print(", surrounded "); print←scaled(width(p)); end; end @ @<Display ligature |p|@>= begin print←font←and←char(lig←char(p)); print(" (ligature "); font←in←short←display:=font(lig←char(p)); short←display(lig←ptr(p)); print←char(")"); end @ @<Display penalty |p|@>= begin print←esc("penalty "); print←int(penalty(p)); end @ The |post←break| list of a discretionary node is indicated by a prefixed `\.{\char'174}' instead of the `\..' before the |pre←break| list. @<Display discretionary |p|@>= begin print←esc("discretionary"); if replace←count(p)>0 then begin print(" replacing "); print←int(replace←count(p)); end; node←list←display(pre←break(p)); {recursive call} append←char("|"); show←node←list(post←break(p)); flush←char; {recursive call} end @ @<Display mark |p|@>= begin print←esc("mark"); print←mark(mark←ptr(p)); end @ @<Display adjustment |p|@>= begin print←esc("vadjust"); node←list←display(adjust←ptr(p)); {recursive call} end @ The recursive machinery is started by calling |show←box|. @↑recursion@> @p procedure show←box(@!p:pointer); begin @<Assign the values |depth←threshold:=show←box←depth| and |breadth←max:=show←box←breadth|@>; if breadth←max<=0 then breadth←max:=5; if pool←ptr+depth←threshold>=pool←size then depth←threshold:=pool←size-pool←ptr-1; {now there's enough room for prefix string} show←node←list(p); {the show starts at |p|} print←ln; end; @* \[13] Destroying boxes. When we are done with a node list, we are obliged to return it to free storage, including all of its sublists. The recursive procedure |flush←node←list| does this for us. @ First, however, we shall consider two non-recursive procedures that do simpler tasks. The first of these, |delete←token←ref|, is called when a pointer to a token list's reference count is being removed. This means that the token list should disappear if the reference count was |null|, otherwise the count should be decreased by one. @↑reference counts@> @d token←ref←count(#) == info(#) {reference count preceding a token list} @p procedure delete←token←ref(@!p:pointer); {|p| points to the reference count of a token list that is losing one reference} begin if token←ref←count(p)=null then flush←list(p) else decr(token←ref←count(p)); end; @ Similarly, |delete←glue←ref| is called when a pointer to a glue specification is being withdrawn. @↑reference counts@> @d fast←delete←glue←ref(#)==@t@>@;@/ begin if glue←ref←count(#)=null then free←node(#,glue←spec←size) else decr(glue←ref←count(#)); end @p procedure delete←glue←ref(@!p:pointer); {|p| points to a glue specification} fast←delete←glue←ref(p); @ Now we are ready to delete any node list, recursively. In practice, the nodes deleted are usually charnodes (about 2/3 of the time), and they are glue nodes in about half of the remaining cases. @↑recursion@> @p procedure flush←node←list(@!p:pointer); {erase list of nodes starting at |p|} label done; {go here when node |p| has been freed} var q:pointer; {successor to node |p|} begin while p<>null do @↑inner loop@> begin q:=link(p); if is←char←node(p) then free←avail(p) else begin case type(p) of hlist←node,vlist←node,unset←node: begin flush←node←list(list←ptr(p)); free←node(p,box←node←size); goto done; end; rule←node: begin free←node(p,rule←node←size); goto done; end; ins←node: begin flush←node←list(ins←ptr(p)); delete←glue←ref(split←top←ptr(p)); free←node(p,ins←node←size); goto done; end; whatsit←node: @<Wipe out the whatsit node |p| and |goto done|@>; glue←node: begin fast←delete←glue←ref(glue←ptr(p)); if leader←ptr(p)<>null then flush←node←list(leader←ptr(p)); end; kern←node,math←node,penalty←node: do←nothing; ligature←node: flush←node←list(lig←ptr(p)); mark←node: delete←token←ref(mark←ptr(p)); disc←node: begin flush←node←list(pre←break(p)); flush←node←list(post←break(p)); end; adjust←node: flush←node←list(adjust←ptr(p)); @t\4@>@<Cases of |flush←node←list| that arise in mlists only@>@; othercases confusion("flushing") @:this can't happen flushing}{\quad flushing@> endcases;@/ free←node(p,small←node←size); done:end; p:=q; end; end; @* \[14] Copying boxes. Another recursive operation that acts on boxes is sometimes needed: The procedure |copy←node←list| returns a pointer to another node list that has the same structure and meaning as the original. Note that since glue specifications and token lists have reference counts, we need not make copies of them. Reference counts can never get too large to fit in a halfword, since each pointer to a node is in a different memory address, and the total number of memory addresses fits in a halfword. @↑recursion@> @↑reference counts@> (Well, there actually are also references from outside |mem|; if the |save←stack| is made arbitrarily large, it would theoretically be possible to break \TeX\ by overflowing a reference count. But who would want to do that?) @d add←token←ref(#)==incr(token←ref←count(#)) {new reference to a token list} @d add←glue←ref(#)==incr(glue←ref←count(#)) {new reference to a glue spec} @ The copying procedure copies words en masse without bothering to look at their individual fields. If the node format changes---for example, if the size is altered, or if some link field is moved to another relative position---then this code may need to be changed too. @↑data structure assumptions@> @p function copy←node←list(@!p:pointer):pointer; {makes a duplicate of the node list that starts at |p| and returns a pointer to the new list} var h:pointer; {temporary head of copied list} @!q:pointer; {previous position in new list} @!r:pointer; {current node being fabricated for new list} @!words:0..5; {number of words remaining to be copied} begin h:=get←avail; q:=h; while p<>null do begin @<Make a copy of node |p| in node |r|@>; link(q):=r; q:=r; p:=link(p); end; link(q):=null; q:=link(h); free←avail(h); copy←node←list:=q; end; @ @<Make a copy of node |p|...@>= words:=1; {this setting occurs in more branches than any other} if is←char←node(p) then r:=get←avail else @<Case statement to copy different types and set |words| to the number of initial words not yet copied@>; while words>0 do begin decr(words); mem[r+words]:=mem[p+words]; end @ @<Case statement to copy...@>= case type(p) of hlist←node,vlist←node,unset←node: begin r:=get←node(box←node←size); mem[r+6]:=mem[p+6]; mem[r+5]:=mem[p+5]; {copy the last two words} list←ptr(r):=copy←node←list(list←ptr(p)); {this affects |mem[r+5]|} words:=5; end; rule←node: begin r:=get←node(rule←node←size); words:=rule←node←size; end; ins←node: begin r:=get←node(ins←node←size); mem[r+4]:=mem[p+4]; add←glue←ref(split←top←ptr(p)); ins←ptr(r):=copy←node←list(ins←ptr(p)); {this affects |mem[r+4]|} words:=ins←node←size-1; end; whatsit←node:@<Make a partial copy of the whatsit node |p| and make |r| point to it; set |words| to the number of initial words not yet copied@>; glue←node: begin r:=get←node(small←node←size); add←glue←ref(glue←ptr(p)); glue←ptr(r):=glue←ptr(p); leader←ptr(r):=copy←node←list(leader←ptr(p)); end; kern←node,math←node,penalty←node: begin r:=get←node(small←node←size); words:=small←node←size; end; ligature←node: begin r:=get←node(small←node←size); mem[lig←char(r)]:=mem[lig←char(p)]; {copy |font| and |character|} lig←ptr(r):=copy←node←list(lig←ptr(p)); end; disc←node: begin r:=get←node(small←node←size); pre←break(r):=copy←node←list(pre←break(p)); post←break(r):=copy←node←list(post←break(p)); end; mark←node: begin r:=get←node(small←node←size); add←token←ref(mark←ptr(p)); words:=small←node←size; end; adjust←node: begin r:=get←node(small←node←size); adjust←ptr(r):=copy←node←list(adjust←ptr(p)); end; {|words=1=small←node←size-1|} othercases confusion("copying") @:this can't happen copying}{\quad copying@> endcases @* \[15] The command codes. Before we can go any further, we need to define symbolic names for the internal code numbers that represent the various commands obeyed by \TeX. These codes are somewhat arbitrary, but not completely so. For example, the command codes for character types are fixed by the language, since a user says, e.g., `\.{\\catcode \`\\\${} = 3}' to make \.{\char'44} a math delimiter, and the command code |math←shift| is equal to~3. Some other codes have been made adjacent so that |case| statements in the program need not consider cases that are widely spaced, or so that |case| statements can be replaced by |if| statements. At any rate, here is the list, for future reference. First come the ``catcode'' commands, several of which share their numeric codes with ordinary commands when the catcode cannot emerge from \TeX's scanning routine. @d escape=0 {escape delimiter (called \.\\ in {\sl The \TeX book\/})} @:TeXbook}{\sl The \TeX book@> @d relax=0 { do nothing ( \.{\\relax} )} @d left←brace=1 {beginning of a group ( \.\{ )} @d right←brace=2 {ending of a group ( \.\} )} @d math←shift=3 {mathematics shift character ( \.\$ )} @d tab←mark=4 {alignment delimiter ( \.\&, \.{\\span} )} @d car←ret=5 {end of line ( |carriage←return|, \.{\\cr}, \.{\\crcr} )} @d out←param=5 {output a macro parameter} @d mac←param=6 {macro parameter symbol ( \.\# )} @d sup←mark=7 {superscript ( \.{\char'136} )} @d sub←mark=8 {subscript ( \.{\char'176} )} @d ignore=9 {characters to ignore ( \.{\↑\↑J} )} @d endv=9 {end of \<v←j> list in alignment template} @d spacer=10 {characters equivalent to blank space ( \.{\ } )} @d letter=11 {characters regarded as letters ( \.{A..Z}, \.{a..z} )} @d other←char=12 {none of the special character types} @d active←char=13 {characters that invoke macros ( \.{\↑\↑[} )} @d par←end=13 {end of paragraph ( \.{\\par} )} @d match=13 {match a macro parameter} @d comment=14 {characters that introduce comments ( \.\% )} @d end←match=14 {end of parameters to macro} @d stop=14 {end of job ( \.{\\end}, \.{\\dump} )} @d invalid←char=15 {characters that shouldn't appear ( \.{\↑\↑?} )} @d delim←num=15 {specify delimiter numerically ( \.{\\delimiter} )} @d max←char←code=15 {largest catcode for individual characters} @ Next are the ordinary run-of-the-mill command codes. Codes that are |min←internal| or more represent internal quantities that might be expanded by `\.{\\the}'. @d char←num=16 {character specified numerically ( \.{\\char} )} @d math←char←num=17 {explicit math code ( \.{\\mathchar} )} @d mark=18 {mark definition ( \.{\\mark} )} @d xray=19 {peek inside of \TeX\ ( \.{\\show}, \.{\\showbox}, etc.~)} @d make←box=20 {make a box ( \.{\\box}, \.{\\copy}, \.{\\hbox}, etc.~)} @d hmove=21 {horizontal motion ( \.{\\moveleft}, \.{\\moveright} )} @d vmove=22 {vertical motion ( \.{\\raise}, \.{\\lower} )} @d un←hbox=23 {unglue a box ( \.{\\unhbox}, \.{\\unhcopy} )} @d un←vbox=24 {unglue a box ( \.{\\unvbox}, \.{\\unvcopy} )} @d remove←item=25 {nullify last item ( \.{\\unpenalty}, \.{\\unkern}, \.{\\unskip} )} @d hskip=26 {horizontal glue ( \.{\\hskip}, \.{\\hfil}, etc.~)} @d vskip=27 {vertical glue ( \.{\\vskip}, \.{\\vfil}, etc.~)} @d mskip=28 {math glue ( \.{\\mskip} )} @d kern=29 {fixed space ( \.{\\kern})} @d mkern=30 {math kern ( \.{\\mkern} )} @d leader←ship=31 {use a box ( \.{\\shipout}, \.{\\leaders}, etc.~)} @d halign=32 {horizontal table alignment ( \.{\\halign} )} @d valign=33 {vertical table alignment ( \.{\\valign} )} @d no←align=34 {temporary escape from alignment ( \.{\\noalign} )} @d vrule=35 {vertical rule ( \.{\\vrule} )} @d hrule=36 {horizontal rule ( \.{\\hrule} )} @d insert=37 {vlist inserted in box ( \.{\\insert} )} @d vadjust=38 {vlist inserted in enclosing paragraph ( \.{\\vadjust} )} @d ignore←spaces=39 {gobble |spacer| tokens ( \.{\\ignorespaces} )} @d after←assignment=40 {save till assignment is done ( \.{\\afterassignment} )} @d after←group=41 {save till group is done ( \.{\\aftergroup} )} @d break←penalty=42 {additional badness ( \.{\\penalty} )} @d start←par=43 {begin paragraph ( \.{\\indent}, \.{\\noindent} )} @d ital←corr=44 {italic correction ( \.{\\/} )} @d accent=45 {attach accent in text ( \.{\\accent} )} @d math←accent=46 {attach accent in math ( \.{\\mathaccent} )} @d discretionary=47 {discretionary texts ( \.{\\-}, \.{\\discretionary} )} @d eq←no=48 {equation number ( \.{\\eqno}, \.{\\leqno} )} @d left←right=49 {variable delimiter ( \.{\\left}, \.{\\right} )} @d math←comp=50 {component of formula ( \.{\\mathbin}, etc.~)} @d limit←switch=51 {diddle limit conventions ( \.{\\displaylimits}, etc.~)} @d above=52 {generalized fraction ( \.{\\above}, \.{\\atop}, etc.~)} @d math←style=53 {style specification ( \.{\\displaystyle}, etc.~)} @d math←choice=54 {choice specification ( \.{\\mathchoice} )} @d non←script=55 {conditional math glue ( \.{\\nonscript} )} @d vcenter=56 {vertically center a vbox ( \.{\\vcenter} )} @d case←shift=57 {force specific case ( \.{\\lowercase}, \.{\\uppercase}~)} @d message=58 {send to user ( \.{\\message}, \.{\\errmessage} )} @d extension=59 {extensions to \TeX\ ( \.{\\write}, \.{\\special}, etc.~)} @d in←stream=60 {files for reading ( \.{\\openin}, \.{\\closein} )} @d begin←group=61 {begin local grouping ( \.{\\begingroup} )} @d end←group=62 {end local grouping ( \.{\\endgroup} )} @d omit=63 {omit alignment template ( \.{\\omit} )} @d ex←space=64 {explicit space ( \.{\\\ } )} @d radical=65 {square root and similar signs ( \.{\\radical} )} @d end←cs←name=66 {end control sequence ( \.{\\endcsname} )} @d min←internal=67 {the smallest code that can follow \.{\\the}} @d char←given=67 {character code defined by \.{\\chardef}} @d math←given=68 {math code defined by \.{\\mathchardef}} @d last←item=69 {most recent item ( \.{\\lastpenalty}, \.{\\lastkern}, \.{\\lastskip} )} @d max←non←prefixed←command=69 {largest command code that can't be \.{\\global}} @ The next codes are special; they all relate to mode-independent assignment of values to \TeX's internal registers or tables. Codes that are |max←internal| or less represent internal quantities that might be expanded by `\.{\\the}'. @d toks←register=70 {token list register ( \.{\\toks} )} @d assign←toks=71 {special token list ( \.{\\output}, \.{\\everypar}, etc.~)} @d assign←int=72 {user-defined integer ( \.{\\tolerance}, \.{\\day}, etc.~)} @d assign←dimen=73 {user-defined length ( \.{\\hsize}, etc.~)} @d assign←glue=74 {user-defined glue ( \.{\\baselineskip}, etc.~)} @d assign←mu←glue=75 {user-defined muglue ( \.{\\thinmuskip}, etc.~)} @d assign←font←dimen=76 {user-defined font dimension ( \.{\\fontdimen} )} @d assign←font←int=77 {user-defined font integer ( \.{\\hyphenchar}, \.{\\skewchar} )} @d set←aux=78 {specify state info ( \.{\\spacefactor}, \.{\\prevdepth} )} @d set←prev←graf=79 {specify state info ( \.{\\prevgraf} )} @d set←page←dimen=80 {specify state info ( \.{\\pagegoal}, etc.~)} @d set←page←int=81 {specify state info ( \.{\\deadcycles}, \.{\\insertpenalties} )} @d set←box←dimen=82 {change dimension of box ( \.{\\wd}, \.{\\ht}, \.{\\dp} )} @d set←shape=83 {specify fancy paragraph shape ( \.{\\parshape} )} @d def←code=84 {define a character code ( \.{\\catcode}, etc.~)} @d def←family=85 {declare math fonts ( \.{\\textfont}, etc.~)} @d set←font=86 {set current font ( font identifiers )} @d def←font=87 {define a font file ( \.{\\font} )} @d register=88 {internal register ( \.{\\count}, \.{\\dimen}, etc.~)} @d max←internal=88 {the largest code that can follow \.{\\the}} @d advance=89 {advance a register or parameter ( \.{\\advance} )} @d multiply=90 {multiply a register or parameter ( \.{\\multiply} )} @d divide=91 {divide a register or parameter ( \.{\\divide} )} @d prefix=92 {qualify a definition ( \.{\\global}, \.{\\long}, \.{\\outer} )} @d let=93 {assign a command code ( \.{\\let}, \.{\\futurelet} )} @d shorthand←def=94 {code definition ( \.{\\chardef}, \.{\\countdef}, etc.~)} @d read←to←cs=95 {read into a control sequence ( \.{\\read} )} @d def=96 {macro definition ( \.{\\def}, \.{\\gdef}, \.{\\xdef}, \.{\\edef} )} @d set←box=97 {set a box ( \.{\\setbox} )} @d hyph←data=98 {hyphenation data ( \.{\\hyphenation}, \.{\\patterns} )} @d set←interaction=99 {define level of interaction ( \.{\\batchmode}, etc.~)} @d max←command=99 {the largest command code seen at |big←switch|} @ The remaining command codes are extra special, since they cannot get through \TeX's scanner to the main control routine. They have been given values higher than |max←command| so that their special nature is easily discernable. The ``expandable'' commands come first. @d undefined←cs=max←command+1 {initial state of most |eq←type| fields} @d expand←after=max←command+2 {special expansion ( \.{\\expandafter} )} @d no←expand=max←command+3 {special nonexpansion ( \.{\\noexpand} )} @d input=max←command+4 {input a source file ( \.{\\input}, \.{\\endinput} )} @d if←test=max←command+5 {conditional text ( \.{\\if}, \.{\\ifcase}, etc.~)} @d fi←or←else=max←command+6 {delimiters for conditionals ( \.{\\else}, etc.~)} @d cs←name=max←command+7 {make a control sequence from tokens ( \.{\\csname} )} @d convert=max←command+8 {convert to text ( \.{\\number}, \.{\\string}, etc.~)} @d the=max←command+9 {expand an internal quantity ( \.{\\the} )} @d top←bot←mark=max←command+10 {inserted mark ( \.{\\topmark}, etc.~)} @d call=max←command+11 {non-long, non-outer control sequence} @d long←call=max←command+12 {long, non-outer control sequence} @d outer←call=max←command+13 {non-long, outer control sequence} @d long←outer←call=max←command+14 {long, outer control sequence} @d end←template=max←command+15 {end of an alignment template} @d dont←expand=max←command+16 {the following token was marked by \.{\\noexpand}} @d glue←ref=max←command+17 {the equivalent points to a glue specification} @d shape←ref=max←command+18 {the equivalent points to a parshape specification} @d box←ref=max←command+19 {the equivalent points to a box node, or is |null|} @d data=max←command+20 {the equivalent is simply a halfword number} @* \[16] The semantic nest. \TeX\ is typically in the midst of building many lists at once. For example, when a math formula is being processed, \TeX\ is in math mode and working on an mlist; this formula has temporarily interrupted \TeX\ from being in horizontal mode and building the hlist of a paragraph; and this paragraph has temporarily interrupted \TeX\ from being in vertical mode and building the vlist for the next page of a document. Similarly, when a \.{\\vbox} occurs inside of an \.{\\hbox}, \TeX\ is temporarily interrupted from working in restricted horizontal mode, and it enters internal vertical mode. The ``semantic nest'' is a stack that keeps track of what lists and modes are currently suspended. At each level of processing we are in one of six modes: \yskip\hang|vmode| stands for vertical mode (the page builder); \hang|hmode| stands for horizontal mode (the paragraph builder); \hang|mmode| stands for displayed formula mode; \hang|-vmode| stands for internal vertical mode (e.g., in a \.{\\vbox}); \hang|-hmode| stands for restricted horizontal mode (e.g., in an \.{\\hbox}); \hang|-mmode| stands for math formula mode (not displayed). \yskip\noindent The mode is temporarily set to zero while processing \.{\\write} texts in the |ship←out| routine. Numeric values are assigned to |vmode|, |hmode|, and |mmode| so that \TeX's ``big semantic switch'' can select the appropriate thing to do by computing the value |abs(mode)+cur←cmd|, where |mode| is the current mode and |cur←cmd| is the current command code. @d vmode=1 {vertical mode} @d hmode=vmode+max←command+1 {horizontal mode} @d mmode=hmode+max←command+1 {math mode} @p procedure print←mode(@!m:integer); {prints the current mode} begin if m>0 then case m div (max←command+1) of 0:print("vertical"); 1:print("horizontal"); 2:print("display math"); end else if m=0 then print("no") else case (-m) div (max←command+1) of 0:print("internal vertical"); 1:print("restricted horizontal"); 2:print("math"); end; print(" mode"); end; @ The state of affairs at any semantic level can be represented by five values: \yskip\hang|mode| is the number representing the semantic mode, as just explained. \yskip\hang|head| is a |pointer| to a list head for the list being built; |link(head)| therefore points to the first element of the list, or to |null| if the list is empty. \yskip\hang|tail| is a |pointer| to the final node of the list being built; thus, |tail=head| if and only if the list is empty. \yskip\hang|prev←graf| is the number of lines of the current paragraph that have already been put into the present vertical list. \yskip\hang|aux| is an auxiliary integer that gives further information that is needed to characterize the situation. \yskip\noindent In vertical mode, |aux| is also known as |prev←depth|; it is the scaled value representing the depth of the previous box, for use in baseline calculations, or it is |<=-1000|pt if the next box on the vertical list is to be exempt from baseline calculations. In horizontal mode, |aux| is also known as |space←factor|; it holds the current space factor used in spacing calculations. In math mode, |aux| is also known as |incompleat←noad|; if not |null|, it points to a record that represents the numerator of a generalized fraction for which the denominator is currently being formed in the current list. There is also a sixth quantity, |mode←line|, which is used to correlate the semantic nest with the user's input; |mode←line| contains the source line number at which the current level of nesting was entered. The negative of this line number is used as the |mode←line| at the level of the user's output routine. The semantic nest is an array called |nest| that holds the |mode|, |head|, |tail|, |prev←graf|, |aux|, and |mode←line| values for all semantic levels below the currently active one. Information about the currently active level is kept in the global quantities |mode|, |head|, |tail|, |prev←graf|, |aux|, and |mode←line|, which live in a \PASCAL\ record that is ready to be pushed onto |nest| if necessary. @d ignore←depth==-65536000 {|prev←depth| value that is ignored} @<Types...@>= @!list←state←record=record@!mode←field:-mmode..mmode; @!head←field,@!tail←field: pointer; @!pg←field,@!aux←field,@!ml←field: integer; end; @ @d mode==cur←list.mode←field {current mode} @d head==cur←list.head←field {header node of current list} @d tail==cur←list.tail←field {final node on current list} @d prev←graf==cur←list.pg←field {number of paragraph lines accumulated} @d aux==cur←list.aux←field {auxiliary data about the current list} @d prev←depth==aux {the name of |aux| in vertical mode} @d space←factor==aux {the name of |aux| in horizontal mode} @d incompleat←noad==aux {the name of |aux| in math mode} @d mode←line==cur←list.ml←field {source file line number at beginning of list} @<Glob...@>= @!nest:array[0..nest←size] of list←state←record; @!nest←ptr:0..nest←size; {first unused location of |nest|} @!max←nest←stack:0..nest←size; {maximum of |nest←ptr| when pushing} @!cur←list:list←state←record; {the ``top'' semantic state} @!shown←mode:-mmode..mmode; {most recent mode shown by \.{\\tracingcommands}} @ Here is a common way to make the current list grow: @d tail←append(#)==begin link(tail):=#; tail:=link(tail); end @ We will see later that the vertical list at the bottom semantic level is split into two parts; the ``current page'' runs from |page←head| to |page←tail|, and the ``contribution list'' runs from |contrib←head| to |tail| of semantic level zero. The idea is that contributions are first formed in vertical mode, then ``contributed'' to the current page (during which time the page-breaking decisions are made). For now, we don't need to know any more details about the page-building process. @<Set init...@>= nest←ptr:=0; max←nest←stack:=0; mode:=vmode; head:=contrib←head; tail:=contrib←head; prev←depth:=ignore←depth; mode←line:=0; prev←graf:=0; shown←mode:=0;@/ @<Start a new current page@>; @ When \TeX's work on one level is interrupted, the state is saved by calling |push←nest|. This routine changes |head| and |tail| so that a new (empty) list is begun; it does not change |mode| or |aux|. @p procedure push←nest; {enter a new semantic level, save the old} begin if nest←ptr>max←nest←stack then begin max←nest←stack:=nest←ptr; if nest←ptr=nest←size then overflow("semantic nest size",nest←size); @:TeX capacity exceeded semantic nest size}{\quad semantic nest size@> end; nest[nest←ptr]:=cur←list; {stack the record} incr(nest←ptr); head:=get←avail; tail:=head; prev←graf:=0; mode←line:=line; end; @ Conversely, when \TeX\ is finished on the current level, the former state is restored by calling |pop←nest|. This routine will never be called at the lowest semantic level, nor will it be called unless |head| is a node that should be returned to free memory. @p procedure pop←nest; {leave a semantic level, re-enter the old} begin free←avail(head); decr(nest←ptr); cur←list:=nest[nest←ptr]; end; @ Here is a procedure that displays what \TeX\ is working on, at all levels. @p procedure@?print←totals; forward;@t\2@> procedure show←activities; var p:0..nest←size; {index into |nest|} @!m:-mmode..mmode; {mode} @!a:integer; {auxiliary} @!q,@!r:pointer; {for showing the current page} @!t:integer; {ditto} begin nest[nest←ptr]:=cur←list; {put the top level into the array} print←nl(""); print←ln; for p:=nest←ptr downto 0 do begin m:=nest[p].mode←field; a:=nest[p].aux←field; print←nl("### "); print←mode(m); print(" entered at line "); print←int(abs(nest[p].ml←field)); if nest[p].ml←field<0 then print(" (\output routine)"); if p=0 then begin @<Show the status of the current page@>; if link(contrib←head)<>null then print←nl("### recent contributions:"); end; show←box(link(nest[p].head←field)); @<Show the auxiliary field, |a|@>; end; end; @ @<Show the auxiliary...@>= case abs(m) div (max←command+1) of 0: begin print←nl("prevdepth "); if a<=ignore←depth then print("ignored") else print←scaled(a); if nest[p].pg←field<>0 then begin print(", prevgraf "); print←int(nest[p].pg←field); print(" line"); if nest[p].pg←field<>1 then print←char("s"); end; end; 1: begin print←nl("spacefactor "); print←int(a); end; 2: if a<>null then begin print("this will be denominator of:"); show←box(a); end; end {there are no other cases} @* \[17] The table of equivalents. Now that we have studied the data structures for \TeX's semantic routines, let us consider the data structures used by its syntactic routines. In other words, we turn now to the tables that \TeX\ looks at when it is scanning what the user has written. The biggest and most important such table is called |eqtb|. It holds the current ``equivalents'' of things; i.e., it explains what things mean or what their current values are, for all quantities that are subject to the nesting structure provided by \TeX's grouping mechanism. There are six parts to |eqtb|: \yskip\hang 1) |eqtb[active←base..(hash←base-1)]| holds the current equivalents of single-character control sequences. \yskip\hang 2) |eqtb[hash←base..(glue←base-1)]| holds the current equivalents of multiletter control sequences. \yskip\hang 3) |eqtb[glue←base..(local←base-1)]| holds the current equivalents of glue parameters like the current baselineskip. \yskip\hang 4) |eqtb[local←base..(int←base-1)]| holds the current equivalents of local halfword quantities like the current box registers, the current ``catcodes,'' the current font, and a pointer to the current paragraph shape. \yskip\hang 5) |eqtb[int←base..(dimen←base-1)]| holds the current equivalents of fullword integer parameters like the current hyphenation penalty. \yskip\hang 6) |eqtb[dimen←base..eqtb←size]| holds the current equivalents of fullword dimension parameters like the current hsize or amount of hanging indentation. \yskip\noindent Note that, for example, the current amount of baselineskip glue is determined by the setting of a particular location in region~3 of |eqtb|, while the current meaning of the control sequence `\.{\\baselineskip}' (which might have been changed by \.{\\def} or \.{\\let}) appears in region~2. @ Each entry in |eqtb| is a |memory←word|. Most of these words are of type |two←halves|, and subdivided into three fields: \yskip\hang 1) The |eq←level| (a quarterword) is the level of grouping at which this equivalent was defined. If the level is |level←zero|, the equivalent has never been defined; |level←one| refers to the outer level (outside of all groups), and this level is also used for global definitions that never go away. Higher levels are for equivalents that will disappear at the end of their group. @↑global definitions@> \yskip\hang 2) The |eq←type| (another quarterword) specifies what kind of entry this is. There are many types, since each \TeX\ primitive like \.{\\hbox}, \.{\\def}, etc., has its own special code. The list of command codes above includes all possible settings of the |eq←type| field. \yskip\hang 3) The |equiv| (a halfword) is the current equivalent value. This may be a font number, a pointer into |mem|, or a variety of other things. @d eq←level←field(#)==#.hh.b1 @d eq←type←field(#)==#.hh.b0 @d equiv←field(#)==#.hh.rh @d eq←level(#)==eq←level←field(eqtb[#]) {level of definition} @d eq←type(#)==eq←type←field(eqtb[#]) {command code for equivalent} @d equiv(#)==equiv←field(eqtb[#]) {equivalent value} @d level←zero=min←quarterword {level for undefined quantities} @d level←one=level←zero+1 {outermost level for defined quantities} @ Many locations in |eqtb| have symbolic names. The purpose of the next paragraphs is to define these names, and to set up the initial values of the equivalents. In the first region we have 128 equivalents for single-character control sequences, followed by 128 equivalents for ``active characters'' that act as control sequences. Then comes region~2, which corresponds to the hash table that we will define later. The maximum address in this region is used for a dummy control sequence that is perpetually undefined. There also are several locations for control sequences that are perpetually defined (since they are used in error recovery). @d active←base=1 {beginning of region 1, for active character equivalents} @d single←base=active←base+128 {equivalents of one-letter control sequences} @d null←cs=single←base+128 {equivalent of \.{\\csname\\endcsname}} @d hash←base=null←cs+1 {beginning of region 2, for the hash table} @d frozen←control←sequence=hash←base+hash←size {for error recovery} @d frozen←protection=frozen←control←sequence {inaccessible but definable} @d frozen←cr=frozen←control←sequence+1 {permanent `\.{\\cr}'} @d frozen←end←group=frozen←control←sequence+2 {permanent `\.{\\endgroup}'} @d frozen←right=frozen←control←sequence+3 {permanent `\.{\\right}'} @d frozen←fi=frozen←control←sequence+4 {permanent `\.{\\fi}'} @d frozen←end←template=frozen←control←sequence+5 {permanent `\.{\\endtemplate}'} @d frozen←endv=frozen←control←sequence+6 {second permanent `\.{\\endtemplate}'} @d frozen←relax=frozen←control←sequence+7 {permanent `\.{\\relax}'} @d end←write=frozen←control←sequence+8 {permanent `\.{\\endwrite}'} @d frozen←dont←expand=frozen←control←sequence+9 {permanent `\.{\\notexpanded:}'} @d frozen←null←font=frozen←control←sequence+10 {permanent `\.{\\nullfont}'} @d font←id←base=frozen←null←font-font←base {begins table of 257 permanent font identifiers} @d undefined←control←sequence=frozen←null←font+257 {dummy location} @d glue←base=undefined←control←sequence+1 {beginning of region 3} @<Initialize table entries...@>= eq←type(undefined←control←sequence):=undefined←cs; equiv(undefined←control←sequence):=null; eq←level(undefined←control←sequence):=level←zero; for k:=active←base to undefined←control←sequence-1 do eqtb[k]:=eqtb[undefined←control←sequence]; @ Here is a routine that displays the current meaning of an |eqtb| entry in region 1 or~2. (Similar routines for the other regions will appear below.) @<Show equivalent |n|, in region 1 or 2@>= begin sprint←cs(n); print←char("="); print←cmd←chr(eq←type(n),equiv(n)); if eq←type(n)>=call then begin print←char(":"); show←token←list(link(equiv(n)),null,32); end; end @ Region 3 of |eqtb| contains the 256 \.{\\skip} registers, as well as the glue parameters defined here. It is important that the ``muskip'' parameters have larger numbers than the others. @d line←skip←code=0 {interline glue if |baseline←skip| is infeasible} @d baseline←skip←code=1 {desired glue between baselines} @d par←skip←code=2 {extra glue just above a paragraph} @d above←display←skip←code=3 {extra glue just above displayed math} @d below←display←skip←code=4 {extra glue just below displayed math} @d above←display←short←skip←code=5 {glue above displayed math following short lines} @d below←display←short←skip←code=6 {glue below displayed math following short lines} @d left←skip←code=7 {glue at left of justified lines} @d right←skip←code=8 {glue at right of justified lines} @d top←skip←code=9 {glue at top of main pages} @d split←top←skip←code=10 {glue at top of split pages} @d tab←skip←code=11 {glue between aligned entries} @d space←skip←code=12 {glue between words (if not |zero←glue|)} @d xspace←skip←code=13 {glue after sentences (if not |zero←glue|)} @d par←fill←skip←code=14 {glue on last line of paragraph} @d thin←mu←skip←code=15 {thin space in math formula} @d med←mu←skip←code=16 {medium space in math formula} @d thick←mu←skip←code=17 {thick space in math formula} @d glue←pars=18 {total number of glue parameters} @d skip←base=glue←base+glue←pars {table of 256 ``skip'' registers} @d mu←skip←base=skip←base+256 {table of 256 ``muskip'' registers} @d local←base=mu←skip←base+256 {beginning of region 4} @# @d skip(#)==equiv(skip←base+#) {|mem| location of glue specification} @d mu←skip(#)==equiv(mu←skip←base+#) {|mem| location of math glue spec} @d glue←par(#)==equiv(glue←base+#) {|mem| location of glue specification} @d line←skip==glue←par(line←skip←code) @d baseline←skip==glue←par(baseline←skip←code) @d par←skip==glue←par(par←skip←code) @d above←display←skip==glue←par(above←display←skip←code) @d below←display←skip==glue←par(below←display←skip←code) @d above←display←short←skip==glue←par(above←display←short←skip←code) @d below←display←short←skip==glue←par(below←display←short←skip←code) @d left←skip==glue←par(left←skip←code) @d right←skip==glue←par(right←skip←code) @d top←skip==glue←par(top←skip←code) @d split←top←skip==glue←par(split←top←skip←code) @d tab←skip==glue←par(tab←skip←code) @d space←skip==glue←par(space←skip←code) @d xspace←skip==glue←par(xspace←skip←code) @d par←fill←skip==glue←par(par←fill←skip←code) @d thin←mu←skip==glue←par(thin←mu←skip←code) @d med←mu←skip==glue←par(med←mu←skip←code) @d thick←mu←skip==glue←par(thick←mu←skip←code) @<Current |mem| equivalent of glue parameter number |n|@>=glue←par(n) @ Sometimes we need to convert \TeX's internal code numbers into symbolic form. The |print←skip←param| routine gives the symbolic name of a glue parameter. @<Declare the procedure called |print←skip←param|@>= procedure print←skip←param(@!n:integer); begin case n of line←skip←code: print←esc("lineskip"); baseline←skip←code: print←esc("baselineskip"); par←skip←code: print←esc("parskip"); above←display←skip←code: print←esc("abovedisplayskip"); below←display←skip←code: print←esc("belowdisplayskip"); above←display←short←skip←code: print←esc("abovedisplayshortskip"); below←display←short←skip←code: print←esc("belowdisplayshortskip"); left←skip←code: print←esc("leftskip"); right←skip←code: print←esc("rightskip"); top←skip←code: print←esc("topskip"); split←top←skip←code: print←esc("splittopskip"); tab←skip←code: print←esc("tabskip"); space←skip←code: print←esc("spaceskip"); xspace←skip←code: print←esc("xspaceskip"); par←fill←skip←code: print←esc("parfillskip"); thin←mu←skip←code: print←esc("thinmuskip"); med←mu←skip←code: print←esc("medmuskip"); thick←mu←skip←code: print←esc("thickmuskip"); othercases print("[unknown glue parameter!]") endcases; end; @ The symbolic names for glue parameters are put into \TeX's hash table by using the routine called |primitive|, defined below. Let us enter them now, so that we don't have to list all those parameter names anywhere else. @<Put each of \TeX's primitives into the hash table@>= primitive("lineskip",assign←glue,glue←base+line←skip←code);@/ @!@:line←skip←}{\.{\\lineskip} primitive@> primitive("baselineskip",assign←glue,glue←base+baseline←skip←code);@/ @!@:baseline←skip←}{\.{\\baselineskip} primitive@> primitive("parskip",assign←glue,glue←base+par←skip←code);@/ @!@:par←skip←}{\.{\\parskip} primitive@> primitive("abovedisplayskip",assign←glue,glue←base+above←display←skip←code);@/ @!@:above←display←skip←}{\.{\\abovedisplayskip} primitive@> primitive("belowdisplayskip",assign←glue,glue←base+below←display←skip←code);@/ @!@:below←display←skip←}{\.{\\belowdisplayskip} primitive@> primitive("abovedisplayshortskip", assign←glue,glue←base+above←display←short←skip←code);@/ @!@:above←display←short←skip←}{\.{\\abovedisplayshortskip} primitive@> primitive("belowdisplayshortskip", assign←glue,glue←base+below←display←short←skip←code);@/ @!@:below←display←short←skip←}{\.{\\belowdisplayshortskip} primitive@> primitive("leftskip",assign←glue,glue←base+left←skip←code);@/ @!@:left←skip←}{\.{\\leftskip} primitive@> primitive("rightskip",assign←glue,glue←base+right←skip←code);@/ @!@:right←skip←}{\.{\\rightskip} primitive@> primitive("topskip",assign←glue,glue←base+top←skip←code);@/ @!@:top←skip←}{\.{\\topskip} primitive@> primitive("splittopskip",assign←glue,glue←base+split←top←skip←code);@/ @!@:split←top←skip←}{\.{\\splittopskip} primitive@> primitive("tabskip",assign←glue,glue←base+tab←skip←code);@/ @!@:tab←skip←}{\.{\\tabskip} primitive@> primitive("spaceskip",assign←glue,glue←base+space←skip←code);@/ @!@:space←skip←}{\.{\\spaceskip} primitive@> primitive("xspaceskip",assign←glue,glue←base+xspace←skip←code);@/ @!@:xspace←skip←}{\.{\\xspaceskip} primitive@> primitive("parfillskip",assign←glue,glue←base+par←fill←skip←code);@/ @!@:par←fill←skip←}{\.{\\parfillskip} primitive@> primitive("thinmuskip",assign←mu←glue,glue←base+thin←mu←skip←code);@/ @!@:thin←mu←skip←}{\.{\\thinmuskip} primitive@> primitive("medmuskip",assign←mu←glue,glue←base+med←mu←skip←code);@/ @!@:med←mu←skip←}{\.{\\medmuskip} primitive@> primitive("thickmuskip",assign←mu←glue,glue←base+thick←mu←skip←code);@/ @!@:thick←mu←skip←}{\.{\\thickmuskip} primitive@> @ @<Cases of |print←cmd←chr| for symbolic printing of primitives@>= assign←glue,assign←mu←glue: if chr←code<skip←base then print←skip←param(chr←code-glue←base) else if chr←code<mu←skip←base then begin print←esc("skip"); print←int(chr←code-skip←base); end else begin print←esc("muskip"); print←int(chr←code-mu←skip←base); end; @ All glue parameters and registers are initially `\.{0pt plus0pt minus0pt}'. @<Initialize table entries...@>= equiv(glue←base):=zero←glue; eq←level(glue←base):=level←one; eq←type(glue←base):=glue←ref; for k:=glue←base+1 to local←base-1 do eqtb[k]:=eqtb[glue←base]; glue←ref←count(zero←glue):=glue←ref←count(zero←glue)+local←base-glue←base; @ @<Show equivalent |n|, in region 3@>= if n<skip←base then begin print←skip←param(n-glue←base); print←char("="); if n<glue←base+thin←mu←skip←code then print←spec(equiv(n),"pt") else print←spec(equiv(n),"mu"); end else if n<mu←skip←base then begin print←esc("skip"); print←int(n-skip←base); print←char("="); print←spec(equiv(n),"pt"); end else begin print←esc("muskip"); print←int(n-mu←skip←base); print←char("="); print←spec(equiv(n),"mu"); end @ Region 4 of |eqtb| contains the local quantities defined here. The bulk of this region is taken up by five tables that are indexed by seven-bit characters; these tables are important to both the syntactic and semantic portions of \TeX. There are also a bunch of special things like font and token parameters, as well as the tables of \.{\\toks} and \.{\\box} registers. @d par←shape←loc=local←base {specifies paragraph shape} @d output←routine←loc=local←base+1 {points to token list for \.{\\output}} @d every←par←loc=local←base+2 {points to token list for \.{\\everypar}} @d every←math←loc=local←base+3 {points to token list for \.{\\everymath}} @d every←display←loc=local←base+4 {points to token list for \.{\\everydisplay}} @d every←hbox←loc=local←base+5 {points to token list for \.{\\everyhbox}} @d every←vbox←loc=local←base+6 {points to token list for \.{\\everyvbox}} @d every←job←loc=local←base+7 {points to token list for \.{\\everyjob}} @d every←cr←loc=local←base+8 {points to token list for \.{\\everycr}} @d err←help←loc=local←base+9 {points to token list for \.{\\errhelp}} @d toks←base=local←base+10 {table of 256 token list registers} @d box←base=toks←base+256 {table of 256 box registers} @d cur←font←loc=box←base+256 {internal font number outside math mode} @d math←font←base=cur←font←loc+1 {table of 48 math font numbers} @d cat←code←base=math←font←base+48 {table of 128 command codes (the ``catcodes'')} @d lc←code←base=cat←code←base+128 {table of 128 lowercase mappings} @d uc←code←base=lc←code←base+128 {table of 128 uppercase mappings} @d sf←code←base=uc←code←base+128 {table of 128 spacefactor mappings} @d math←code←base=sf←code←base+128 {table of 128 math mode mappings} @d int←base=math←code←base+128 {beginning of region 5} @# @d par←shape←ptr==equiv(par←shape←loc) @d output←routine==equiv(output←routine←loc) @d every←par==equiv(every←par←loc) @d every←math==equiv(every←math←loc) @d every←display==equiv(every←display←loc) @d every←hbox==equiv(every←hbox←loc) @d every←vbox==equiv(every←vbox←loc) @d every←job==equiv(every←job←loc) @d every←cr==equiv(every←cr←loc) @d err←help==equiv(err←help←loc) @d toks(#)==equiv(toks←base+#) @d box(#)==equiv(box←base+#) @d cur←font==equiv(cur←font←loc) @d fam←fnt(#)==equiv(math←font←base+#) @d cat←code(#)==equiv(cat←code←base+#) @d lc←code(#)==equiv(lc←code←base+#) @d uc←code(#)==equiv(uc←code←base+#) @d sf←code(#)==equiv(sf←code←base+#) @d math←code(#)==equiv(math←code←base+#) {Note: |math←code(c)| is the true math code plus |min←halfword|} @<Put each...@>= primitive("output",assign←toks,output←routine←loc); @!@:output←}{\.{\\output} primitive@> primitive("everypar",assign←toks,every←par←loc); @!@:every←par←}{\.{\\everypar} primitive@> primitive("everymath",assign←toks,every←math←loc); @!@:every←math←}{\.{\\everymath} primitive@> primitive("everydisplay",assign←toks,every←display←loc); @!@:every←display←}{\.{\\everydisplay} primitive@> primitive("everyhbox",assign←toks,every←hbox←loc); @!@:every←hbox←}{\.{\\everyhbox} primitive@> primitive("everyvbox",assign←toks,every←vbox←loc); @!@:every←vbox←}{\.{\\everyvbox} primitive@> primitive("everyjob",assign←toks,every←job←loc); @!@:every←job←}{\.{\\everyjob} primitive@> primitive("everycr",assign←toks,every←cr←loc); @!@:every←cr←}{\.{\\everycr} primitive@> primitive("errhelp",assign←toks,err←help←loc); @!@:err←help←}{\.{\\errhelp} primitive@> @ @<Cases of |print←cmd←chr|...@>= assign←toks: if chr←code>=toks←base then begin print←esc("toks"); print←int(chr←code-toks←base); end else case chr←code of output←routine←loc: print←esc("output"); every←par←loc: print←esc("everypar"); every←math←loc: print←esc("everymath"); every←display←loc: print←esc("everydisplay"); every←hbox←loc: print←esc("everyhbox"); every←vbox←loc: print←esc("everyvbox"); every←job←loc: print←esc("everyjob"); every←cr←loc: print←esc("everycr"); othercases print←esc("errhelp") endcases; @ We initialize most things to null or undefined values. An undefined font is represented by the internal code |font←base|. However, the character code tables are given initial values based on the conventional interpretation of ASCII code. These initial values should not be changed when \TeX\ is adapted for use with non-English languages; all changes to the initialization conventions should be made in format packages, not in \TeX\ itself, so that global interchange of formats is possible. @d null←font==font←base @d var←code==@'70000 {math code meaning ``use the current family''} @<Initialize table entries...@>= par←shape←ptr:=null; eq←type(par←shape←loc):=shape←ref; eq←level(par←shape←loc):=level←one;@/ for k:=output←routine←loc to toks←base+255 do eqtb[k]:=eqtb[undefined←control←sequence]; box(0):=null; eq←type(box←base):=box←ref; eq←level(box←base):=level←one; for k:=box←base+1 to box←base+255 do eqtb[k]:=eqtb[box←base]; cur←font:=null←font; eq←type(cur←font←loc):=data; eq←level(cur←font←loc):=level←one;@/ for k:=math←font←base to math←font←base+47 do eqtb[k]:=eqtb[cur←font←loc]; equiv(cat←code←base):=0; eq←type(cat←code←base):=data; eq←level(cat←code←base):=level←one;@/ for k:=cat←code←base+1 to int←base-1 do eqtb[k]:=eqtb[cat←code←base]; for k:=0 to 127 do begin cat←code(k):=other←char; math←code(k):=hi(k); sf←code(k):=1000; end; cat←code(carriage←return):=car←ret; cat←code(" "):=spacer; cat←code("\"):=escape; cat←code("%"):=comment; cat←code(invalid←code):=invalid←char; cat←code(null←code):=ignore; for k:="0" to "9" do math←code(k):=hi(k+var←code); for k:="A" to "Z" do begin cat←code(k):=letter; cat←code(k+"a"-"A"):=letter;@/ math←code(k):=hi(k+var←code+@"100); math←code(k+"a"-"A"):=hi(k+"a"-"A"+var←code+@"100);@/ lc←code(k):=k+"a"-"A"; lc←code(k+"a"-"A"):=k+"a"-"A";@/ uc←code(k):=k; uc←code(k+"a"-"A"):=k;@/ sf←code(k):=999; end; @ @<Show equivalent |n|, in region 4@>= if n=par←shape←loc then begin print←esc("parshape"); print←char("="); if par←shape←ptr=null then print←char("0") else print←int(info(par←shape←ptr)); end else if n<toks←base then begin print←cmd←chr(assign←toks,n); print←char("="); if equiv(n)<>null then show←token←list(link(equiv(n)),null,32); end else if n<box←base then begin print←esc("toks"); print←int(n-toks←base); print←char("="); if equiv(n)<>null then show←token←list(link(equiv(n)),null,32); end else if n<cur←font←loc then begin print←esc("box"); print←int(n-box←base); print←char("="); if equiv(n)=null then print("void") else begin depth←threshold:=0; breadth←max:=1; show←node←list(equiv(n)); end; end else if n<cat←code←base then @<Show the font identifier in |eqtb[n]|@> else @<Show the halfword code in |eqtb[n]|@> @ @<Show the font identifier in |eqtb[n]|@>= begin if n=cur←font←loc then print("current font") else if n<math←font←base+16 then begin print←esc("textfont"); print←int(n-math←font←base); end else if n<math←font←base+32 then begin print←esc("scriptfont"); print←int(n-math←font←base-16); end else begin print←esc("scriptscriptfont"); print←int(n-math←font←base-32); end; print←char("=");@/ print←esc(hash[font←id←base+equiv(n)].rh); {that's |text(font←id←base+equiv(n))|} end @ @<Show the halfword code in |eqtb[n]|@>= if n<math←code←base then begin if n<lc←code←base then begin print←esc("catcode"); print←int(n-cat←code←base); end else if n<uc←code←base then begin print←esc("lccode"); print←int(n-lc←code←base); end else if n<sf←code←base then begin print←esc("uccode"); print←int(n-uc←code←base); end else begin print←esc("sfcode"); print←int(n-sf←code←base); end; print←char("="); print←int(equiv(n)); end else begin print←esc("mathcode"); print←int(n-math←code←base); print←char("="); print←int(ho(equiv(n))); end @ Region 5 of |eqtb| contains the integer parameters and registers defined here, as well as the |del←code| table. The latter table differs from the |cat←code..math←code| tables that precede it, since delimiter codes are fullword integers while the other kinds of codes occupy at most a halfword. This is what makes region~5 different from region~4. We will store the |eq←level| information in an auxiliary array of quarterwords that will be defined later. @d pretolerance←code=0 {badness tolerance before hyphenation} @d tolerance←code=1 {badness tolerance after hyphenation} @d line←penalty←code=2 {added to the badness of every line} @d hyphen←penalty←code=3 {penalty for break after discretionary hyphen} @d ex←hyphen←penalty←code=4 {penalty for break after explicit hyphen} @d club←penalty←code=5 {penalty for creating a club line} @d widow←penalty←code=6 {penalty for creating a widow line} @d display←widow←penalty←code=7 {ditto, just before a display} @d broken←penalty←code=8 {penalty for breaking a page at a broken line} @d bin←op←penalty←code=9 {penalty for breaking after a binary operation} @d rel←penalty←code=10 {penalty for breaking after a relation} @d pre←display←penalty←code=11 {penalty for breaking just before a displayed formula} @d post←display←penalty←code=12 {penalty for breaking just after a displayed formula} @d inter←line←penalty←code=13 {additional penalty between lines} @d double←hyphen←demerits←code=14 {demerits for double hyphen break} @d final←hyphen←demerits←code=15 {demerits for final hyphen break} @d adj←demerits←code=16 {demerits for adjacent incompatible lines} @d mag←code=17 {magnification ratio} @d delimiter←factor←code=18 {ratio for variable-size delimiters} @d looseness←code=19 {change in number of lines for a paragraph} @d time←code=20 {current time of day} @d day←code=21 {current day of the month} @d month←code=22 {current month of the year} @d year←code=23 {current year of our Lord} @d show←box←breadth←code=24 {nodes per level in |show←box|} @d show←box←depth←code=25 {maximum level in |show←box|} @d hbadness←code=26 {hboxes exceeding this badness will be shown by |hpack|} @d vbadness←code=27 {vboxes exceeding this badness will be shown by |vpack|} @d pausing←code=28 {pause after each line is read from a file} @d tracing←online←code=29 {show diagnostic output on terminal} @d tracing←macros←code=30 {show macros as they are being expanded} @d tracing←stats←code=31 {show memory usage if \TeX\ knows it} @d tracing←paragraphs←code=32 {show line-break calculations} @d tracing←pages←code=33 {show page-break calculations} @d tracing←output←code=34 {show boxes when they are shipped out} @d tracing←lost←chars←code=35 {show characters that aren't in the font} @d tracing←commands←code=36 {show command codes at |big←switch|} @d tracing←restores←code=37 {show equivalents when they are restored} @d uc←hyph←code=38 {hyphenate words beginning with a capital letter} @d output←penalty←code=39 {penalty found at current page break} @d max←dead←cycles←code=40 {bound on consecutive dead cycles of output} @d hang←after←code=41 {hanging indentation changes after this many lines} @d floating←penalty←code=42 {penalty for insertions heldover after a split} @d global←defs←code=43 {override \.{\\global} specifications} @d cur←fam←code=44 {current family} @d escape←char←code=45 {escape character for token output} @d default←hyphen←char←code=46 {value of \.{\\hyphenchar} when a font is loaded} @d default←skew←char←code=47 {value of \.{\\skewchar} when a font is loaded} @d end←line←char←code=48 {character placed at the right end of the buffer} @d new←line←char←code=49 {character that prints as |print←ln|} @d int←pars=50 {total number of integer parameters} @d count←base=int←base+int←pars {256 user \.{\\count} registers} @d del←code←base=count←base+256 {128 delimiter code mappings} @d dimen←base=del←code←base+128 {beginning of region 6} @# @d del←code(#)==eqtb[del←code←base+#].int @d count(#)==eqtb[count←base+#].int @d int←par(#)==eqtb[int←base+#].int {an integer parameter} @d pretolerance==int←par(pretolerance←code) @d tolerance==int←par(tolerance←code) @d line←penalty==int←par(line←penalty←code) @d hyphen←penalty==int←par(hyphen←penalty←code) @d ex←hyphen←penalty==int←par(ex←hyphen←penalty←code) @d club←penalty==int←par(club←penalty←code) @d widow←penalty==int←par(widow←penalty←code) @d display←widow←penalty==int←par(display←widow←penalty←code) @d broken←penalty==int←par(broken←penalty←code) @d bin←op←penalty==int←par(bin←op←penalty←code) @d rel←penalty==int←par(rel←penalty←code) @d pre←display←penalty==int←par(pre←display←penalty←code) @d post←display←penalty==int←par(post←display←penalty←code) @d inter←line←penalty==int←par(inter←line←penalty←code) @d double←hyphen←demerits==int←par(double←hyphen←demerits←code) @d final←hyphen←demerits==int←par(final←hyphen←demerits←code) @d adj←demerits==int←par(adj←demerits←code) @d mag==int←par(mag←code) @d delimiter←factor==int←par(delimiter←factor←code) @d looseness==int←par(looseness←code) @d time==int←par(time←code) @d day==int←par(day←code) @d month==int←par(month←code) @d year==int←par(year←code) @d show←box←breadth==int←par(show←box←breadth←code) @d show←box←depth==int←par(show←box←depth←code) @d hbadness==int←par(hbadness←code) @d vbadness==int←par(vbadness←code) @d pausing==int←par(pausing←code) @d tracing←online==int←par(tracing←online←code) @d tracing←macros==int←par(tracing←macros←code) @d tracing←stats==int←par(tracing←stats←code) @d tracing←paragraphs==int←par(tracing←paragraphs←code) @d tracing←pages==int←par(tracing←pages←code) @d tracing←output==int←par(tracing←output←code) @d tracing←lost←chars==int←par(tracing←lost←chars←code) @d tracing←commands==int←par(tracing←commands←code) @d tracing←restores==int←par(tracing←restores←code) @d uc←hyph==int←par(uc←hyph←code) @d output←penalty==int←par(output←penalty←code) @d max←dead←cycles==int←par(max←dead←cycles←code) @d hang←after==int←par(hang←after←code) @d floating←penalty==int←par(floating←penalty←code) @d global←defs==int←par(global←defs←code) @d cur←fam==int←par(cur←fam←code) @d escape←char==int←par(escape←char←code) @d default←hyphen←char==int←par(default←hyphen←char←code) @d default←skew←char==int←par(default←skew←char←code) @d end←line←char==int←par(end←line←char←code) @d new←line←char==int←par(new←line←char←code) @<Assign the values |depth←threshold:=show←box←depth|...@>= depth←threshold:=show←box←depth; breadth←max:=show←box←breadth @ We can print the symbolic name of an integer parameter as follows. @p procedure print←param(@!n:integer); begin case n of pretolerance←code:print←esc("pretolerance"); tolerance←code:print←esc("tolerance"); line←penalty←code:print←esc("linepenalty"); hyphen←penalty←code:print←esc("hyphenpenalty"); ex←hyphen←penalty←code:print←esc("exhyphenpenalty"); club←penalty←code:print←esc("clubpenalty"); widow←penalty←code:print←esc("widowpenalty"); display←widow←penalty←code:print←esc("displaywidowpenalty"); broken←penalty←code:print←esc("brokenpenalty"); bin←op←penalty←code:print←esc("binoppenalty"); rel←penalty←code:print←esc("relpenalty"); pre←display←penalty←code:print←esc("predisplaypenalty"); post←display←penalty←code:print←esc("postdisplaypenalty"); inter←line←penalty←code:print←esc("interlinepenalty"); double←hyphen←demerits←code:print←esc("doublehyphendemerits"); final←hyphen←demerits←code:print←esc("finalhyphendemerits"); adj←demerits←code:print←esc("adjdemerits"); mag←code:print←esc("mag"); delimiter←factor←code:print←esc("delimiterfactor"); looseness←code:print←esc("looseness"); time←code:print←esc("time"); day←code:print←esc("day"); month←code:print←esc("month"); year←code:print←esc("year"); show←box←breadth←code:print←esc("showboxbreadth"); show←box←depth←code:print←esc("showboxdepth"); hbadness←code:print←esc("hbadness"); vbadness←code:print←esc("vbadness"); pausing←code:print←esc("pausing"); tracing←online←code:print←esc("tracingonline"); tracing←macros←code:print←esc("tracingmacros"); tracing←stats←code:print←esc("tracingstats"); tracing←paragraphs←code:print←esc("tracingparagraphs"); tracing←pages←code:print←esc("tracingpages"); tracing←output←code:print←esc("tracingoutput"); tracing←lost←chars←code:print←esc("tracinglostchars"); tracing←commands←code:print←esc("tracingcommands"); tracing←restores←code:print←esc("tracingrestores"); uc←hyph←code:print←esc("uchyph"); output←penalty←code:print←esc("outputpenalty"); max←dead←cycles←code:print←esc("maxdeadcycles"); hang←after←code:print←esc("hangafter"); floating←penalty←code:print←esc("floatingpenalty"); global←defs←code:print←esc("globaldefs"); cur←fam←code:print←esc("fam"); escape←char←code:print←esc("escapechar"); default←hyphen←char←code:print←esc("defaulthyphenchar"); default←skew←char←code:print←esc("defaultskewchar"); end←line←char←code:print←esc("endlinechar"); new←line←char←code:print←esc("newlinechar"); othercases print("[unknown integer parameter!]") endcases; end; @ The integer parameter names must be entered into the hash table. @<Put each...@>= primitive("pretolerance",assign←int,int←base+pretolerance←code);@/ @!@:pretolerance←}{\.{\\pretolerance} primitive@> primitive("tolerance",assign←int,int←base+tolerance←code);@/ @!@:tolerance←}{\.{\\tolerance} primitive@> primitive("linepenalty",assign←int,int←base+line←penalty←code);@/ @!@:line←penalty←}{\.{\\linepenalty} primitive@> primitive("hyphenpenalty",assign←int,int←base+hyphen←penalty←code);@/ @!@:hyphen←penalty←}{\.{\\hyphenpenalty} primitive@> primitive("exhyphenpenalty",assign←int,int←base+ex←hyphen←penalty←code);@/ @!@:ex←hyphen←penalty←}{\.{\\exhyphenpenalty} primitive@> primitive("clubpenalty",assign←int,int←base+club←penalty←code);@/ @!@:club←penalty←}{\.{\\clubpenalty} primitive@> primitive("widowpenalty",assign←int,int←base+widow←penalty←code);@/ @!@:widow←penalty←}{\.{\\widowpenalty} primitive@> primitive("displaywidowpenalty", assign←int,int←base+display←widow←penalty←code);@/ @!@:display←widow←penalty←}{\.{\\displaywidowpenalty} primitive@> primitive("brokenpenalty",assign←int,int←base+broken←penalty←code);@/ @!@:broken←penalty←}{\.{\\brokenpenalty} primitive@> primitive("binoppenalty",assign←int,int←base+bin←op←penalty←code);@/ @!@:bin←op←penalty←}{\.{\\binoppenalty} primitive@> primitive("relpenalty",assign←int,int←base+rel←penalty←code);@/ @!@:rel←penalty←}{\.{\\relpenalty} primitive@> primitive("predisplaypenalty",assign←int,int←base+pre←display←penalty←code);@/ @!@:pre←display←penalty←}{\.{\\predisplaypenalty} primitive@> primitive("postdisplaypenalty",assign←int,int←base+post←display←penalty←code);@/ @!@:post←display←penalty←}{\.{\\postdisplaypenalty} primitive@> primitive("interlinepenalty",assign←int,int←base+inter←line←penalty←code);@/ @!@:inter←line←penalty←}{\.{\\interlinepenalty} primitive@> primitive("doublehyphendemerits", assign←int,int←base+double←hyphen←demerits←code);@/ @!@:double←hyphen←demerits←}{\.{\\doublehyphendemerits} primitive@> primitive("finalhyphendemerits", assign←int,int←base+final←hyphen←demerits←code);@/ @!@:final←hyphen←demerits←}{\.{\\finalhyphendemerits} primitive@> primitive("adjdemerits",assign←int,int←base+adj←demerits←code);@/ @!@:adj←demerits←}{\.{\\adjdemerits} primitive@> primitive("mag",assign←int,int←base+mag←code);@/ @!@:mag←}{\.{\\mag} primitive@> primitive("delimiterfactor",assign←int,int←base+delimiter←factor←code);@/ @!@:delimiter←factor←}{\.{\\delimiterfactor} primitive@> primitive("looseness",assign←int,int←base+looseness←code);@/ @!@:looseness←}{\.{\\looseness} primitive@> primitive("time",assign←int,int←base+time←code);@/ @!@:time←}{\.{\\time} primitive@> primitive("day",assign←int,int←base+day←code);@/ @!@:day←}{\.{\\day} primitive@> primitive("month",assign←int,int←base+month←code);@/ @!@:month←}{\.{\\month} primitive@> primitive("year",assign←int,int←base+year←code);@/ @!@:year←}{\.{\\year} primitive@> primitive("showboxbreadth",assign←int,int←base+show←box←breadth←code);@/ @!@:show←box←breadth←}{\.{\\showboxbreadth} primitive@> primitive("showboxdepth",assign←int,int←base+show←box←depth←code);@/ @!@:show←box←depth←}{\.{\\showboxdepth} primitive@> primitive("hbadness",assign←int,int←base+hbadness←code);@/ @!@:hbadness←}{\.{\\hbadness} primitive@> primitive("vbadness",assign←int,int←base+vbadness←code);@/ @!@:vbadness←}{\.{\\vbadness} primitive@> primitive("pausing",assign←int,int←base+pausing←code);@/ @!@:pausing←}{\.{\\pausing} primitive@> primitive("tracingonline",assign←int,int←base+tracing←online←code);@/ @!@:tracing←online←}{\.{\\tracingonline} primitive@> primitive("tracingmacros",assign←int,int←base+tracing←macros←code);@/ @!@:tracing←macros←}{\.{\\tracingmacros} primitive@> primitive("tracingstats",assign←int,int←base+tracing←stats←code);@/ @!@:tracing←stats←}{\.{\\tracingstats} primitive@> primitive("tracingparagraphs",assign←int,int←base+tracing←paragraphs←code);@/ @!@:tracing←paragraphs←}{\.{\\tracingparagraphs} primitive@> primitive("tracingpages",assign←int,int←base+tracing←pages←code);@/ @!@:tracing←pages←}{\.{\\tracingpages} primitive@> primitive("tracingoutput",assign←int,int←base+tracing←output←code);@/ @!@:tracing←output←}{\.{\\tracingoutput} primitive@> primitive("tracinglostchars",assign←int,int←base+tracing←lost←chars←code);@/ @!@:tracing←lost←chars←}{\.{\\tracinglostchars} primitive@> primitive("tracingcommands",assign←int,int←base+tracing←commands←code);@/ @!@:tracing←commands←}{\.{\\tracingcommands} primitive@> primitive("tracingrestores",assign←int,int←base+tracing←restores←code);@/ @!@:tracing←restores←}{\.{\\tracingrestores} primitive@> primitive("uchyph",assign←int,int←base+uc←hyph←code);@/ @!@:uc←hyph←}{\.{\\uchyph} primitive@> primitive("outputpenalty",assign←int,int←base+output←penalty←code);@/ @!@:output←penalty←}{\.{\\outputpenalty} primitive@> primitive("maxdeadcycles",assign←int,int←base+max←dead←cycles←code);@/ @!@:max←dead←cycles←}{\.{\\maxdeadcycles} primitive@> primitive("hangafter",assign←int,int←base+hang←after←code);@/ @!@:hang←after←}{\.{\\hangafter} primitive@> primitive("floatingpenalty",assign←int,int←base+floating←penalty←code);@/ @!@:floating←penalty←}{\.{\\floatingpenalty} primitive@> primitive("globaldefs",assign←int,int←base+global←defs←code);@/ @!@:global←defs←}{\.{\\globaldefs} primitive@> primitive("fam",assign←int,int←base+cur←fam←code);@/ @!@:fam←}{\.{\\fam} primitive@> primitive("escapechar",assign←int,int←base+escape←char←code);@/ @!@:escape←char←}{\.{\\escapechar} primitive@> primitive("defaulthyphenchar",assign←int,int←base+default←hyphen←char←code);@/ @!@:default←hyphen←char←}{\.{\\defaulthyphenchar} primitive@> primitive("defaultskewchar",assign←int,int←base+default←skew←char←code);@/ @!@:default←skew←char←}{\.{\\defaultskewchar} primitive@> primitive("endlinechar",assign←int,int←base+end←line←char←code);@/ @!@:end←line←char←}{\.{\\endlinechar} primitive@> primitive("newlinechar",assign←int,int←base+new←line←char←code);@/ @!@:new←line←char←}{\.{\\newlinechar} primitive@> @ @<Cases of |print←cmd←chr|...@>= assign←int: if chr←code<count←base then print←param(chr←code-int←base) else begin print←esc("count"); print←int(chr←code-count←base); end; @ The integer parameters should really be initialized by a macro package; the following initialization does the minimum to keep \TeX\ from complete failure. @↑null delimiter@> @<Initialize table entries...@>= for k:=int←base to del←code←base-1 do eqtb[k].int:=0; mag:=1000; tolerance:=10000; hang←after:=1; max←dead←cycles:=25; escape←char:="\"; end←line←char:=carriage←return; for k:=0 to 127 do del←code(k):=-1; del←code("."):=0; {this null delimiter is used in error recovery} @ The following procedure, which is called just before \TeX\ initializes its input and output, establishes the initial values of the date and time. @↑system dependencies@> Since standard \PASCAL\ cannot provide such information, something special is needed. The program here simply specifies July 4, 1776, at noon; but users probably want a better approximation to the truth. @p procedure fix←date←and←time; begin time:=12*60; {minutes since midnight} day:=4; {fourth day of the month} month:=7; {seventh month of the year} year:=1776; {Anno Domini} end; @ @<Show equivalent |n|, in region 5@>= begin if n<count←base then print←param(n-int←base) else if n<del←code←base then begin print←esc("count"); print←int(n-count←base); end else begin print←esc("delcode"); print←int(n-del←code←base); end; print←char("="); print←int(eqtb[n].int); end @ @<Set variable |c| to the current escape character@>=c:=escape←char @ @<Character |s| is the current new-line character@>=s=new←line←char @ \TeX\ is occasionally supposed to print diagnostic information that goes only into the transcript file, unless |tracing←online| is positive. Here are two routines that adjust the destination of print commands: @p procedure begin←diagnostic; {prepare to do some tracing} begin old←setting:=selector; if history=spotless then history:=warning←issued; if (tracing←online<=0)and(selector=term←and←log) then decr(selector); end; @# procedure end←diagnostic(@!blank←line:boolean); {restore proper conditions after tracing} begin print←nl(""); if blank←line then print←ln; selector:=old←setting; end; @ Of course we had better declare another global variable, if the previous routines are going to work. @<Glob...@>= @!old←setting:0..max←selector; @ The final region of |eqtb| contains the dimension parameters defined here, and the 256 \.{\\dimen} registers. @d par←indent←code=0 {indentation of paragraphs} @d math←surround←code=1 {space around math in text} @d line←skip←limit←code=2 {threshold for |line←skip| instead of |baseline←skip|} @d hsize←code=3 {line width in horizontal mode} @d vsize←code=4 {page height in vertical mode} @d max←depth←code=5 {maximum depth of boxes on main pages} @d split←max←depth←code=6 {maximum depth of boxes on split pages} @d box←max←depth←code=7 {maximum depth of explicit vboxes} @d hfuzz←code=8 {tolerance for overfull hbox messages} @d vfuzz←code=9 {tolerance for overfull vbox messages} @d delimiter←shortfall←code=10 {maximum amount uncovered by variable delimiters} @d null←delimiter←space←code=11 {blank space in null delimiters} @d script←space←code=12 {extra space after subscript or superscript} @d pre←display←size←code=13 {length of text preceding a display} @d display←width←code=14 {length of line for displayed equation} @d display←indent←code=15 {indentation of line for displayed equation} @d overfull←rule←code=16 {width of rule that identifies overfull hboxes} @d hang←indent←code=17 {amount of hanging indentation} @d h←offset←code=18 {amount of horizontal offset when shipping pages out} @d v←offset←code=19 {amount of vertical offset when shipping pages out} @d dimen←pars=20 {total number of dimension parameters} @d scaled←base=dimen←base+dimen←pars {table of 256 user-defined \.{\\dimen} registers} @d eqtb←size=scaled←base+255 {largest subscript of |eqtb|} @# @d dimen(#)==eqtb[scaled←base+#].sc @d dimen←par(#)==eqtb[dimen←base+#].sc {a scaled quantity} @d par←indent==dimen←par(par←indent←code) @d math←surround==dimen←par(math←surround←code) @d line←skip←limit==dimen←par(line←skip←limit←code) @d hsize==dimen←par(hsize←code) @d vsize==dimen←par(vsize←code) @d max←depth==dimen←par(max←depth←code) @d split←max←depth==dimen←par(split←max←depth←code) @d box←max←depth==dimen←par(box←max←depth←code) @d hfuzz==dimen←par(hfuzz←code) @d vfuzz==dimen←par(vfuzz←code) @d delimiter←shortfall==dimen←par(delimiter←shortfall←code) @d null←delimiter←space==dimen←par(null←delimiter←space←code) @d script←space==dimen←par(script←space←code) @d pre←display←size==dimen←par(pre←display←size←code) @d display←width==dimen←par(display←width←code) @d display←indent==dimen←par(display←indent←code) @d overfull←rule==dimen←par(overfull←rule←code) @d hang←indent==dimen←par(hang←indent←code) @d h←offset==dimen←par(h←offset←code) @d v←offset==dimen←par(v←offset←code) @p procedure print←length←param(@!n:integer); begin case n of par←indent←code:print←esc("parindent"); math←surround←code:print←esc("mathsurround"); line←skip←limit←code:print←esc("lineskiplimit"); hsize←code:print←esc("hsize"); vsize←code:print←esc("vsize"); max←depth←code:print←esc("maxdepth"); split←max←depth←code:print←esc("splitmaxdepth"); box←max←depth←code:print←esc("boxmaxdepth"); hfuzz←code:print←esc("hfuzz"); vfuzz←code:print←esc("vfuzz"); delimiter←shortfall←code:print←esc("delimitershortfall"); null←delimiter←space←code:print←esc("nulldelimiterspace"); script←space←code:print←esc("scriptspace"); pre←display←size←code:print←esc("predisplaysize"); display←width←code:print←esc("displaywidth"); display←indent←code:print←esc("displayindent"); overfull←rule←code:print←esc("overfullrule"); hang←indent←code:print←esc("hangindent"); h←offset←code:print←esc("hoffset"); v←offset←code:print←esc("voffset"); othercases print("[unknown dimen parameter!]") endcases; end; @ @<Put each...@>= primitive("parindent",assign←dimen,dimen←base+par←indent←code);@/ @!@:par←indent←}{\.{\\parindent} primitive@> primitive("mathsurround",assign←dimen,dimen←base+math←surround←code);@/ @!@:math←surround←}{\.{\\mathsurround} primitive@> primitive("lineskiplimit",assign←dimen,dimen←base+line←skip←limit←code);@/ @!@:line←skip←limit←}{\.{\\lineskiplimit} primitive@> primitive("hsize",assign←dimen,dimen←base+hsize←code);@/ @!@:hsize←}{\.{\\hsize} primitive@> primitive("vsize",assign←dimen,dimen←base+vsize←code);@/ @!@:vsize←}{\.{\\vsize} primitive@> primitive("maxdepth",assign←dimen,dimen←base+max←depth←code);@/ @!@:max←depth←}{\.{\\maxdepth} primitive@> primitive("splitmaxdepth",assign←dimen,dimen←base+split←max←depth←code);@/ @!@:split←max←depth←}{\.{\\splitmaxdepth} primitive@> primitive("boxmaxdepth",assign←dimen,dimen←base+box←max←depth←code);@/ @!@:box←max←depth←}{\.{\\boxmaxdepth} primitive@> primitive("hfuzz",assign←dimen,dimen←base+hfuzz←code);@/ @!@:hfuzz←}{\.{\\hfuzz} primitive@> primitive("vfuzz",assign←dimen,dimen←base+vfuzz←code);@/ @!@:vfuzz←}{\.{\\vfuzz} primitive@> primitive("delimitershortfall", assign←dimen,dimen←base+delimiter←shortfall←code);@/ @!@:delimiter←shortfall←}{\.{\\delimitershortfall} primitive@> primitive("nulldelimiterspace", assign←dimen,dimen←base+null←delimiter←space←code);@/ @!@:null←delimiter←space←}{\.{\\nulldelimiterspace} primitive@> primitive("scriptspace",assign←dimen,dimen←base+script←space←code);@/ @!@:script←space←}{\.{\\scriptspace} primitive@> primitive("predisplaysize",assign←dimen,dimen←base+pre←display←size←code);@/ @!@:pre←display←size←}{\.{\\predisplaysize} primitive@> primitive("displaywidth",assign←dimen,dimen←base+display←width←code);@/ @!@:display←width←}{\.{\\displaywidth} primitive@> primitive("displayindent",assign←dimen,dimen←base+display←indent←code);@/ @!@:display←indent←}{\.{\\displayindent} primitive@> primitive("overfullrule",assign←dimen,dimen←base+overfull←rule←code);@/ @!@:overfull←rule←}{\.{\\overfullrule} primitive@> primitive("hangindent",assign←dimen,dimen←base+hang←indent←code);@/ @!@:hang←indent←}{\.{\\hangindent} primitive@> primitive("hoffset",assign←dimen,dimen←base+h←offset←code);@/ @!@:h←offset←}{\.{\\hoffset} primitive@> primitive("voffset",assign←dimen,dimen←base+v←offset←code);@/ @!@:v←offset←}{\.{\\voffset} primitive@> @ @<Cases of |print←cmd←chr|...@>= assign←dimen: if chr←code<scaled←base then print←length←param(chr←code-dimen←base) else begin print←esc("dimen"); print←int(chr←code-scaled←base); end; @ @<Initialize table entries...@>= for k:=dimen←base to eqtb←size do eqtb[k].sc:=0; @ @<Show equivalent |n|, in region 6@>= begin if n<scaled←base then print←length←param(n-dimen←base) else begin print←esc("dimen"); print←int(n-scaled←base); end; print←char("="); print←scaled(eqtb[n].sc); print("pt"); end @ Here is a procedure that displays the contents of |eqtb[n]| symbolically. @p@t\4@>@<Declare the procedure called |print←cmd←chr|@>@;@/ @!stat procedure show←eqtb(@!n:pointer); begin if n<active←base then print←char("?") {this can't happen} else if n<glue←base then @<Show equivalent |n|, in region 1 or 2@> else if n<local←base then @<Show equivalent |n|, in region 3@> else if n<int←base then @<Show equivalent |n|, in region 4@> else if n<dimen←base then @<Show equivalent |n|, in region 5@> else if n<=eqtb←size then @<Show equivalent |n|, in region 6@> else print←char("?"); {this can't happen either} end; tats @ The last two regions of |eqtb| have fullword values instead of the three fields |eq←level|, |eq←type|, and |equiv|. An |eq←type| is unnecessary, but \TeX\ needs to store the |eq←level| information in another array called |xeq←level|. @<Glob...@>= @!eqtb:array[active←base..eqtb←size] of memory←word; @!xeq←level:array[int←base..eqtb←size] of quarterword; @ @<Set init...@>= for k:=int←base to eqtb←size do xeq←level[k]:=level←one; @ When the debugging routine |search←mem| is looking for pointers having a given value, it is interested only in regions 1 to~3 of~|eqtb|, and in the first part of region~4. @<Search |eqtb| for equivalents equal to |p|@>= for q:=active←base to box←base+255 do begin if equiv(q)=p then begin print←nl("EQUIV("); print←int(q); print←char(")"); end; end @* \[18] The hash table. Control sequences are stored and retrieved by means of a fairly standard hash table algorithm called the method of ``coalescing lists'' (cf.\ Algorithm 6.4C in {\sl The Art of Computer Programming\/}). Once a control sequence enters the table, it is never removed, because there are complicated situations involving \.{\\gdef} where the removal of a control sequence at the end of a group would be a mistake preventable only by the introduction of a complicated reference-count mechanism. The actual sequence of letters forming a control sequence identifier is stored in the |str←pool| array together with all the other strings. An auxiliary array |hash| consists of items with two halfword fields per word. The first of these, called |next(p)|, points to the next identifier belonging to the same coalesced list as the identifier corresponding to~|p|; and the other, called |text(p)|, points to the |str←start| entry for |p|'s identifier. If position~|p| of the hash table is empty, we have |text(p)=0|; if position |p| is either empty or the end of a coalesced hash list, we have |next(p)=0|. An auxiliary pointer variable called |hash←used| is maintained in such a way that all locations |p>=hash←used| are nonempty. The global variable |cs←count| tells how many multiletter control sequences have been defined, if statistics are being kept. A global boolean variable called |no←new←control←sequence| is set to |true| during the time that new hash table entries are forbidden. @d next(#) == hash[#].lh {link for coalesced lists} @d text(#) == hash[#].rh {string number for control sequence name} @d hash←is←full == (hash←used=hash←base) {test if all positions are occupied} @d font←id←text(#) == text(font←id←base+#) {a frozen font identifier's name} @<Glob...@>= @!hash: array[hash←base..undefined←control←sequence-1] of two←halves; {the hash table} @!hash←used:pointer; {allocation pointer for |hash|} @!no←new←control←sequence:boolean; {are new identifiers legal?} @!cs←count:integer; {total number of known identifiers} @ @<Set init...@>= no←new←control←sequence:=true; {new identifiers are usually forbidden} next(hash←base):=0; text(hash←base):=0; for k:=hash←base+1 to undefined←control←sequence-1 do hash[k]:=hash[hash←base]; @ @<Initialize table entries...@>= hash←used:=frozen←control←sequence; {nothing is used} cs←count:=0; eq←type(frozen←dont←expand):=dont←expand; text(frozen←dont←expand):="notexpanded:"; @.notexpanded:@> @ Here is the subroutine that searches the hash table for an identifier that matches a given string of length |l>1| appearing in |buffer[j.. (j+l-1)]|. If the identifier is found, the corresponding hash table address is returned. Otherwise, if the global variable |no←new←control←sequence| is |true|, the dummy address |undefined←control←sequence| is returned. Otherwise the identifier is inserted into the hash table and its location is returned. @p function id←lookup(@!j,@!l:integer):pointer; {search the hash table} label found; {go here if you found it} var h:integer; {hash code} @!p:pointer; {index in |hash| array} @!k:pointer; {index in |buffer| array} begin @<Compute the hash code |h|@>; p:=h+hash←base; {we start searching here; note that |0<=h<hash←prime|} loop@+begin if text(p)>0 then if length(text(p))=l then if str←eq←buf(text(p),j) then goto found; if next(p)=0 then begin if no←new←control←sequence then p:=undefined←control←sequence else @<Insert a new control sequence after |p|, then make |p| point to it@>; goto found; end; p:=next(p); end; found: id←lookup:=p; end; @ @<Insert a new control...@>= begin if text(p)>0 then begin repeat if hash←is←full then overflow("hash size",hash←size); @:TeX capacity exceeded hash size}{\quad hash size@> decr(hash←used); until text(hash←used)=0; {search for an empty location in |hash|} next(p):=hash←used; p:=hash←used; end; str←room(l); for k:=j to j+l-1 do append←char(buffer[k]); text(p):=make←string; @!stat incr(cs←count);@+tats@;@/ end @ The value of |hash←prime| should be roughly 85\% of |hash←size|, and it should be a prime number. The theory of hashing tells us to expect fewer than two table probes, on the average, when the search is successful. @<Compute the hash code |h|@>= h:=buffer[j]; for k:=j+1 to j+l-1 do begin h:=h+h+buffer[k]; while h>=hash←prime do h:=h-hash←prime; end @ Single-character control sequences do not need to be looked up in a hash table, since we can use the character code itself as a direct address. The procedure |print←cs| prints the name of a control sequence, given a pointer to its address in |eqtb|. A space is printed after the name unless it is a single nonletter or an active character. This procedure might be invoked with invalid data, so it is ``extra robust.'' The individual characters must be printed one at a time using |print|, since they may be unprintable. @<Basic printing...@>= procedure print←cs(@!p:integer); {prints a purported control sequence} begin if p<hash←base then {single character} if p>=single←base then if p=null←cs then begin print←esc("csname"); print←esc("endcsname"); end else begin print←esc(p-single←base); if cat←code(p-single←base)=letter then print←char(" "); end else if p<active←base then print←esc("IMPOSSIBLE.") @.IMPOSSIBLE@> else print(p-active←base) else if p>=undefined←control←sequence then print←esc("IMPOSSIBLE.") else if (text(p)<0)or(text(p)>=str←ptr) then print←esc("NONEXISTENT.") @.NONEXISTENT@> else begin print←esc(""); slow←print(text(p)); print←char(" "); end; end; @ Here is a similar procedure; it avoids the error checks, and it never prints a space after the control sequence. @<Basic printing procedures@>= procedure sprint←cs(@!p:pointer); {prints a control sequence} begin if p<hash←base then if p<single←base then print(p-active←base) else if p<null←cs then print←esc(p-single←base) else begin print←esc("csname"); print←esc("endcsname"); end else begin print←esc(""); slow←print(text(p)); end; end; @ We need to put \TeX's ``primitive'' control sequences into the hash table, together with their command code (which will be the |eq←type|) and an operand (which will be the |equiv|). The |primitive| procedure does this, in a way that no \TeX\ user can. The global value |cur←val| contains the new |eqtb| pointer after |primitive| has acted. @p @!init procedure primitive(@!s:str←number;@!c:quarterword;@!o:halfword); var k:pool←pointer; {index into |str←pool|} @!j:small←number; {index into |buffer|} @!l:small←number; {length of the string} begin if s<128 then cur←val:=s+single←base else begin k:=str←start[s]; l:=str←start[s+1]-k; {we will move |s| into the (empty) |buffer|} for j:=0 to l-1 do buffer[j]:=str←pool[k+j]; cur←val:=id←lookup(0,l); {|no←new←control←sequence| is |false|} flush←string; text(cur←val):=s; {we don't want to have the string twice} end; eq←level(cur←val):=level←one; eq←type(cur←val):=c; equiv(cur←val):=o; end; tini @ Many of \TeX's primitives need no |equiv|, since they are identifiable by their |eq←type| alone. These primitives are loaded into the hash table as follows: @<Put each of \TeX's primitives into the hash table@>= primitive(" ",ex←space,0);@/ @!@:Single-character primitives /}{\quad\.{\\\ }@> primitive("/",ital←corr,0);@/ @!@:Single-character primitives /}{\quad\.{\\/}@> primitive("accent",accent,0);@/ @!@:accent←}{\.{\\accent} primitive@> primitive("advance",advance,0);@/ @!@:advance←}{\.{\\advance} primitive@> primitive("afterassignment",after←assignment,0);@/ @!@:after←assignment←}{\.{\\afterassignment} primitive@> primitive("aftergroup",after←group,0);@/ @!@:after←group←}{\.{\\aftergroup} primitive@> primitive("begingroup",begin←group,0);@/ @!@:begin←group←}{\.{\\begingroup} primitive@> primitive("char",char←num,0);@/ @!@:char←}{\.{\\char} primitive@> primitive("csname",cs←name,0);@/ @!@:cs←name←}{\.{\\csname} primitive@> primitive("delimiter",delim←num,0);@/ @!@:delimiter←}{\.{\\delimiter} primitive@> primitive("divide",divide,0);@/ @!@:divide←}{\.{\\divide} primitive@> primitive("endcsname",end←cs←name,0);@/ @!@:end←cs←name←}{\.{\\endcsname} primitive@> primitive("endgroup",end←group,0); @!@:end←group←}{\.{\\endgroup} primitive@> text(frozen←end←group):="endgroup"; eqtb[frozen←end←group]:=eqtb[cur←val];@/ primitive("expandafter",expand←after,0);@/ @!@:expand←after←}{\.{\\expandafter} primitive@> primitive("font",def←font,0);@/ @!@:font←}{\.{\\font} primitive@> primitive("fontdimen",assign←font←dimen,0);@/ @!@:font←dimen←}{\.{\\fontdimen} primitive@> primitive("halign",halign,0);@/ @!@:halign←}{\.{\\halign} primitive@> primitive("hrule",hrule,0);@/ @!@:hrule←}{\.{\\hrule} primitive@> primitive("ignorespaces",ignore←spaces,0);@/ @!@:ignore←spaces←}{\.{\\ignorespaces} primitive@> primitive("insert",insert,0);@/ @!@:insert←}{\.{\\insert} primitive@> primitive("mark",mark,0);@/ @!@:mark←}{\.{\\mark} primitive@> primitive("mathaccent",math←accent,0);@/ @!@:math←accent←}{\.{\\mathaccent} primitive@> primitive("mathchar",math←char←num,0);@/ @!@:math←char←}{\.{\\mathchar} primitive@> primitive("mathchoice",math←choice,0);@/ @!@:math←choice←}{\.{\\mathchoice} primitive@> primitive("multiply",multiply,0);@/ @!@:multiply←}{\.{\\multiply} primitive@> primitive("noalign",no←align,0);@/ @!@:no←align←}{\.{\\noalign} primitive@> primitive("noexpand",no←expand,0);@/ @!@:no←expand←}{\.{\\noexpand} primitive@> primitive("nonscript",non←script,0);@/ @!@:non←script←}{\.{\\nonscript} primitive@> primitive("omit",omit,0);@/ @!@:omit←}{\.{\\omit} primitive@> primitive("parshape",set←shape,0);@/ @!@:par←shape←}{\.{\\parshape} primitive@> primitive("penalty",break←penalty,0);@/ @!@:penalty←}{\.{\\penalty} primitive@> primitive("prevgraf",set←prev←graf,0);@/ @!@:prev←graf←}{\.{\\prevgraf} primitive@> primitive("radical",radical,0);@/ @!@:radical←}{\.{\\radical} primitive@> primitive("read",read←to←cs,0);@/ @!@:read←}{\.{\\read} primitive@> primitive("relax",relax,256); {cf.\ |scan←file←name|} @!@:relax←}{\.{\\relax} primitive@> text(frozen←relax):="relax"; eqtb[frozen←relax]:=eqtb[cur←val];@/ primitive("setbox",set←box,0);@/ @!@:set←box←}{\.{\\setbox} primitive@> primitive("the",the,0);@/ @!@:the←}{\.{\\the} primitive@> primitive("toks",toks←register,0);@/ @!@:toks←}{\.{\\toks} primitive@> primitive("vadjust",vadjust,0);@/ @!@:vadjust←}{\.{\\vadjust} primitive@> primitive("valign",valign,0);@/ @!@:valign←}{\.{\\valign} primitive@> primitive("vcenter",vcenter,0);@/ @!@:vcenter←}{\.{\\vcenter} primitive@> primitive("vrule",vrule,0);@/ @!@:vrule←}{\.{\\vrule} primitive@> @ Each primitive has a corresponding inverse, so that it is possible to display the cryptic numeric contents of |eqtb| in symbolic form. Every call of |primitive| in this program is therefore accompanied by some straightforward code that forms part of the |print←cmd←chr| routine explained below. @<Cases of |print←cmd←chr|...@>= accent: print←esc("accent"); advance: print←esc("advance"); after←assignment: print←esc("afterassignment"); after←group: print←esc("aftergroup"); assign←font←dimen: print←esc("fontdimen"); begin←group: print←esc("begingroup"); break←penalty: print←esc("penalty"); char←num: print←esc("char"); cs←name: print←esc("csname"); def←font: print←esc("font"); delim←num: print←esc("delimiter"); divide: print←esc("divide"); end←cs←name: print←esc("endcsname"); end←group: print←esc("endgroup"); ex←space: print←esc(" "); expand←after: print←esc("expandafter"); halign: print←esc("halign"); hrule: print←esc("hrule"); ignore←spaces: print←esc("ignorespaces"); insert: print←esc("insert"); ital←corr: print←esc("/"); mark: print←esc("mark"); math←accent: print←esc("mathaccent"); math←char←num: print←esc("mathchar"); math←choice: print←esc("mathchoice"); multiply: print←esc("multiply"); no←align: print←esc("noalign"); no←expand: print←esc("noexpand"); non←script: print←esc("nonscript"); omit: print←esc("omit"); radical: print←esc("radical"); read←to←cs: print←esc("read"); relax: print←esc("relax"); set←box: print←esc("setbox"); set←prev←graf: print←esc("prevgraf"); set←shape: print←esc("parshape"); the: print←esc("the"); toks←register: print←esc("toks"); vadjust: print←esc("vadjust"); valign: print←esc("valign"); vcenter: print←esc("vcenter"); vrule: print←esc("vrule"); @ We will deal with the other primitives later, at some point in the program where their |eq←type| and |equiv| values are more meaningful. For example, the primitives for math mode will be loaded when we consider the routines that deal with formulas. It is easy to find where each particular primitive was treated by looking in the index at the end; for example, the section where |"radical"| entered |eqtb| is listed under `\.{\\radical} primitive'. (Primitives consisting of a single nonalphabetic character, @!like `\.{\\/}', are listed under `Single-character primitives'.) @!@↑Single-character primitives@> Meanwhile, this is a convenient place to catch up on something we were unable to do before the hash table was defined: @<Print the font identifier for |font(p)|@>= print←esc(font←id←text(font(p))) @* \[19] Saving and restoring equivalents. The nested structure provided by `$\.{\char'173}\ldots\.{\char'175}$' groups in \TeX\ means that |eqtb| entries valid in outer groups should be saved and restored later if they are overridden inside the braces. When a new |eqtb| value is being assigned, the program therefore checks to see if the previous entry belongs to an outer level. In such a case, the old value is placed on the |save←stack| just before the new value enters |eqtb|. At the end of a grouping level, i.e., when the right brace is sensed, the |save←stack| is used to restore the outer values, and the inner ones are destroyed. Entries on the |save←stack| are of type |memory←word|. The top item on this stack is |save←stack[p]|, where |p=save←ptr-1|; it contains three fields called |save←type|, |save←level|, and |save←index|, and it is interpreted in one of four ways: \yskip\hang 1) If |save←type(p)=restore←old←value|, then |save←index(p)| is a location in |eqtb| whose current value should be destroyed at the end of the current group and replaced by |save←stack[p-1]|. Furthermore if |save←index(p)>=int←base|, then |save←level(p)| should replace the corresponding entry in |xeq←level|. \yskip\hang 2) If |save←type(p)=restore←zero|, then |save←index(p)| is a location in |eqtb| whose current value should be destroyed at the end of the current group and replaced by |eqtb[undefined←control←sequence]|. \yskip\hang 3) If |save←type(p)=insert←token|, then |save←index(p)| is a token that should be inserted into \TeX's input when the current group ends. \yskip\hang 4) If |save←type(p)=level←boundary|, then |save←level(p)| is a code explaining what kind of group we were previously in, and |save←index(p)| points to the level boundary word at the bottom of the entries for that group. @d save←type(#)==save←stack[#].hh.b0 {classifies a |save←stack| entry} @d save←level(#)==save←stack[#].hh.b1 {saved level for regions 5 and 6, or group code} @d save←index(#)==save←stack[#].hh.rh {|eqtb| location or |save←stack| location} @d restore←old←value=0 {|save←type| when a value should be restored later} @d restore←zero=1 {|save←type| when an undefined entry should be restored} @d insert←token=2 {|save←type| when a token is being saved for later use} @d level←boundary=3 {|save←type| corresponding to beginning of group} @ Here are the group codes that are used to discriminate between different kinds of groups. They allow \TeX\ to decide what special actions, if any, should be performed when a group ends. \def\grp{\.{\char'173...\char'175}} Some groups are not supposed to be ended by right braces. For example, the `\.\$' that begins a math formula causes a |math←shift←group| to be started, and this should be terminated by a matching `\.\$'. Similarly, a group that starts with \.{\\left} should end with \.{\\right}, and one that starts with \.{\\begingroup} should end with \.{\\endgroup}. @d bottom←level=0 {group code for the outside world} @d simple←group=1 {group code for local structure only} @d hbox←group=2 {code for `\.{\\hbox}\grp'} @d adjusted←hbox←group=3 {code for `\.{\\hbox}\grp' in vertical mode} @d vbox←group=4 {code for `\.{\\vbox}\grp'} @d vtop←group=5 {code for `\.{\\vtop}\grp'} @d align←group=6 {code for `\.{\\halign}\grp', `\.{\\valign}\grp'} @d no←align←group=7 {code for `\.{\\noalign}\grp'} @d output←group=8 {code for output routine} @d math←group=9 {code for, e.g, `\.{\char'136}\grp'} @d disc←group=10 {code for `\.{\\discretionary}\grp\grp\grp'} @d insert←group=11 {code for `\.{\\insert}\grp', `\.{\\vadjust}\grp'} @d vcenter←group=12 {code for `\.{\\vcenter}\grp'} @d math←choice←group=13 {code for `\.{\\mathchoice}\grp\grp\grp\grp'} @d semi←simple←group=14 {code for `\.{\\begingroup...\\endgroup}'} @d math←shift←group=15 {code for `\.{\$...\$}'} @d math←left←group=16 {code for `\.{\\left...\\right}'} @d max←group←code=16 @<Types...@>= @!group←code=0..max←group←code; {|save←index| for a level boundary} @ The global variable |cur←group| keeps track of what sort of group we are currently in. Another global variable, |cur←boundary|, points to the topmost |level←boundary| word. And |cur←level| is the current depth of nesting. The routines are designed to preserve the condition that no entry in the |save←stack| or in |eqtb| ever has a level greater than |cur←level|. @ @<Glob...@>= @!save←stack : array[0..save←size] of memory←word; @!save←ptr : 0..save←size; {first unused entry on |save←stack|} @!max←save←stack:0..save←size; {maximum usage of save stack} @!cur←level: quarterword; {current nesting level for groups} @!cur←group: group←code; {current group type} @!cur←boundary: 0..save←size; {where the current level begins} @ At this time it might be a good idea for the reader to review the introduction to |eqtb| that was given above just before the long lists of parameter names. Recall that the ``outer level'' of the program is |level←one|, since undefined control sequences are assumed to be ``defined'' at |level←zero|. @<Set init...@>= save←ptr:=0; cur←level:=level←one; cur←group:=bottom←level; cur←boundary:=0; max←save←stack:=0; @ The following macro is used to test if there is room for up to six more entries on |save←stack|. By making a conservative test like this, we can get by with testing for overflow in only a few places. @d check←full←save←stack==if save←ptr>max←save←stack then begin max←save←stack:=save←ptr; if max←save←stack>save←size-6 then overflow("save size",save←size); @:TeX capacity exceeded save size}{\quad save size@> end @ Procedure |new←save←level| is called when a group begins. The argument is a group identification code like `|hbox←group|'. After calling this routine, it is safe to put five more entries on |save←stack|. In some cases a few integer-valued items are placed onto the |save←stack| just below a |level←boundary| word, because this is a convenient place to keep information that is supposed to ``pop up'' just when the group has finished. For example, when `\.{\\hbox to 100pt}\grp' is being treated, the 100pt dimension is stored on |save←stack| just before |new←save←level| is called. We use the notation |saved(k)| to stand for an integer item that appears in location |save←ptr+k| of the save stack. @d saved(#)==save←stack[save←ptr+#].int @p procedure new←save←level(@!c:group←code); {begin a new level of grouping} begin check←full←save←stack; save←type(save←ptr):=level←boundary; save←level(save←ptr):=cur←group; save←index(save←ptr):=cur←boundary; if cur←level=max←quarterword then overflow("grouping levels", @:TeX capacity exceeded grouping levels}{\quad grouping levels@> max←quarterword-min←quarterword); {quit if |(cur←level+1)| is too big to be stored in |eqtb|} cur←boundary:=save←ptr; incr(cur←level); incr(save←ptr); cur←group:=c; end; @ Just before an entry of |eqtb| is changed, the following procedure should be called to update the other data structures properly. It is important to keep in mind that reference counts in |mem| include references from within |save←stack|, so these counts must be handled carefully. @p procedure eq←destroy(@!w:memory←word); {gets ready to forget |w|} var q:pointer; {|equiv| field of |w|} begin case eq←type←field(w) of call,long←call,outer←call,long←outer←call: delete←token←ref(equiv←field(w)); glue←ref: delete←glue←ref(equiv←field(w)); shape←ref: begin q:=equiv←field(w); {we need to free a \.{\\parshape} block} if q<>null then free←node(q,info(q)+info(q)+1); end; {such a block is |2n+1| words long, where |n=info(q)|} box←ref: flush←node←list(equiv←field(w)); othercases do←nothing endcases; end; @ To save a value of |eqtb[p]| that was established at level |l|, we can use the following subroutine. @p procedure eq←save(@!p:pointer;@!l:quarterword); {saves |eqtb[p]|} begin check←full←save←stack; if l=level←zero then save←type(save←ptr):=restore←zero else begin save←stack[save←ptr]:=eqtb[p]; incr(save←ptr); save←type(save←ptr):=restore←old←value; end; save←level(save←ptr):=l; save←index(save←ptr):=p; incr(save←ptr); end; @ The procedure |eq←define| defines an |eqtb| entry having specified |eq←type| and |equiv| fields, and saves the former value if appropriate. This procedure is used only for entries in the first four regions of |eqtb|, i.e., only for entries that have |eq←type| and |equiv| fields. After calling this routine, it is safe to put four more entries on |save←stack|, provided that there was room for four more entries before the call, since |eq←save| makes the necessary test. @p procedure eq←define(@!p:pointer;@!t:quarterword;@!e:halfword); {new data for |eqtb|} begin if eq←level(p)=cur←level then eq←destroy(eqtb[p]) else if cur←level>level←one then eq←save(p,eq←level(p)); eq←level(p):=cur←level; eq←type(p):=t; equiv(p):=e; end; @ The counterpart of |eq←define| for the remaining (fullword) positions in |eqtb| is called |eq←word←define|. Since |xeq←level[p]>=level←one| for all |p|, a `|restore←zero|' will never be used in this case. @p procedure eq←word←define(@!p:pointer;@!w:integer); begin if xeq←level[p]<>cur←level then begin eq←save(p,xeq←level[p]); xeq←level[p]:=cur←level; end; eqtb[p].int:=w; end; @ The |eq←define| and |eq←word←define| routines take care of local definitions. @↑global definitions@> Global definitions are done in almost the same way, but there is no need to save old values, and the new value is associated with |level←one|. @p procedure geq←define(@!p:pointer;@!t:quarterword;@!e:halfword); {global |eq←define|} begin eq←destroy(eqtb[p]); eq←level(p):=level←one; eq←type(p):=t; equiv(p):=e; end; @# procedure geq←word←define(@!p:pointer;@!w:integer); {global |eq←word←define|} begin eqtb[p].int:=w; xeq←level[p]:=level←one; end; @ Subroutine |save←for←after| puts a token on the stack for save-keeping. @p procedure save←for←after(@!t:halfword); begin check←full←save←stack; save←type(save←ptr):=insert←token; save←level(save←ptr):=level←zero; save←index(save←ptr):=t; incr(save←ptr); end; @ The |unsave| routine goes the other way, taking items off of |save←stack|. This routine takes care of restoration when a level ends; everything belonging to the topmost group is cleared off of the save stack. @p@t\4@>@<Declare the procedure called |restore←trace|@>@;@/ procedure@?back←input; forward; @t\2@> procedure unsave; {pops the top level off the save stack} label done; var p:pointer; {position to be restored} @!l:quarterword; {saved level, if in fullword regions of |eqtb|} @!t:halfword; {saved value of |cur←tok|} begin if cur←level>level←one then begin decr(cur←level); @<Clear off top level from |save←stack|@>; end else confusion("curlevel"); {|unsave| is not used when |cur←group=bottom←level|} @:this can't happen curlevel}{\quad curlevel@> end; @ @<Clear off...@>= loop@+begin decr(save←ptr); if save←type(save←ptr)=level←boundary then goto done; p:=save←index(save←ptr); if save←type(save←ptr)=insert←token then @<Insert token |p| into \TeX's input@> else begin if save←type(save←ptr)=restore←old←value then begin l:=save←level(save←ptr); decr(save←ptr); end else save←stack[save←ptr]:=eqtb[undefined←control←sequence]; @<Store \(s)|save←stack[save←ptr]| in |eqtb[p]|, unless |eqtb[p]| holds a global value@>; end; end; done: cur←group:=save←level(save←ptr); cur←boundary:=save←index(save←ptr) @ A global definition, which sets the level to |level←one|, @↑global definitions@> will not be undone by |unsave|. If at least one global definition of |eqtb[p]| has been carried out within the group that just ended, the last such definition will therefore survive. @<Store \(s)|save...@>= if p<int←base then if eq←level(p)=level←one then begin eq←destroy(save←stack[save←ptr]); {destroy the saved value} @!stat if tracing←restores>0 then restore←trace(p,"retaining");@+tats@;@/ end else begin eq←destroy(eqtb[p]); {destroy the current value} eqtb[p]:=save←stack[save←ptr]; {restore the saved value} @!stat if tracing←restores>0 then restore←trace(p,"restoring");@+tats@;@/ end else if xeq←level[p]<>level←one then begin eqtb[p]:=save←stack[save←ptr]; xeq←level[p]:=l; @!stat if tracing←restores>0 then restore←trace(p,"restoring");@+tats@;@/ end else begin @!stat if tracing←restores>0 then restore←trace(p,"retaining");@+tats@;@/ end @ @<Declare the procedure called |restore←trace|@>= @!stat procedure restore←trace(@!p:pointer;@!s:str←number); {|eqtb[p]| has just been restored or retained} begin begin←diagnostic; print←char("{"); print(s); print←char(" "); show←eqtb(p); print←char("}"); end←diagnostic(false); end; tats @ When looking for possible pointers to a memory location, it is helpful to look for references from |eqtb| that might be waiting on the save stack. Of course, we might find spurious pointers too; but this routine is merely an aid when debugging, and at such times we are grateful for any scraps of information, even if they prove to be irrelevant. @↑dirty \PASCAL@> @<Search |save←stack| for equivalents that point to |p|@>= if save←ptr>0 then for q:=0 to save←ptr-1 do begin if equiv←field(save←stack[q])=p then begin print←nl("SAVE("); print←int(q); print←char(")"); end; end @ Most of the parameters kept in |eqtb| can be changed freely, but there's an exception: The magnification should not be used with two different values during any \TeX\ job, since a single magnification is applied to an entire run. The global variable |mag←set| is set to the current magnification whenever it becomes necessary to ``freeze'' it at a particular value. @<Glob...@>= @!mag←set:integer; {if nonzero, this magnification should be used henceforth} @ @<Set init...@>= mag←set:=0; @ The |prepare←mag| subroutine is called whenever \TeX\ wants to use |mag| for magnification. @p procedure prepare←mag; begin if (mag←set>0)and(mag<>mag←set) then begin print←err("Incompatible magnification ("); print←int(mag); @.Incompatible magnification@> print(");"); print←nl(" the previous value will be retained"); help2("I can handle only one magnification ratio per job. So I've")@/ ("reverted to the magnification you used earlier on this run.");@/ int←error(mag←set); geq←word←define(int←base+mag←code,mag←set); {|mag:=mag←set|} end; if (mag<=0)or(mag>32768) then begin print←err("Illegal magnification has been changed to 1000");@/ @.Illegal magnification...@> help1("The magnification ratio must be between 1 and 32768."); int←error(mag); geq←word←define(int←base+mag←code,1000); end; mag←set:=mag; end; @* \[20] Token lists. A \TeX\ token is either a character or a control sequence, and it is @↑token@> represented internally in one of two ways: (1)~A character whose ASCII code number is |c| and whose command code is |m| is represented as the number $2↑8m+c$; the command code is in the range |1<=m<=14|. (2)~A control sequence whose |eqtb| address is |p| is represented as the number |cs←token←flag+p|. Here |cs←token←flag=@t$2↑{12}$@>| is larger than $2↑8m+c$, yet it is small enough that |cs←token←flag+p< max←halfword|; thus, a token fits comfortably in a halfword. A token |t| represents a |left←brace| command if and only if |t<left←brace←limit|; it represents a |right←brace| command if and only if we have |left←brace←limit<=t<right←brace←limit|; and it represents a |match| or |end←match| command if and only if |match←token<=t<=end←match←token|. The following definitions take care of these token-oriented constants and a few others. @d cs←token←flag==@'10000 {amount added to the |eqtb| location in a token that stands for a control sequence; is a multiple of~256} @d left←brace←token=@'0400 {$2↑8\cdot|left←brace|$} @d left←brace←limit=@'1000 {$2↑8\cdot(|left←brace|+1)$} @d right←brace←token=@'1000 {$2↑8\cdot|right←brace|$} @d right←brace←limit=@'1400 {$2↑8\cdot(|right←brace|+1)$} @d math←shift←token=@'1400 {$2↑8\cdot|math←shift|$} @d tab←token=@'2000 {$2↑8\cdot|tab←mark|$} @d out←param←token=@'2400 {$2↑8\cdot|out←param|$} @d space←token=@'5040 {$2↑8\cdot|spacer|+|" "|$} @d letter←token=@'5400 {$2↑8\cdot|letter|$} @d other←token=@'6000 {$2↑8\cdot|other←char|$} @d match←token=@'6400 {$2↑8\cdot|match|$} @d end←match←token=@'7000 {$2↑8\cdot|end←match|$} @ @<Check the ``constant''...@>= if cs←token←flag+undefined←control←sequence>max←halfword then bad:=21; @ A token list is a singly linked list of one-word nodes in |mem|, where each word contains a token and a link. Macro definitions, output-routine definitions, marks, and \.{\\write} texts are kept in \TeX's memory in the form of token lists, preceded by a node that has a reference count in its |token←ref←count| field. The token stored in location |p| is called |info(p)|. Three special commands appear in the token lists of macro definitions. When |m=match|, it means that \TeX\ should scan a parameter for the current macro; when |m=end←match|, it means that parameter matching should end and \TeX\ should start reading the macro text; and when |m=out←param|, it means that \TeX\ should insert parameter number |c| into the text at this point. The enclosing \.{\char'173} and \.{\char'175} characters of a macro definition are omitted, but the final right brace of an output routine is included at the end of its token list. Here is an example macro definition that illustrates these conventions. After \TeX\ processes the text $$\.{\\def\\mac a\#1\#2 \\b \{\#1\\-a \#\#1\#2 \#2\}}$$ the definition of \.{\\mac} is represented as a token list containing $$\def\,{\hskip2pt} \vbox{\halign{\hfil#\hfil\cr (reference count), |letter|\,\.a, |match|\,\#, |match|\,\#, |spacer|\,\.\ , \.{\\b}, |end←match|,\cr |out←param|\,1, \.{\\-}, |letter|\,\.a, |spacer|\,\.\ , |mac←param|\,\#, |other←char|\,\.1,\cr |out←param|\,2, |spacer|\,\.\ , |out←param|\,2.\cr}}$$ The procedure |scan←toks| builds such token lists, and |macro←call| does the parameter matching. @↑reference counts@> Examples such as $$\.{\\def\\m\{\\def\\m\{a\}\ b\}}$$ explain why reference counts would be needed even if \TeX\ had no \.{\\let} operation: When the token list for \.{\\m} is being read, the redefinition of \.{\\m} changes the |eqtb| entry before the token list has been fully consumed, so we dare not simply destroy a token list when its control sequence is being redefined. If the parameter-matching part of a definition ends with `\.{\#\{}', the corresponding token list will have `\.\{' just before the `|end←match|' and also at the very end. The first `\.\{' is used to delimit the parameter; the second one keeps the first from disappearing. @ The procedure |show←token←list|, which prints a symbolic form of the token list that starts at a given node |p|, illustrates these conventions. The token list being displayed should not begin with a reference count. However, the procedure is intended to be robust, so that if the memory links are awry or if |p| is not really a pointer to a token list, nothing catastrophic will happen. An additional parameter |q| is also given; this parameter is either null or it points to a node in the token list where a certain magic computation takes place that will be explained later. (Basically, |q| is non-null when we are printing the two-line context information at the time of an error message; |q| marks the place corresponding to where the second line should begin.) For example, if |p| points to the node containing the first \.a in the token list above, then |show←token←list| will print the string $$\hbox{`\.{a\#1\#2\ \\b\ ->\#1\\-a\ \#\#1\#2\ \#2}';}$$ and if |q| points to the node containing the second \.a, the magic computation will be performed just before the second \.a is printed. The generation will stop, and `\.{\\ETC.}' will be printed, if the length of printing exceeds a given limit~|l|. Anomalous entries are printed in the form of control sequences that are not followed by a blank space, e.g., `\.{\\BAD.}'; this cannot be confused with actual control sequences because a real control sequence named \.{BAD} would come out `\.{\\BAD\ }'. @<Declare the procedure called |show←token←list|@>= procedure show←token←list(@!p,@!q:integer;@!l:integer); label exit; var m,@!c:integer; {pieces of a token} @!match←chr:ASCII←code; {character used in a `|match|'} @!n:ASCII←code; {the highest parameter number, as an ASCII digit} begin match←chr:="#"; n:="0"; tally:=0; while (p<>null) and (tally<l) do begin if p=q then @<Do magic computation@>; @<Display token |p|, and |return| if there are problems@>; p:=link(p); end; if p<>null then print←esc("ETC."); @.ETC@> exit: end; @ @<Display token |p|...@>= if (p<hi←mem←min) or (p>mem←end) then begin print←esc("CLOBBERED."); return; @.CLOBBERED@> end; if info(p)>=cs←token←flag then print←cs(info(p)-cs←token←flag) else begin m:=info(p) div @'400; c:=info(p) mod @'400; if (info(p)<0)or(c>127) then print←esc("BAD.") @.BAD@> else @<Display the token $(|m|,|c|)$@>; end @ The procedure usually ``learns'' the character code used for macro parameters by seeing one in a |match| command before it runs into any |out←param| commands. @<Display the token ...@>= case m of left←brace,right←brace,math←shift,tab←mark,sup←mark,sub←mark,spacer, letter,other←char: print(c); mac←param: begin print(c); print(c); end; out←param: begin print(match←chr); if c<=9 then print←char(c+"0") else begin print←char("!"); return; end; end; match: begin match←chr:=c; print(c); incr(n); print←char(n); if n>"9" then return; end; end←match: print("->"); @.->@> othercases print←esc("BAD.") @.BAD@> endcases @ Here's the way we sometimes want to display a token list, given a pointer to its reference count; the pointer may be null. @p procedure token←show(@!p:pointer); begin if p<>null then show←token←list(link(p),null,1000); end; @ The |print←meaning| subroutine displays |cur←cmd| and |cur←chr| in symbolic form, including the expansion of a macro or mark. @p procedure print←meaning; begin print←cmd←chr(cur←cmd,cur←chr); if cur←cmd>=call then begin print←char(":"); print←ln; token←show(cur←chr); end else if cur←cmd=top←bot←mark then begin print←char(":"); print←ln; token←show(cur←mark[cur←chr]); end; end; @* \[21] Introduction to the syntactic routines. Let's pause a moment now and try to look at the Big Picture. The \TeX\ program consists of three main parts: syntactic routines, semantic routines, and output routines. The chief purpose of the syntactic routines is to deliver the user's input to the semantic routines, one token at a time. The semantic routines act as an interpreter responding to these tokens, which may be regarded as commands. And the output routines are periodically called on to convert box-and-glue lists into a compact set of instructions that will be sent to a typesetter. We have discussed the basic data structures and utility routines of \TeX, so we are good and ready to plunge into the real activity by considering the syntactic routines. Our current goal is to come to grips with the |get←next| procedure, which is the keystone of \TeX's input mechanism. Each call of |get←next| sets the value of three variables |cur←cmd|, |cur←chr|, and |cur←cs|, representing the next input token. $$\vbox{\halign{#\hfil\cr \hbox{|cur←cmd| denotes a command code from the long list of codes given above;}\cr \hbox{|cur←chr| denotes a character code or other modifier of the command code;}\cr \hbox{|cur←cs| is the |eqtb| location of the current control sequence,}\cr \hbox{\qquad if the current token was a control sequence otherwise it's zero.}\cr}}$$ Underlying this external behavior of |get←next| is all the machinery necessary to convert from character files to tokens. At a given time we may be only partially finished with the reading of several files (for which \.{\\input} was specified), and partially finished with the expansion of some user-defined macros and/or some macro parameters, and partially finished with the generation of some text in a template for \.{\\halign}, and so on. When reading a character file, special characters must be classified as math delimiters, etc.; comments and extra blank spaces must be removed, paragraphs must be recognized, and control sequences must be found in the hash table. Furthermore there are occasions in which the scanning routines have looked ahead for a word like `\.{plus}' but only part of that word was found, hence a few characters must be put back into the input and scanned again. To handle these situations, which might all be present simultaneously, \TeX\ uses various stacks that hold information about the incomplete activities, and there is a finite state control for each level of the input mechanism. These stacks record the current state of an implicitly recursive process, but the |get←next| procedure is not recursive. Therefore it will not be difficult to translate these algorithms into low-level languages that do not support recursion. @<Glob...@>= @!cur←cmd: eight←bits; {current command set by |get←next|} @!cur←chr: halfword; {operand of current command} @!cur←cs: pointer; {control sequence found here, zero if none found} @!cur←tok: halfword; {packed representative of |cur←cmd| and |cur←chr|} @ The |print←cmd←chr| routine prints a symbolic interpretation of a command code and its modifier. This is used in certain `\.{You can\'t}' error messages, and in the implementation of diagnostic routines like \.{\\show}. The body of |print←cmd←chr| is a rather tedious listing of print commands, and most of it is essentially an inverse to the |primitive| routine that enters a \TeX\ primitive into |eqtb|. Therefore much of this procedure appears elsewhere in the program, together with the corresponding |primitive| calls. @d chr←cmd(#)==begin print(#); print←ASCII(chr←code); end @<Declare the procedure called |print←cmd←chr|@>= procedure print←cmd←chr(@!cmd:quarterword;@!chr←code:halfword); begin case cmd of left←brace: chr←cmd("begin-group character "); right←brace: chr←cmd("end-group character "); math←shift: chr←cmd("math shift character "); mac←param: chr←cmd("macro parameter character "); sup←mark: chr←cmd("superscript character "); sub←mark: chr←cmd("subscript character "); endv: print("end of alignment template"); spacer: chr←cmd("blank space "); letter: chr←cmd("the letter "); other←char: chr←cmd("the character "); @t\4@>@<Cases of |print←cmd←chr| for symbolic printing of primitives@>@/ othercases print("[unknown command code!]") endcases; end; @ Here is a procedure that displays the current command. @p procedure show←cur←cmd←chr; begin begin←diagnostic; print←nl("{"); if mode<>shown←mode then begin print←mode(mode); print(": "); shown←mode:=mode; end; print←cmd←chr(cur←cmd,cur←chr); print←char("}"); end←diagnostic(false); end; @* \[22] Input stacks and states. \TeX\ uses two different conventions for representing sequential stacks. @↑stack conventions@>@↑conventions for representing stacks@> \yskip\hang 1) If there is frequent access to the top entry, and if the stack is essentially never empty, then the top entry is kept in a global variable (even better would be a machine register), and the other entries appear in the array $\\{stack}[0\to(\\{ptr}-1)]$. For example, the semantic stack described above is handled this way, and so is the input stack that we are about to study. \yskip\hang 2) If there is infrequent top access, the entire stack contents are in the array $\\{stack}[0\to(\\{ptr}-1)]$. For example, the |save←stack| is treated this way, as we have seen. \yskip\noindent The state of \TeX's input mechanism appears in the input stack, whose entries are records with six fields, called |state|, |index|, |start|, |loc|, |limit|, and |name|. This stack is maintained with convention~(1), so it is declared in the following way: @<Types...@>= @!in←state←record = record @!state←field, @!index←field: quarterword; @!start←field,@!loc←field, @!limit←field, @!name←field: halfword; end; @ @<Glob...@>= @!input←stack : array[0..stack←size] of in←state←record; @!input←ptr : 0..stack←size; {first unused location of |input←stack|} @!max←in←stack: 0..stack←size; {largest value of |input←ptr| when pushing} @!cur←input : in←state←record; {the ``top'' input state, according to convention (1)} @ We've already defined the special variable |@!loc==cur←input.loc←field| in our discussion of basic input-output routines. The other components of |cur←input| are defined in the same way: @d state==cur←input.state←field {current scanner state} @d index==cur←input.index←field {reference for buffer information} @d start==cur←input.start←field {starting position in |buffer|} @d limit==cur←input.limit←field {end of current line in |buffer|} @d name==cur←input.name←field {name of the current file} @ Let's look more closely now at the six control variables (|state|,~|index|,~|start|,~|loc|,~|limit|,~|name|), assuming that \TeX\ is reading a line of characters that have been input from some file or from the user's terminal. There is an array called |buffer| that acts as a stack of all lines of characters that are currently being read from files, including all lines on subsidiary levels of the input stack that are not yet completed. We will return to the other lines when we are finished with the present input file. (Incidentally, on a machine with byte-oriented addressing, it would be appropriate to combine |buffer| with the |str←pool| array, letting the buffer entries grow downward from the top of the string pool and checking that these two tables don't bump into each other.) The line we are currently working on begins in position |start| of the buffer; the next character we are about to read is |buffer[loc]|; and |limit| is the location of the last character present. If |loc>limit|, the line has been completely read. Usually |buffer[limit]| is the |end←line←char|, denoting the end of a line, but this is not true if the current line is an insertion that was entered on the user's terminal in response to an error message. The |name| variable is a string number that designates the name of the current file, if we are reading a text file. It is zero if we are reading from the terminal; it is |n+1| if we are reading from input stream |n|, where |0<=n<=16|. (Input stream 16 stands for an invalid stream number; in such cases the input is actually from the terminal, under control of the procedure |read←toks|.) The |state| variable has one of three values, when we are scanning such files: $$\baselineskip 15pt\vbox{\halign{#\hfil\cr 1) |state=mid←line| is the normal state.\cr 2) |state=skip←blanks| is like |mid←line|, but blanks are ignored.\cr 3) |state=new←line| is the state at the beginning of a line.\cr}}$$ These state values are assigned numeric codes so that if we add the state code to the next character's command code, we get distinct values. For example, `|mid←line+spacer|' stands for the case that a blank space character occurs in the middle of a line when it is not being ignored; after this case is processed, the next value of |state| will be |skip←blanks|. @d mid←line=1 {|state| code when scanning a line of characters} @d skip←blanks=2+max←char←code {|state| code when ignoring blanks} @d new←line=3+max←char←code+max←char←code {|state| code at start of line} @ Additional information about the current line is available via the |index| variable, which counts how many lines of characters are present in the buffer below the current level. We have |index=0| when reading from the terminal and prompting the user for each line; then if the user types, e.g., `\.{\\input paper}', we will have |index=1| while reading the file \.{paper.tex}. However, it does not follow that |index| is the same as the input stack pointer, since many of the levels on the input stack may come from token lists. For example, the instruction `\.{\\input paper}' might occur in a token list. The global variable |in←open| is equal to the |index| value of the highest non-token-list level. Thus, the number of partially read lines in the buffer is |in←open+1|, and we have |in←open=index| when we are not reading a token list. If we are not currently reading from the terminal, or from an input stream, we are reading from the file variable |input←file[index]|. We use the notation |terminal←input| as a convenient abbreviation for |name=0|, and |cur←file| as an abbreviation for |input←file[index]|. The global variable |line| contains the line number in the topmost open file, for use in error messages. If we are not reading from the terminal, |line←stack[index]| holds the line number for the enclosing level, so that |line| can be restored when the current file has been read. Line numbers should never be negative, since the negative of the current line number is used to identify the user's output routine in the |mode←line| field of the semantic nest entries. If more information about the input state is needed, it can be included in small arrays like those shown here. For example, the current page or segment number in the input file might be put into a variable |@!page|, maintained for enclosing levels in `\ignorespaces|@!page←stack:array[1..max←in←open] of integer|\unskip' by analogy with |line←stack|. @↑system dependencies@> @d terminal←input==(name=0) {are we reading from the terminal?} @d cur←file==input←file[index] {the current |alpha←file| variable} @<Glob...@>= @!in←open : 0..max←in←open; {the number of lines in the buffer, less one} @!input←file : array[1..max←in←open] of alpha←file; @!line : integer; {current line number in the current source file} @!line←stack : array[1..max←in←open] of integer; @ Users of \TeX\ sometimes forget to balance left and right braces properly, and one of the ways \TeX\ tries to spot such errors is by considering an input file as broken into subfiles by control sequences that are declared to be \.{\\outer}. A variable called |scanner←status| tells \TeX\ whether or not to complain when a subfile ends. This variable has five possible values: \yskip\hang|normal|, means that a subfile can safely end here without incident. \yskip\hang|skipping|, means that a subfile can safely end here, but not a file, because we're reading past some conditional text that was not selected. \yskip\hang|defining|, means that a subfile shouldn't end now because a macro is being defined. \yskip\hang|matching|, means that a subfile shouldn't end now because a macro is being used and we are searching for the end of its arguments. \yskip\hang|aligning|, means that a subfile shouldn't end now because we are not finished with the preamble of an \.{\\halign} or \.{\\valign}. \yskip\hang|absorbing|, means that a subfile shouldn't end now because we are reading a balanced token list for \.{\\message}, \.{\\write}, etc. \yskip\noindent If the |scanner←status| is not |normal|, the variable |warning←index| points to the |eqtb| location for the relevant control sequence name to print in an error message. @d skipping=1 {|scanner←status| when passing conditional text} @d defining=2 {|scanner←status| when reading a macro definition} @d matching=3 {|scanner←status| when reading macro arguments} @d aligning=4 {|scanner←status| when reading an alignment preamble} @d absorbing=5 {|scanner←status| when reading a balanced text} @<Glob...@>= @!scanner←status : normal..absorbing; {can a subfile end now?} @!warning←index : pointer; {identifier relevant to non-|normal| scanner status} @!def←ref : pointer; {reference count of token list being defined} @ Here is a procedure that uses |scanner←status| to print a warning message when a subfile has ended, and at certain other crucial times: @<Declare the procedure called |runaway|@>= procedure runaway; var p:pointer; {head of runaway list} begin if scanner←status>skipping then begin print←nl("Runaway "); @.Runaway...@> case scanner←status of defining: begin print("definition"); p:=def←ref; end; matching: begin print("argument"); p:=temp←head; end; aligning: begin print("preamble"); p:=hold←head; end; absorbing: begin print("text"); p:=def←ref; end; end; {there are no other cases} print←char("?");print←ln; show←token←list(link(p),null,error←line-10); end; end; @ However, all this discussion about input state really applies only to the case that we are inputting from a file. There is another important case, namely when we are currently getting input from a token list. In this case |state=token←list|, and the conventions about the other state variables are different: \yskip\hang|loc| is a pointer to the current node in the token list, i.e., the node that will be read next. If |loc=null|, the token list has been fully read. \yskip\hang|start| points to the first node of the token list; this node may or may not contain a reference count, depending on the type of token list involved. \yskip\hang|token←type|, which takes the place of |index| in the discussion above, is a code number that explains what kind of token list is being scanned. \yskip\hang|name| points to the |eqtb| address of the control sequence being expanded, if the current token list is a macro. \yskip\hang|param←start|, which takes the place of |limit|, tells where the parameters of the current macro begin in the |param←stack|, if the current token list is a macro. \yskip\noindent The |token←type| can take several values, depending on where the current token list came from: \yskip\hang|parameter|, if a parameter is being scanned; \hang|u←template|, if the \<u←j> part of an alignment template is being scanned; \hang|v←template|, if the \<v←j> part of an alignment template is being scanned; \hang|backed←up|, if the token list being scanned has been inserted as `to be read again'. \hang|inserted|, if the token list being scanned has been inserted as the text expansion of a \.{\\count} or similar variable; \hang|macro|, if a user-defined control sequence is being scanned; \hang|output←text|, if an \.{\\output} routine is being scanned; \hang|every←par←text|, if the text of \.{\\everypar} is being scanned; \hang|every←math←text|, if the text of \.{\\everymath} is being scanned; \hang|every←display←text|, if the text of \.{\\everydisplay} is being scanned; \hang|every←hbox←text|, if the text of \.{\\everyhbox} is being scanned; \hang|every←vbox←text|, if the text of \.{\\everyvbox} is being scanned; \hang|every←job←text|, if the text of \.{\\everyjob} is being scanned; \hang|every←cr←text|, if the text of \.{\\everycr} is being scanned; \hang|mark←text|, if the text of a \.{\\mark} is being scanned; \hang|write←text|, if the text of a \.{\\write} is being scanned. \yskip\noindent The codes for |output←text|, |every←par←text|, etc., are equal to a constant plus the corresponding codes for token list parameters |output←routine←loc|, |every←par←loc|, etc. The token list begins with a reference count if and only if |token←type>=macro|. @↑reference counts@> @d token←list=0 {|state| code when scanning a token list} @d token←type==index {type of current token list} @d param←start==limit {base of macro parameters in |param←stack|} @d parameter=0 {|token←type| code for parameter} @d u←template=1 {|token←type| code for \<u←j> template} @d v←template=2 {|token←type| code for \<v←j> template} @d backed←up=3 {|token←type| code for text to be reread} @d inserted=4 {|token←type| code for inserted texts} @d macro=5 {|token←type| code for defined control sequences} @d output←text=6 {|token←type| code for output routines} @d every←par←text=7 {|token←type| code for \.{\\everypar}} @d every←math←text=8 {|token←type| code for \.{\\everymath}} @d every←display←text=9 {|token←type| code for \.{\\everydisplay}} @d every←hbox←text=10 {|token←type| code for \.{\\everyhbox}} @d every←vbox←text=11 {|token←type| code for \.{\\everyvbox}} @d every←job←text=12 {|token←type| code for \.{\\everyjob}} @d every←cr←text=13 {|token←type| code for \.{\\everycr}} @d mark←text=14 {|token←type| code for \.{\\topmark}, etc.} @d write←text=15 {|token←type| code for \.{\\write}} @ The |param←stack| is an auxiliary array used to hold pointers to the token lists for parameters at the current level and subsidiary levels of input. This stack is maintained with convention (2), and it grows at a different rate from the others. @<Glob...@>= @!param←stack:array [0..param←size] of pointer; {token list pointers for parameters} @!param←ptr:0..param←size; {first unused entry in |param←stack|} @!max←param←stack:integer; {largest value of |param←ptr|, will be |<=param←size+9|} @ The input routines must also interact with the processing of \.{\\halign} and \.{\\valign}, since the appearance of tab marks and \.{\\cr} in certain places is supposed to trigger the beginning of special \<v←j> template text in the scanner. This magic is accomplished by an |align←state| variable that is increased by~1 when a `\.{\char'173}' is scanned and decreased by~1 when a `\.{\char'175}' is scanned. The |align←state| is nonzero during the \<u←j> template, after which it is set to zero; the \<v←j> template begins when a tab mark or \.{\\cr} occurs at a time that |align←state=0|. @<Glob...@>= @!align←state:integer; {group level with respect to current alignment} @ Thus, the ``current input state'' can be very complicated indeed; there can be many levels and each level can arise in a variety of ways. The |show←context| procedure, which is used by \TeX's error-reporting routine to print out the current input state on all levels down to the most recent line of characters from an input file, illustrates most of these conventions. The global variable |base←ptr| contains the lowest level that was displayed by this procedure. @<Glob...@>= @!base←ptr:0..stack←size; {shallowest level shown by |show←context|} @ The status at each level is indicated by printing two lines, where the first line indicates what was read so far and the second line shows what remains to be read. The context is cropped, if necessary, so that the first line contains at most |half←error←line| characters, and the second contains at most |error←line|. Non-current input levels whose |token←type| is `|backed←up|' are shown only if they have not been fully read. @p procedure show←context; {prints where the scanner is} label done; var old←setting:0..max←selector; {saved |selector| setting} @<Local variables for formatting calculations@>@/ begin base←ptr:=input←ptr; input←stack[base←ptr]:=cur←input; {store current state} loop@+begin cur←input:=input←stack[base←ptr]; {enter into the context} @<Display the current context@>; if (state<>token←list) then if (name>17) or (base←ptr=0) then goto done; decr(base←ptr); end; done: cur←input:=input←stack[input←ptr]; {restore original state} end; @ @<Display the current context@>= if (base←ptr=input←ptr) or (state<>token←list) or (token←type<>backed←up) or (loc<>null) then {we omit backed-up token lists that have already been read} begin tally:=0; {get ready to count characters} old←setting:=selector; if state<>token←list then begin @<Print location of current line@>; @<Pseudoprint the line@>; end else begin @<Print type of token list@>; @<Pseudoprint the token list@>; end; selector:=old←setting; {stop pseudoprinting} @<Print two lines using the tricky pseudoprinted information@>; end @ This routine should be changed, if necessary, to give the best possible indication of where the current line resides in the input file. For example, on some systems it is best to print both a page and line number. @↑system dependencies@> @<Print location of current line@>= if name<=17 then if terminal←input then if base←ptr=0 then print←nl("<*>") else print←nl("<insert> ") else begin print←nl("<read "); if name=17 then print←char("*")@+else print←int(name-1); print←char(">"); end else begin print←nl("l."); print←int(line); end; print←char(" ") @ @<Print type of token list@>= case token←type of parameter: print←nl("<argument> "); u←template,v←template: print←nl("<template> "); backed←up: if loc=null then print←nl("<recently read> ") else print←nl("<to be read again> "); inserted: print←nl("<inserted text> "); macro: begin print←ln; print←cs(name); end; output←text: print←nl("<output> "); every←par←text: print←nl("<everypar> "); every←math←text: print←nl("<everymath> "); every←display←text: print←nl("<everydisplay> "); every←hbox←text: print←nl("<everyhbox> "); every←vbox←text: print←nl("<everyvbox> "); every←job←text: print←nl("<everyjob> "); every←cr←text: print←nl("<everycr> "); mark←text: print←nl("<mark> "); write←text: print←nl("<write> "); othercases print←nl("?") {this should never happen} endcases @ Here it is necessary to explain a little trick. We don't want to store a long string that corresponds to a token list, because that string might take up lots of memory; and we are printing during a time when an error message is being given, so we dare not do anything that might overflow one of \TeX's tables. So `pseudoprinting' is the answer: We enter a mode of printing that stores characters into a buffer of length |error←line|, where character $k+1$ is placed into \hbox{|trick←buf[k mod error←line]|} if |k<trick←count|, otherwise character |k| is dropped. Initially we set |tally:=0| and |trick←count:=1000000|; then when we reach the point where transition from line 1 to line 2 should occur, we set |first←count:=tally| and |trick←count:=@tmax@>(error←line, tally+1+error←line-half←error←line)|. At the end of the pseudoprinting, the values of |first←count|, |tally|, and |trick←count| give us all the information we need to print the two lines, and all of the necessary text is in |trick←buf|. Namely, let |l| be the length of the descriptive information that appears on the first line. The length of the context information gathered for that line is |k=first←count|, and the length of the context information gathered for line~2 is $m=\min(|tally|, |trick←count|)-k$. If |l+k<=h|, where |h=half←error←line|, we print |trick←buf[0..k-1]| after the descriptive information on line~1, and set |n:=l+k|; here |n| is the length of line~1. If $l+k>h$, some cropping is necessary, so we set |n:=h| and print `\.{...}' followed by $$\hbox{|trick←buf[(l+k-h+3)..k-1]|,}$$ where subscripts of |trick←buf| are circular modulo |error←line|. The second line consists of |n|~spaces followed by |trick←buf[k..(k+m-1)]|, unless |n+m>error←line|; in the latter case, further cropping is done. This is easier to program than to explain. @<Local variables for formatting...@>= @!i:0..buf←size; {index into |buffer|} @!j:0..buf←size; {end of current line in |buffer|} @!l:0..half←error←line; {length of descriptive information on line 1} @!m:integer; {context information gathered for line 2} @!n:0..error←line; {length of line 1} @!p: integer; {starting or ending place in |trick←buf|} @!q: integer; {temporary index} @ The following code sets up the print routines so that they will gather the desired information. @d begin←pseudoprint== begin l:=tally; tally:=0; selector:=pseudo; trick←count:=1000000; end @d set←trick←count== begin first←count:=tally; trick←count:=tally+1+error←line-half←error←line; if trick←count<error←line then trick←count:=error←line; end @ And the following code uses the information after it has been gathered. @<Print two lines using the tricky pseudoprinted information@>= if trick←count=1000000 then set←trick←count; {|set←trick←count| must be performed} if tally<trick←count then m:=tally-first←count else m:=trick←count-first←count; {context on line 2} if l+first←count<=half←error←line then begin p:=0; n:=l+first←count; end else begin print("..."); p:=l+first←count-half←error←line+3; n:=half←error←line; end; for q:=p to first←count-1 do print←char(trick←buf[q mod error←line]); print←ln; for q:=1 to n do print←char(" "); {print |n| spaces to begin line~2} if m+n<=error←line then p:=first←count+m else p:=first←count+(error←line-n-3); for q:=first←count to p-1 do print←char(trick←buf[q mod error←line]); if m+n>error←line then print("...") @ But the trick is distracting us from our current goal, which is to understand the input state. So let's concentrate on the data structures that are being pseudoprinted as we finish up the |show←context| procedure. @<Pseudoprint the line@>= begin←pseudoprint; if buffer[limit]=end←line←char then j:=limit else j:=limit+1; {determine the effective end of the line} if j>0 then for i:=start to j-1 do begin if i=loc then set←trick←count; print(buffer[i]); end @ @<Pseudoprint the token list@>= begin←pseudoprint; if token←type<macro then show←token←list(start,loc,100000) else show←token←list(link(start),loc,100000) {avoid reference count} @ Here is the missing piece of |show←token←list| that is activated when the token beginning line~2 is about to be shown: @<Do magic computation@>=set←trick←count @* \[23] Maintaining the input stacks. The following subroutines change the input status in commonly needed ways. First comes |push←input|, which stores the current state and creates a new level (having, initially, the same properties as the old). @d push←input==@t@> {enter a new input level, save the old} begin if input←ptr>max←in←stack then begin max←in←stack:=input←ptr; if input←ptr=stack←size then overflow("input stack size",stack←size); @:TeX capacity exceeded input stack size}{\quad input stack size@> end; input←stack[input←ptr]:=cur←input; {stack the record} incr(input←ptr); end @ And of course what goes up must come down. @d pop←input==@t@> {leave an input level, re-enter the old} begin decr(input←ptr); cur←input:=input←stack[input←ptr]; end @ Here is a procedure that starts a new level of token-list input, given a token list |p| and its type |t|. If |t=macro|, the calling routine should set |name| and |loc|. @d back←list(#)==begin←token←list(#,backed←up) {backs up a simple token list} @d ins←list(#)==begin←token←list(#,inserted) {inserts a simple token list} @p procedure begin←token←list(@!p:pointer;@!t:quarterword); begin push←input; state:=token←list; start:=p; token←type:=t; if t>=macro then {the token list starts with a reference count} begin add←token←ref(p); if t=macro then param←start:=param←ptr else begin loc:=link(p); if tracing←macros>1 then begin begin←diagnostic; print←nl(""); case t of mark←text:print←esc("mark"); write←text:print←esc("write"); othercases print←cmd←chr(assign←toks,t-output←text+output←routine←loc) endcases;@/ print("->"); token←show(p); end←diagnostic(false); end; end; end else loc:=p; end; @ When a token list has been fully scanned, the following computations should be done as we leave that level of input. The |token←type| tends to be equal to either |backed←up| or |inserted| about 2/3 of the time. @↑inner loop@> @p procedure end←token←list; {leave a token-list input level} begin if token←type>=backed←up then {token list to be deleted} begin if token←type<=inserted then flush←list(start) else begin delete←token←ref(start); {update reference count} if token←type=macro then {parameters must be flushed} while param←ptr>param←start do begin decr(param←ptr); flush←list(param←stack[param←ptr]); end; end; end else if token←type=u←template then align←state:=0; pop←input; check←interrupt; end; @ Sometimes \TeX\ has read too far and wants to ``unscan'' what it has seen. The |back←input| procedure takes care of this by putting the token just scanned back into the input stream, ready to be read again. This procedure can be used only if |cur←tok| represents the token to be replaced. Some applications of \TeX\ use this procedure a lot, so it has been slightly optimized for speed. @↑inner loop@> @p procedure back←input; {undoes one token of input} var p:pointer; {a token list of length one} begin while (state=token←list)and(loc=null) do end←token←list; {conserve stack space} p:=get←avail; info(p):=cur←tok; if cur←tok<right←brace←limit then if cur←tok<left←brace←limit then decr(align←state) else incr(align←state); push←input; state:=token←list; start:=p; token←type:=backed←up; loc:=p; {that was |back←list(p)|, without procedure overhead} end; @ @<Insert token |p| into \TeX's input@>= begin t:=cur←tok; cur←tok:=p; back←input; cur←tok:=t; end @ The |back←error| routine is used when we want to replace an offending token just before issuing an error message. This routine, like |back←input|, requires that |cur←tok| has been set. We disable interrupts during the call of |back←input| so that the help message won't be lost. @p procedure back←error; {back up one token and call |error|} begin OK←to←interrupt:=false; back←input; OK←to←interrupt:=true; error; end; @# procedure ins←error; {back up one inserted token and call |error|} begin OK←to←interrupt:=false; back←input; token←type:=inserted; OK←to←interrupt:=true; error; end; @ The |begin←file←reading| procedure starts a new level of input for lines of characters to be read from a file, or as an insertion from the terminal. It does not take care of opening the file, nor does it set |loc| or |limit| or |line|. @↑system dependencies@> @p procedure begin←file←reading; begin if in←open=max←in←open then overflow("text input levels",max←in←open); @:TeX capacity exceeded text input levels}{\quad text input levels@> if first=buf←size then overflow("buffer size",buf←size); @:TeX capacity exceeded buffer size}{\quad buffer size@> incr(in←open); push←input; index:=in←open; line←stack[index]:=line; start:=first; state:=mid←line; name:=0; {|terminal←input| is now |true|} end; @ Conversely, the variables must be downdated when such a level of input is finished: @p procedure end←file←reading; begin first:=start; line:=line←stack[index]; if name>17 then a←close(cur←file); {forget it} pop←input; decr(in←open); end; @ In order to keep the stack from overflowing during a long sequence of inserted `\.{\\show}' commands, the following routine removes completed error-inserted lines from memory. @p procedure clear←for←error←prompt; begin while (state<>token←list)and terminal←input and@| (input←ptr>0)and(loc>limit) do end←file←reading; print←ln; clear←terminal; end; @ To get \TeX's whole input mechanism going, we perform the following actions. @<Initialize the input routines@>= begin input←ptr:=0; max←in←stack:=0; in←open:=0; max←buf←stack:=0; param←ptr:=0; max←param←stack:=0; for first:=0 to buf←size do buffer[first]:=0; scanner←status:=normal; warning←index:=null; first:=1; state:=new←line; start:=1; index:=0; line:=0; name:=0; force←eof:=false; align←state:=1000000;@/ if not init←terminal then goto final←end; limit:=last; first:=last+1; {|init←terminal| has set |loc| and |last|} end @* \[24] Getting the next token. The heart of \TeX's input mechanism is the |get←next| procedure, which we shall develop in the next few sections of the program. Perhaps we shouldn't actually call it the ``heart,'' however, because it really acts as \TeX's eyes and mouth, reading the source files and gobbling them up. And it also helps \TeX\ to regurgitate stored token lists that are to be processed again. @↑eyes and mouth@> The main duty of |get←next| is to input one token and to set |cur←cmd| and |cur←chr| to that token's command code and modifier. Furthermore, if the input token is a control sequence, the |eqtb| location of that control sequence is stored in |cur←cs|; otherwise |cur←cs| is set to zero. Underlying this simple description is a certain amount of complexity because of all the cases that need to be handled, as explained above. However, the inner loop of |get←next| is reasonably short and fast. When |get←next| is asked to get the next token of a \.{\\read} line, it sets |cur←cmd=cur←chr=cur←cs=0| in the case that no more tokens appear on that line. (There might not be any tokens at all, if the |end←line←char| has |ignore| as its catcode.) @ The value of |par←loc| is the |eqtb| address of `\.{\\par}'. This quantity is needed because a blank line of input is supposed to be exactly equivalent to the appearance of \.{\\par}; we must set |cur←cs:=par←loc| when detecting a blank line. @<Glob...@>= @!par←loc:pointer; {location of `\.{\\par}' in |eqtb|} @!par←token:halfword; {token representing '\.{\\par}'} @ @<Put each...@>= primitive("par",par←end,0); par←loc:=cur←val; par←token:=cs←token←flag+par←loc; @!@:par←}{\.{\\par} primitive@> @ @<Cases of |print←cmd←chr|...@>= par←end:print←esc("par"); @ Before getting into |get←next|, let's consider the subroutine that is called when an `\.{\\outer}' control sequence has been scanned or when the end of a file has been reached. These two cases are distinguished by |cur←cs|, which is zero at the end of a file. @p procedure check←outer←validity; var p:pointer; {points to inserted token list} @!q:pointer; {auxiliary pointer} begin if scanner←status<>normal then begin deletions←allowed:=false; @<Back up an outer control sequence so that it can be reread@>; if scanner←status>skipping then @<Tell the user what has run away and try to recover@> else begin print←err("Incomplete "); print←cmd←chr(if←test,cur←if); @.Incomplete \\if...@> print("; all text was ignored after line "); print←int(skip←line); help3("A forbidden control sequence occurred in skipped text.")@/ ("This kind of error happens when you say `\if...' and forget")@/ ("the matching `\fi'. I've inserted a `\fi'; this might work."); if cur←cs<>0 then cur←cs:=0 else help←line[2]:=@| "The file ended while I was skipping conditional text."; cur←tok:=cs←token←flag+frozen←fi; ins←error; end; deletions←allowed:=true; end; end; @ An outer control sequence that occurs in a \.{\\read} will not be reread, since the error recovery for \.{\\read} is not very powerful. @<Back up an outer control sequence so that it can be reread@>= if cur←cs<>0 then begin if (state=token←list)or(name<1)or(name>17) then begin p:=get←avail; info(p):=cs←token←flag+cur←cs; back←list(p); {prepare to read the control sequence again} end; cur←cmd:=spacer; cur←chr:=" "; {replace it by a space} end @ @<Tell the user what has run away...@>= begin runaway; {print a definition, argument, or preamble} if cur←cs=0 then print←err("File ended") @.File ended while scanning...@> else begin cur←cs:=0; print←err("Forbidden control sequence found"); @.Forbidden control sequence...@> end; print(" while scanning "); @<Print either `\.{definition}' or `\.{use}' or `\.{preamble}' or `\.{text}', and insert tokens that should lead to recovery@>; print(" of "); sprint←cs(warning←index); help4("I suspect you have forgotten a `}', causing me")@/ ("to read past where you wanted me to stop.")@/ ("I'll try to recover; but if the error is serious,")@/ ("you'd better type `E' or `X' now and fix your file.");@/ error; end @ The recovery procedure can't be fully understood without knowing more about the \TeX\ routines that should be aborted, but we can sketch the ideas here: For a runaway definition we will insert a right brace; for a runaway preamble, we will insert a special \.{\\cr} token and a right brace; and for a runaway argument, we will set |long←state| to |outer←call| and insert \.{\\par}. @<Print either `\.{definition}' or ...@>= p:=get←avail; case scanner←status of defining:begin print("definition"); info(p):=right←brace←token+"}"; end; matching:begin print("use"); info(p):=par←token; long←state:=outer←call; end; aligning:begin print("preamble"); info(p):=right←brace←token+"}"; q:=p; p:=get←avail; link(p):=q; info(p):=cs←token←flag+frozen←cr; align←state:=-1000000; end; absorbing:begin print("text"); info(p):=right←brace←token+"}"; end; end; {there are no other cases} ins←list(p) @ We need to mention a procedure here that may be called by |get←next|. @p procedure@?firm←up←the←line; forward; @ Now we're ready to take the plunge into |get←next| itself. Parts of this routine are executed more often than any other instructions of \TeX. @↑mastication@>@↑inner loop@> @d switch=25 {a label in |get←next|} @d start←cs=26 {another} @p procedure get←next; {sets |cur←cmd|, |cur←chr|, |cur←cs| to next token} label restart, {go here to get the next input token} switch, {go here to eat the next character from a file} reswitch, {go here to digest it again} start←cs, {go here to start looking for a control sequence} found, {go here when a control sequence has been found} exit; {go here when the next input token has been got} var k:0..buf←size; {an index into |buffer|} @!t:halfword; {a token} @!cat:0..15; {|cat←code(cur←chr)|, usually} begin restart: cur←cs:=0; if state<>token←list then @<Input from external file, |goto restart| if no input found@> else @<Input from token list, |goto restart| if end of list or if a parameter needs to be expanded@>; @<If an alignment entry has just ended, take appropriate action@>; exit:end; @ An alignment entry ends when a tab or \.{\\cr} occurs, provided that the current level of braces is the same as the level that was present at the beginning of that alignment entry; i.e., provided that |align←state| has returned to the value it had after the \<u←j> template for that entry. @↑inner loop@> @<If an alignment entry has just ended, take appropriate action@>= if cur←cmd<=car←ret then if cur←cmd>=tab←mark then if align←state=0 then @<Insert the \(v)\<v←j> template and |goto restart|@> @ @<Input from external file, |goto restart| if no input found@>= @↑inner loop@> begin switch: if loc<=limit then {current line not yet finished} begin cur←chr:=buffer[loc]; incr(loc); reswitch: cur←cmd:=cat←code(cur←chr); @<Change state if necessary, and |goto switch| if the current character should be ignored, or |goto reswitch| if the current character changes to another@>; end else begin state:=new←line;@/ @<Move to next line of file, or |goto restart| if there is no next line, or |return| if a \.{\\read} line has finished@>; check←interrupt; goto switch; end; end @ The following 48-way switch accomplishes the scanning quickly, assuming that a decent \PASCAL\ compiler has translated the code. Note that the numeric values for |mid←line|, |skip←blanks|, and |new←line| are spaced apart from each other by |max←char←code+1|, so we can add a character's command code to the state to get a single number that characterizes both. @d any←state←plus(#) == mid←line+#,skip←blanks+#,new←line+# @<Change state if necessary...@>= case state+cur←cmd of @<Cases where character is ignored@>: goto switch; any←state←plus(escape): @<Scan a control sequence and set |state:=skip←blanks| or |mid←line|@>; any←state←plus(active←char): @<Process an active-character control sequence and set |state:=mid←line|@>; any←state←plus(sup←mark): @<If this |sup←mark| starts a control character like~\.{\↑\↑A}, then |goto reswitch|, otherwise set |state:=mid←line|@>; any←state←plus(invalid←char): @<Decry the invalid character and |goto restart|@>; @t\4@>@<Handle situations involving spaces, braces, changes of state@>@; othercases do←nothing endcases @ @<Cases where character is ignored@>= any←state←plus(ignore),skip←blanks+spacer,new←line+spacer @ We go to |restart| instead of to |switch|, because |state| might equal |token←list| after the error has been dealt with (cf.\ |clear←for←error←prompt|). @<Decry the invalid...@>= begin print←err("Text line contains an invalid character"); @.Text line contains...@> help2("A funny symbol that I can't read has just been input.")@/ ("Continue, and I'll forget that it ever happened.");@/ deletions←allowed:=false; error; deletions←allowed:=true; goto restart; end @ @d add←delims←to(#)==#+math←shift,#+tab←mark,#+mac←param, #+sub←mark,#+letter,#+other←char @<Handle situations involving spaces, braces, changes of state@>= mid←line+spacer:@<Enter |skip←blanks| state, emit a space@>; mid←line+car←ret:@<Finish line, emit a space@>; skip←blanks+car←ret,any←state←plus(comment): @<Finish line, |goto switch|@>; new←line+car←ret:@<Finish line, emit a \.{\\par}@>; mid←line+left←brace: incr(align←state); skip←blanks+left←brace,new←line+left←brace: begin state:=mid←line; incr(align←state); end; mid←line+right←brace: decr(align←state); skip←blanks+right←brace,new←line+right←brace: begin state:=mid←line; decr(align←state); end; add←delims←to(skip←blanks),add←delims←to(new←line): state:=mid←line; @ When a character of type |spacer| gets through, its character code is changed to $\.{"\ "}=@'40$. This means that the ASCII codes for tab and space, and for the space inserted at the end of a line, will be treated alike when macro parameters are being matched. We do this since such characters are indistinguishable on most computer terminal displays. @<Finish line, emit a space@>= begin loc:=limit+1; cur←cmd:=spacer; cur←chr:=" "; end @ The following code is performed only when |cur←cmd=spacer|. @<Enter |skip←blanks| state, emit a space@>= begin state:=skip←blanks; cur←chr:=" "; end @ @<Finish line, |goto switch|@>= begin loc:=limit+1; goto switch; end @ @<Finish line, emit a \.{\\par}@>= begin loc:=limit+1; cur←cs:=par←loc; cur←cmd:=eq←type(cur←cs); cur←chr:=equiv(cur←cs); if cur←cmd>=outer←call then check←outer←validity; end @ @<If this |sup←mark| starts a control character...@>= begin if (cur←chr=buffer[loc])and(loc<limit) then begin if buffer[loc+1]<@'100 then cur←chr:=buffer[loc+1]+@'100 else cur←chr:=buffer[loc+1]-@'100; loc:=loc+2; goto reswitch; end; state:=mid←line; end @ @<Process an active-character...@>= begin cur←cs:=cur←chr+active←base; cur←cmd:=eq←type(cur←cs); cur←chr:=equiv(cur←cs); state:=mid←line; if cur←cmd>=outer←call then check←outer←validity; end @ Control sequence names are scanned only when they appear in some line of a file; once they have been scanned the first time, their |eqtb| location serves as a unique identification, so \TeX\ doesn't need to refer to the original name any more except when it prints the equivalent in symbolic form. The program that scans a control sequence has been written carefully in order to avoid the blowups that might otherwise occur if a malicious user tried something like `\.{\\catcode\'15=0}'. The algorithm might look at |buffer[limit+1]|, but it never looks at |buffer[limit+2]|. If expanded control characters like `\.{\↑\↑A}' appear in or just following a control sequence name, they are converted to single characters in the buffer and the process is repeated, slowly but surely. @<Scan a control...@>= begin if loc>limit then cur←cs:=null←cs {|state| is irrelevant in this case} else begin start←cs: k:=loc; cur←chr:=buffer[k]; cat:=cat←code(cur←chr); incr(k); if cat=letter then state:=skip←blanks else if cat=spacer then state:=skip←blanks else state:=mid←line; if (cat=letter)and(k<=limit) then @<Scan ahead in the buffer until finding a nonletter; if an expanded control code is encountered, reduce it and |goto start←cs|; otherwise if a multiletter control sequence is found, adjust |cur←cs| and |loc|, and |goto found|@> else @<If an expanded control code is present, reduce it and |goto start←cs|@>; cur←cs:=single←base+buffer[loc]; incr(loc); end; found: cur←cmd:=eq←type(cur←cs); cur←chr:=equiv(cur←cs); if cur←cmd>=outer←call then check←outer←validity; end @ Whenever we reach the following piece of code, we will have |cur←chr=buffer[k-1]| and |k<=limit+1| and |cat=cat←code(cur←chr)|. If an expanded control code like \.{\↑\↑A} appears in |buffer[(k-1)..(k+1)]|, we will store the corresponding code in |buffer[k-1]| and shift the rest of the buffer left two places. The value of |cur←chr| may be changed here, but not the value of |cat|. @<If an expanded...@>= begin if buffer[k]=cur←chr then if cat=sup←mark then if k<limit then begin cur←chr:=buffer[k+1]; if cur←chr<@'100 then buffer[k-1]:=cur←chr+@'100 else buffer[k-1]:=cur←chr-@'100; limit:=limit-2; first:=first-2; while k<=limit do begin buffer[k]:=buffer[k+2]; incr(k); end; goto start←cs; end; end @ @<Scan ahead in the buffer...@>= begin repeat cur←chr:=buffer[k]; cat:=cat←code(cur←chr); incr(k); until (cat<>letter)or(k>limit); @<If an expanded...@>; if cat<>letter then decr(k); {now |k| points to first nonletter} if k>loc+1 then {multiletter control sequence has been scanned} begin cur←cs:=id←lookup(loc,k-loc); loc:=k; goto found; end; end @ Let's consider now what happens when |get←next| is looking at a token list. @<Input from token list, |goto restart| if end of list or if a parameter needs to be expanded@>= if loc<>null then {list not exhausted} @↑inner loop@> begin t:=info(loc); loc:=link(loc); {move to next} if t>=cs←token←flag then {a control sequence token} begin cur←cs:=t-cs←token←flag; cur←cmd:=eq←type(cur←cs); cur←chr:=equiv(cur←cs); if cur←cmd>=outer←call then if cur←cmd=dont←expand then @<Get the next token, suppressing expansion@> else check←outer←validity; end else begin cur←cmd:=t div @'400; cur←chr:=t mod @'400; case cur←cmd of left←brace: incr(align←state); right←brace: decr(align←state); out←param: @<Insert macro parameter and |goto restart|@>; othercases do←nothing endcases; end; end else begin {we are done with this token list} end←token←list; goto restart; {resume previous level} end @ The present point in the program is reached only when the |no←expand| routine has inserted a special marker into the input. In this special case, |info(loc)| is known to be a control sequence token, and |link(loc)=null|. @d no←expand←flag=257 {this characterizes a special variant of |relax|} @<Get the next token, suppressing expansion@>= begin cur←cs:=info(loc)-cs←token←flag; loc:=null;@/ cur←cmd:=eq←type(cur←cs); cur←chr:=equiv(cur←cs); if cur←cmd>max←command then begin cur←cmd:=relax; cur←chr:=no←expand←flag; end; end @ @<Insert macro parameter...@>= begin begin←token←list(param←stack[param←start+cur←chr-1],parameter); goto restart; end @ All of the easy branches of |get←next| have now been taken care of. There is one more branch. @<Move to next line of file, or |goto restart|...@>= if name>17 then @<Read next line of file into |buffer|, or |goto restart| if the file has ended@> else begin if not terminal←input then {\.{\\read} line has ended} begin cur←cmd:=0; cur←chr:=0; return; end; if input←ptr>0 then {text was inserted during error recovery} begin end←file←reading; goto restart; {resume previous level} end; if selector<log←only then open←log←file; if interaction>nonstop←mode then begin if limit=start then {previous line was empty} print←nl("(Please type a command or say `\end')"); @.Please type...@> print←ln; first:=start; prompt←input("*"); {input on-line into |buffer|} @.*@> limit:=last; if (end←line←char<0)or(end←line←char>127) then decr(limit) else buffer[limit]:=end←line←char; first:=limit+1; loc:=start; end else fatal←error("*** (job aborted, no legal \end found)"); @.job aborted@> {nonstop mode, which is intended for overnight batch processing, never waits for on-line input} end @ The global variable |force←eof| is normally |false|; it is set |true| by an \.{\\endinput} command. @<Glob...@>= @!force←eof:boolean; {should the next \.{\\input} be aborted early?} @ @<Read next line of file into |buffer|, or |goto restart| if the file has ended@>= begin incr(line); first:=start; if not force←eof then begin if input←ln(cur←file,true) then {not end of file} firm←up←the←line {this sets |limit|} else force←eof:=true; end; if force←eof then begin print←char(")"); force←eof:=false; update←terminal; {show user that file has been read} end←file←reading; {resume previous level} check←outer←validity; goto restart; end; if (end←line←char<0)or(end←line←char>127) then decr(limit) else buffer[limit]:=end←line←char; first:=limit+1; loc:=start; {ready to read} end @ If the user has set the |pausing| parameter to some positive value, and if nonstop mode has not been selected, each line of input is displayed on the terminal and the transcript file, followed by `\.{=>}'. \TeX\ waits for a response. If the response is simply |carriage←return|, the line is accepted as it stands, otherwise the line typed is used instead of the line in the file. @p procedure firm←up←the←line; var k:0..buf←size; {an index into |buffer|} begin limit:=last; if pausing>0 then if interaction>nonstop←mode then begin wake←up←terminal; print←ln; if start<limit then for k:=start to limit-1 do print(buffer[k]); first:=limit; prompt←input("=>"); {wait for user response} @.=>@> if last>first then begin for k:=first to last-1 do {move line down in buffer} buffer[k+start-first]:=buffer[k]; limit:=start+last-first; end; end; end; @ Since |get←next| is used so frequently in \TeX, it is convenient to define three related procedures that do a little more: \yskip\hang|get←token| not only sets |cur←cmd| and |cur←chr|, it also sets |cur←tok|, a packed halfword version of the current token. \yskip\hang|get←x←token|, meaning ``get an expanded token,'' is like |get←token|, but if the current token turns out to be a user-defined control sequence (i.e., a macro call), or a conditional, or something like \.{\\topmark} or \.{\\expandafter} or \.{\\csname}, it is eliminated from the input by beginning the expansion of the macro or the evaluation of the conditional. \yskip\hang|x←token| is like |get←x←token| except that it assumes that |get←next| has already been called. \yskip\noindent In fact, these three procedures account for {\sl all\/} uses of |get←next|, except for two places in the ``inner loop'' when |cur←tok| need not be set, and except when the arguments to \.{\\ifx} are being scanned. @ No new control sequences will be defined except during a call of |get←token|, or when \.{\\csname} compresses a token list, because |no←new←control←sequence| is always |true| at other times. @p procedure get←token; {sets |cur←cmd|, |cur←chr|, |cur←tok|} begin no←new←control←sequence:=false; get←next; no←new←control←sequence:=true; @↑inner loop@> if cur←cs=0 then cur←tok:=(cur←cmd*@'400)+cur←chr else cur←tok:=cs←token←flag+cur←cs; end; @* \[25] Expanding the next token. Only a dozen or so command codes |>max←command| can possibly be returned by |get←next|; in increasing order, they are |undefined←cs|, |expand←after|, |no←expand|, |input|, |if←test|, |fi←or←else|, |cs←name|, |convert|, |the|, |top←bot←mark|, |call|, |long←call|, |outer←call|, |long←outer←call|, and |end←template|. The |expand| subroutine is used when |cur←cmd>max←command|. It removes a ``call'' or a conditional or one of the other special operations just listed. It follows that |expand| might invoke itself recursively. In all cases, |expand| destroys the current token, but it sets things up so that the next |get←next| will deliver the appropriate next token. The value of |cur←tok| need not be known when |expand| is called. Since several of the basic scanning routines communicate via global variables, their values are saved as local variables of |expand| so that recursive calls don't invalidate them. @↑recursion@> @p@t\4@>@<Declare the procedure called |macro←call|@>@;@/ @t\4@>@<Declare the procedure called |insert←relax|@>@;@/ procedure@?pass←text; forward;@t\2@> procedure@?start←input; forward;@t\2@> procedure@?conditional; forward;@t\2@> procedure@?get←x←token; forward;@t\2@> procedure@?conv←toks; forward;@t\2@> procedure@?ins←the←toks; forward;@t\2@> procedure expand; var t:halfword; {token that is being ``expanded after''} @!p,@!q,@!r:pointer; {for list manipulation} @!j:0..buf←size; {index into |buffer|} @!cv←backup:integer; {to save the global quantity |cur←val|} @!cvl←backup,@!radix←backup:small←number; {to save |cur←val←level| and |radix|} @!backup←backup:pointer; {to save |link(backup←head)|} @!save←scanner←status:small←number; {temporary storage of |scanner←status|} begin cv←backup:=cur←val; cvl←backup:=cur←val←level; radix←backup:=radix; backup←backup:=link(backup←head); if cur←cmd<call then @<Expand a nonmacro@> else if cur←cmd<end←template then macro←call else @<Insert a |frozen←endv| token@>; cur←val:=cv←backup; cur←val←level:=cvl←backup; radix:=radix←backup; link(backup←head):=backup←backup; end; @ @<Expand a nonmacro@>= begin if tracing←commands>1 then show←cur←cmd←chr; case cur←cmd of top←bot←mark:@<Insert the \(a)appropriate mark text into the scanner@>; expand←after:@<Expand the token after the next token@>; no←expand:@<Suppress expansion of the next token@>; cs←name:@<Manufacture a control sequence name@>; convert:conv←toks; {this procedure is discussed in part 27 below} the:ins←the←toks; {this procedure is discussed in part 27 below} if←test:conditional; {this procedure is discussed in part 28 below} fi←or←else:@<Terminate the current conditional and skip to \.{\\fi}@>; input:@<Initiate or terminate input from a file@>; othercases @<Complain about an undefined macro@> endcases; end @ It takes only a little shuffling to do what \TeX\ calls \.{\\expandafter}. @<Expand the token after...@>= begin get←token; t:=cur←tok; get←token; if cur←cmd>max←command then expand@+else back←input; cur←tok:=t; back←input; end @ The implement of \.{\\noexpand} is a bit trickier, because it is necessary to insert a special `|dont←expand|' marker into \TeX's reading mechanism. This special marker is processed by |get←next|, but it does not slow down the inner loop. Since \.{\\outer} macros might arise here, we must also clear the |scanner←status| temporarily. @<Suppress expansion...@>= begin save←scanner←status:=scanner←status; scanner←status:=normal; get←token; scanner←status:=save←scanner←status; t:=cur←tok; back←input; {now |start| and |loc| point to the backed-up token |t|} if t>=cs←token←flag then begin p:=get←avail; info(p):=cs←token←flag+frozen←dont←expand; link(p):=loc; start:=p; loc:=p; end; end @ @<Complain about an undefined macro@>= begin print←err("Undefined control sequence"); @.Undefined control sequence@> help5("The control sequence at the end of the top line")@/ ("of your error message was never \def'ed. If you have")@/ ("misspelled it (e.g., `\hobx'), type `I' and the correct")@/ ("spelling (e.g., `I\hbox'). Otherwise just continue,")@/ ("and I'll forget about whatever was undefined."); error; end @ The |expand| procedure and some other routines that construct token lists find it convenient to use the following macros, which are valid only if the variables |p| and |q| are reserved for token-list building. @d store←new←token(#)==begin q:=get←avail; link(p):=q; info(q):=#; p:=q; {|link(p)| is |null|} end @d fast←store←new←token(#)==begin fast←get←avail(q); link(p):=q; info(q):=#; p:=q; {|link(p)| is |null|} end @ @<Manufacture a control...@>= begin r:=get←avail; p:=r; {head of the list of characters} repeat get←x←token; if cur←cs=0 then store←new←token(cur←tok); until cur←cs<>0; if cur←cmd<>end←cs←name then @<Complain about missing \.{\\endcsname}@>; @<Look up the characters of list |r| in the hash table, and set |cur←cs|@>; flush←list(r); if eq←type(cur←cs)=undefined←cs then begin eqtb[cur←cs]:=eqtb[frozen←relax]; end; {the control sequence will now match `\.{\\relax}'} cur←tok:=cur←cs+cs←token←flag; back←input; end @ @<Complain about missing \.{\\endcsname}@>= begin print←err("Missing "); print←esc("endcsname"); print(" inserted"); @.Missing \\endcsname...@> help2("The control sequence marked <to be read again> should")@/ ("not appear between \csname and \endcsname."); back←error; end @ @<Look up the characters of list |r| in the hash table...@>= j:=first; p:=link(r); while p<>null do begin if j>=max←buf←stack then begin max←buf←stack:=j+1; if max←buf←stack=buf←size then overflow("buffer size",buf←size); @:TeX capacity exceeded buffer size}{\quad buffer size@> end; buffer[j]:=info(p) mod @'400; incr(j); p:=link(p); end; if j>first+1 then begin no←new←control←sequence:=false; cur←cs:=id←lookup(first,j-first); no←new←control←sequence:=true; end else if j=first then cur←cs:=null←cs {the list is empty} else cur←cs:=single←base+buffer[first] {the list has length one} @ An |end←template| command is effectively changed to an |endv| command by the following code. (The reason for this is discussed below; the |frozen←end←template| at the end of the template has passed the |check←outer←validity| test, so its mission of error detection has been accomplished.) @<Insert a |frozen←endv| token@>= begin cur←tok:=cs←token←flag+frozen←endv; back←input; end @ The processing of \.{\\input} involves the |start←input| subroutine, which will be declared later; the processing of \.{\\endinput} is trivial. @<Put each...@>= primitive("input",input,0);@/ @!@:input←}{\.{\\input} primitive@> primitive("endinput",input,1);@/ @!@:end←input←}{\.{\\endinput} primitive@> @ @<Cases of |print←cmd←chr|...@>= input: if chr←code=0 then print←esc("input")@+else print←esc("endinput"); @ @<Initiate or terminate input...@>= if cur←chr>0 then force←eof:=true else if name←in←progress then insert←relax else start←input @ Sometimes the expansion looks too far ahead, so we want to insert a harmless \.{\\relax} into the user's input. @<Declare the procedure called |insert←relax|@>= procedure insert←relax; begin cur←tok:=cs←token←flag+cur←cs; back←input; cur←tok:=cs←token←flag+frozen←relax; back←input; token←type:=inserted; end; @ Here is a recursive procedure that is \TeX's usual way to get the next token of input. It has been slightly optimized to take account of common cases. @p procedure get←x←token; {sets |cur←cmd|, |cur←chr|, |cur←tok|, and expands macros} label restart,done; begin restart: get←next; @↑inner loop@> if cur←cmd<=max←command then goto done; if cur←cmd>=call then if cur←cmd<end←template then macro←call else begin cur←cs:=frozen←endv; cur←cmd:=endv; goto done; {|cur←chr=null←list|} end else expand; goto restart; done: if cur←cs=0 then cur←tok:=(cur←cmd*@'400)+cur←chr else cur←tok:=cs←token←flag+cur←cs; end; @ The |get←x←token| procedure is equivalent to two consecutive procedure calls: |get←next; x←token|. @p procedure x←token; {|get←x←token| without the initial |get←next|} begin while cur←cmd>max←command do begin expand; get←next; end; if cur←cs=0 then cur←tok:=(cur←cmd*@'400)+cur←chr else cur←tok:=cs←token←flag+cur←cs; end; @ A control sequence that has been \.{\\def}'ed by the user is expanded by \TeX's |macro←call| procedure. Before we get into the details of |macro←call|, however, let's consider the treatment of primitives like \.{\\topmark}, since they are essentially macros without parameters. The token lists for such marks are kept in a global array of five pointers; we refer to the individual entries of this array by symbolic names |top←mark|, etc. The value of |top←mark| is either |null| or a pointer to the reference count of a token list. @d top←mark←code=0 {the mark in effect at the previous page break} @d first←mark←code=1 {the first mark between |top←mark| and |bot←mark|} @d bot←mark←code=2 {the mark in effect at the current page break} @d split←first←mark←code=3 {the first mark found by \.{\\vsplit}} @d split←bot←mark←code=4 {the last mark found by \.{\\vsplit}} @d top←mark==cur←mark[top←mark←code] @d first←mark==cur←mark[first←mark←code] @d bot←mark==cur←mark[bot←mark←code] @d split←first←mark==cur←mark[split←first←mark←code] @d split←bot←mark==cur←mark[split←bot←mark←code] @<Glob...@>= @!cur←mark:array[top←mark←code..split←bot←mark←code] of pointer; {token lists for marks} @ @<Set init...@>= top←mark:=null; first←mark:=null; bot←mark:=null; split←first←mark:=null; split←bot←mark:=null; @ @<Put each...@>= primitive("topmark",top←bot←mark,top←mark←code); @!@:top←mark←}{\.{\\topmark} primitive@> primitive("firstmark",top←bot←mark,first←mark←code); @!@:first←mark←}{\.{\\firstmark} primitive@> primitive("botmark",top←bot←mark,bot←mark←code); @!@:bot←mark←}{\.{\\botmark} primitive@> primitive("splitfirstmark",top←bot←mark,split←first←mark←code); @!@:split←first←mark←}{\.{\\splitfirstmark} primitive@> primitive("splitbotmark",top←bot←mark,split←bot←mark←code); @!@:split←bot←mark←}{\.{\\splitbotmark} primitive@> @ @<Cases of |print←cmd←chr|...@>= top←bot←mark: case chr←code of first←mark←code: print←esc("firstmark"); bot←mark←code: print←esc("botmark"); split←first←mark←code: print←esc("splitfirstmark"); split←bot←mark←code: print←esc("splitbotmark"); othercases print←esc("topmark") endcases; @ The following code is activated when |cur←cmd=top←bot←mark| and when |cur←chr| is a code like |top←mark←code|. @<Insert the \(a)appropriate mark text into the scanner@>= begin if cur←mark[cur←chr]<>null then begin←token←list(cur←mark[cur←chr],mark←text); end @ Now let's consider |macro←call| itself, which is invoked when \TeX\ is scanning a control sequence whose |cur←cmd| is either |call|, |long←call|, |outer←call|, or |long←outer←call|. The control sequence definition appears in the token list whose reference count is in location |cur←chr| of |mem|. The global variable |long←state| will be set to |call| or to |long←call|, depending on whether or not the control sequence disallows \.{\\par} in its parameters. The |get←next| routine will set |long←state| to |outer←call| and emit \.{\\par}, if a file ends or if an \.{\\outer} control sequence occurs in the midst of an argument. @<Glob...@>= @!long←state:call..long←outer←call; {governs the acceptance of \.{\\par}} @ The parameters, if any, must be scanned before the macro is expanded. Parameters are token lists without reference counts. They are placed on an auxiliary stack called |pstack| while they are being scanned, since the |param←stack| may be losing entries during the matching process. (Note that |param←stack| can't be gaining entries, since |macro←call| is the only routine that puts anything onto |param←stack|, and it is not recursive.) @<Glob...@>= @!pstack:array[0..8] of pointer; {arguments supplied to a macro} @ After parameter scanning is complete, the parameters are moved to the |param←stack|. Then the macro body is fed to the scanner; in other words, |macro←call| places the defined text of the control sequence at the top of the \TeX's input stack, so that |get←next| will proceed to read it next. The global variable |cur←cs| contains the |eqtb| address of the control sequence being expanded, when |macro←call| begins. If this control sequence has not been declared \.{\\long}, i.e., if its command code in the |eq←type| field is not |long←call| or |long←outer←call|, its parameters are not allowed to contain the control sequence \.{\\par}. If an illegal \.{\\par} appears, the macro call is aborted, and the \.{\\par} will be rescanned. @<Declare the procedure called |macro←call|@>= procedure macro←call; {invokes a user-defined control sequence} label exit, continue, done, done1, found; var r:pointer; {current node in the macro's token list} @!p:pointer; {current node in parameter token list being built} @!q:pointer; {new node being put into the token list} @!s:pointer; {backup pointer for parameter matching} @!t:pointer; {cycle pointer for backup recovery} @!u,@!v:pointer; {auxiliary pointers for backup recovery} @!rbrace←ptr:pointer; {one step before the last |right←brace| token} @!n:small←number; {the number of parameters scanned} @!unbalance:halfword; {unmatched left braces in current parameter} @!m:halfword; {the number of tokens or groups (usually)} @!ref←count:pointer; {start of the token list} @!save←scanner←status:small←number; {|scanner←status| upon entry} @!save←warning←index:pointer; {|warning←index| upon entry} @!match←chr:ASCII←code; {character used in parameter} begin save←scanner←status:=scanner←status; save←warning←index:=warning←index; warning←index:=cur←cs; ref←count:=cur←chr; r:=link(ref←count); n:=0; if tracing←macros>0 then @<Show the text of the macro being expanded@>; if info(r)<>end←match←token then @<Scan the parameters and make |link(r)| point to the macro body; but |return| if an illegal \.{\\par} is detected@>; @<Feed the macro body and its parameters to the scanner@>; exit:scanner←status:=save←scanner←status; warning←index:=save←warning←index; end; @ Before we put a new token list on the input stack, it is wise to clean off all token lists that have recently been depleted. Then a user macro that ends with a call to itself will not require unbounded stack space. @<Feed the macro body and its parameters to the scanner@>= while (state=token←list)and(loc=null) do end←token←list; {conserve stack space} begin←token←list(ref←count,macro); name:=warning←index; loc:=link(r); if n>0 then begin if param←ptr+n>max←param←stack then begin max←param←stack:=param←ptr+n; if max←param←stack>param←size then overflow("parameter stack size",param←size); @:TeX capacity exceeded parameter stack size}{\quad parameter stack size@> end; for m:=0 to n-1 do param←stack[param←ptr+m]:=pstack[m]; param←ptr:=param←ptr+n; end @ At this point, the reader will find it advisable to review the explanation of token list format that was presented earlier, since many aspects of that format are of importance chiefly in the |macro←call| routine. The token list might begin with a string of compulsory tokens before the first |match| or |end←match|. In that case the macro name is supposed to be followed by those tokens; the following program will set |s=null| to represent this restriction. Otherwise |s| will be set to the first token of a string that will delimit the next parameter. @<Scan the parameters and make |link(r)| point to the macro body...@>= begin scanner←status:=matching; unbalance:=0; long←state:=eq←type(cur←cs); if long←state>=outer←call then long←state:=long←state-2; repeat if (info(r)>match←token+127)or(info(r)<match←token) then s:=null else begin match←chr:=info(r)-match←token; s:=link(r); r:=s; p:=temp←head; link(p):=null; m:=0; end; @<Scan a parameter until its delimiter string has been found; or, if |s=null|, simply scan the delimiter string@>;@/ {now |info(r)| is a token whose command code is either |match| or |end←match|} until info(r)=end←match←token; end @ If |info(r)| is a |match| or |end←match| command, it cannot be equal to any token found by |get←token|. Therefore an undelimited parameter---i.e., a |match| that is immediately followed by |match| or |end←match|---will always fail the test `|cur←tok=info(r)|' in the following algorithm. @<Scan a parameter until its delimiter string has been found; or, ...@>= continue: get←token; {set |cur←tok| to the next token of input} if cur←tok=info(r) then @<Advance \(r)|r|; |goto found| if the parameter delimiter has been fully matched, otherwise |goto continue|@>; @<Contribute the recently matched tokens to the current parameter, and |goto continue| if a partial match is still in effect; but abort if |s=null|@>; if cur←tok=par←token then if long←state<>long←call then @<Report a runaway argument and abort@>; if cur←tok<right←brace←limit then if cur←tok<left←brace←limit then @<Contribute an entire group to the current parameter@> else @<Report an extra right brace and |goto continue|@> else @<Store the current token, but |goto continue| if it is a blank space that would become an undelimited parameter@>; incr(m); if info(r)>end←match←token then goto continue; if info(r)<match←token then goto continue; found: if s<>null then @<Tidy up the parameter just scanned, and tuck it away@> @ @<Store the current token, but |goto continue| if it is...@>= begin if cur←tok=space←token then if info(r)<=end←match←token then if info(r)>=match←token then goto continue; store←new←token(cur←tok); end @ A slightly subtle point arises here: When the parameter delimiter ends with `\.{\#\{}', the token list will have a left brace both before and after the |end←match|\kern-.4pt. Only one of these should affect the |align←state|, but both will be scanned, so we must make a correction. @<Advance \(r)|r|; |goto found| if the parameter delimiter has been fully...@>= begin r:=link(r); if (info(r)>=match←token)and(info(r)<=end←match←token) then begin if cur←tok<left←brace←limit then decr(align←state); goto found; end else goto continue; end @ @<Report an extra right brace and |goto continue|@>= begin back←input; print←err("Argument of "); sprint←cs(warning←index); @.Argument of \\x has...@> print(" has an extra }"); help6("I've run across a `}' that doesn't seem to match anything.")@/ ("For example, `\def\a#1{...}' and `\a}' would produce")@/ ("this error. If you simply proceed now, the `\par' that")@/ ("I've just inserted will cause me to report a runaway")@/ ("argument that might be the root of the problem. But if")@/ ("your `}' was spurious, just type `2' and it will go away."); incr(align←state); long←state:=call; cur←tok:=par←token; ins←error; end @ If |long←state=outer←call|, a runaway argument has already been reported. @<Report a runaway argument and abort@>= begin if long←state=call then begin runaway; print←err("Paragraph ended before "); @.Paragraph ended before...@> sprint←cs(warning←index); print(" was complete"); help3("I suspect you've forgotten a `}', causing me to apply this")@/ ("control sequence to too much text. How can we recover?")@/ ("My plan is to forget the whole thing and hope for the best."); back←error; end; pstack[n]:=link(temp←head); align←state:=align←state-unbalance; for m:=0 to n do flush←list(pstack[m]); return; end @ When the following code becomes active, we have matched tokens from |s| to the predecessor of |r|, and we have found that |cur←tok<>info(r)|. An interesting situation now presents itself: If the parameter is to be delimited by a string such as `\.{ab}', and if we have scanned `\.{aa}', we want to contribute one `\.a' to the current parameter and resume looking for a `\.b'. The program must account for such partial matches and for others that can be quite complex. But most of the time we have |s=r| and nothing needs to be done. Incidentally, it is possible for \.{\\par} tokens to sneak in to certain parameters of non-\.{\\long} macros. For example, consider a case like `\.{\\def\\a\#1\\par!\{...\}}' where the first \.{\\par} is not followed by an exclamation point. In such situations it does not seem appropriate to prohibit the \.{\\par}, so \TeX\ keeps quiet about this bending of the rules. @<Contribute the recently matched tokens to the current parameter...@>= if s<>r then if s=null then @<Report an improper use of the macro and abort@> else begin t:=s; repeat store←new←token(info(t)); incr(m); u:=link(t); v:=s; loop@+ begin if u=r then if cur←tok<>info(v) then goto done else begin r:=link(v); goto continue; end; if info(u)<>info(v) then goto done; u:=link(u); v:=link(v); end; done: t:=link(t); until t=r; r:=s; {at this point, no tokens are recently matched} end @ @<Report an improper use...@>= begin print←err("Use of "); sprint←cs(warning←index); @.Use of x doesn't match...@> print(" doesn't match its definition"); help4("If you say, e.g., `\def\a1{...}', then you must always")@/ ("put `1' after `\a', since control sequence names are")@/ ("made up of letters only. The macro here has not been")@/ ("followed by the required stuff, so I'm ignoring it."); error; return; end @ @<Contribute an entire group to the current parameter@>= begin unbalance:=1; @↑inner loop@> loop@+ begin fast←store←new←token(cur←tok); get←token; if cur←tok=par←token then if long←state<>long←call then @<Report a runaway argument and abort@>; if cur←tok<right←brace←limit then if cur←tok<left←brace←limit then incr(unbalance) else begin decr(unbalance); if unbalance=0 then goto done1; end; end; done1: rbrace←ptr:=p; store←new←token(cur←tok); end @ If the parameter consists of a single group enclosed in braces, we must strip off the enclosing braces. That's why |rbrace←ptr| was introduced. @<Tidy up the parameter just scanned, and tuck it away@>= begin if (m=1)and(info(p)<right←brace←limit)and(p<>temp←head) then begin link(rbrace←ptr):=null; free←avail(p); p:=link(temp←head); pstack[n]:=link(p); free←avail(p); end else pstack[n]:=link(temp←head); incr(n); if tracing←macros>0 then begin begin←diagnostic; print←nl(match←chr); print←int(n); print("<-"); show←token←list(pstack[n-1],null,1000); end←diagnostic(false); end; end @ @<Show the text of the macro being expanded@>= begin begin←diagnostic; print←ln; print←cs(warning←index); token←show(ref←count); end←diagnostic(false); end @* \[26] Basic scanning subroutines. Let's turn now to some procedures that \TeX\ calls upon frequently to digest certain kinds of patterns in the input. Most of these are quite simple; some are quite elaborate. Almost all of the routines call |get←x←token|, which can cause them to be invoked recursively. @↑stomach@> @↑recursion@> @ The |scan←left←brace| routine is called when a left brace is supposed to be the next non-blank token. (The term ``left brace'' means, more precisely, a character whose catcode is |left←brace|.) \TeX\ allows \.{\\relax} to appear before the |left←brace|. @p procedure scan←left←brace; {reads a mandatory |left←brace|} begin @<Get the next non-blank non-relax non-call token@>; if cur←cmd<>left←brace then begin print←err("Missing { inserted"); @.Missing \{ inserted@> help4("A left brace was mandatory here, so I've put one in.")@/ ("You might want to delete and/or insert some corrections")@/ ("so that I will find a matching right brace soon.")@/ ("(If you're confused by all this, try typing `I}' now.)"); back←error; cur←tok:=left←brace←token+"{"; cur←cmd:=left←brace; cur←chr:="{"; incr(align←state); end; end; @ @<Get the next non-blank non-relax non-call token@>= repeat get←x←token; until (cur←cmd<>spacer)and(cur←cmd<>relax) @ The |scan←optional←equals| routine looks for an optional `\.=' sign preceded by optional spaces; `\.{\\relax}' is not ignored here. @p procedure scan←optional←equals; begin @<Get the next non-blank non-call token@>; if cur←tok<>other←token+"=" then back←input; end; @ @<Get the next non-blank non-call token@>= repeat get←x←token; until cur←cmd<>spacer @ In case you are getting bored, here is a slightly less trivial routine: Given a string of lowercase letters, like `\.{pt}' or `\.{plus}' or `\.{width}', the |scan←keyword| routine checks to see whether the next tokens of input match this string. The match must be exact, except that uppercase letters will match their lowercase counterparts; uppercase equivalents are determined by subtracting |"a"-"A"|, rather than using the |uc←code| table, since \TeX\ uses this routine only for its own limited set of keywords. If a match is found, the characters are effectively removed from the input and |true| is returned. Otherwise |false| is returned, and the input is left essentially unchanged (except for the fact that some macros may have been expanded, etc.). @↑inner loop@> @p function scan←keyword(@!s:str←number):boolean; {look for a given string} label exit; var p:pointer; {tail of the backup list} @!q:pointer; {new node being added to the token list via |store←new←token|} @!k:pool←pointer; {index into |str←pool|} begin p:=backup←head; link(p):=null; k:=str←start[s]; while k<str←start[s+1] do begin get←x←token; {recursion is possible here} if (cur←cs=0)and@| ((cur←chr=str←pool[k])or(cur←chr=str←pool[k]-"a"+"A")) then begin store←new←token(cur←tok); incr(k); end else if (cur←cmd<>spacer)or(p<>backup←head) then begin back←input; if p<>backup←head then back←list(link(backup←head)); scan←keyword:=false; return; end; end; flush←list(link(backup←head)); scan←keyword:=true; exit:end; @ Here is a procedure that sounds an alarm when mu and non-mu units are being switched. @p procedure mu←error; begin print←err("Incompatible glue units"); @.Incompatible glue units@> help1("I'm going to assume that 1mu=1pt when they're mixed."); error; end; @ The next routine `|scan←something←internal|' is used to fetch internal numeric quantities like `\.{\\hsize}', and also to handle the `\.{\\the}' when expanding constructions like `\.{\\the\\toks0}' and `\.{\\the\\baselineskip}'. Soon we will be considering the |scan←int| procedure, which calls |scan←something←internal|; on the other hand, |scan←something←internal| also calls |scan←int|, for constructions like `\.{\\catcode\`\\\$}' or `\.{\\fontdimen} \.3 \.{\\ff}'. So we have to declare |scan←int| as a |forward| procedure. A few other procedures are also declared at this point. @p procedure@?scan←int; forward; {scans an integer value} @t\4\4@>@<Declare procedures that scan restricted classes of integers@>@; @t\4\4@>@<Declare procedures that scan font-related stuff@> @ \TeX\ doesn't know exactly what to expect when |scan←something←internal| begins. For example, an integer or dimension or glue value could occur immediately after `\.{\\hskip}'; and one can even say \.{\\the} with respect to token lists in constructions like `\.{\\xdef\\o\{\\the\\output\}}'. On the other hand, only integers are allowed after a construction like `\.{\\count}'. To handle the various possibilities, |scan←something←internal| has a |level| parameter, which tells the ``highest'' kind of quantity that |scan←something←internal| is allowed to produce. Six levels are distinguished, namely |int←val|, |dimen←val|, |glue←val|, |mu←val|, |ident←val|, and |tok←val|. The output of |scan←something←internal| (and of the other routines |scan←int|, |scan←dimen|, and |scan←glue| below) is put into the global variable |cur←val|, and its level is put into |cur←val←level|. The highest values of |cur←val←level| are special: |mu←val| is used only when |cur←val| points to something in a ``muskip'' register, or to one of the three parameters \.{\\thinmuskip}, \.{\\midmuskip}, \.{\\thickmuskip}; |ident←val| is used only when |cur←val| points to a font identifier; |tok←val| is used only when |cur←val| points to |null| or to the reference count of a token list. The last two cases are allowed only when |scan←something←internal| is called with |level=tok←val|. If the output is glue, |cur←val| will point to a glue specification, and the reference count of that glue will have been updated to reflect this reference; if the output is a nonempty token list, |cur←val| will point to its reference count, but in this case the count will not have been updated. Otherwise |cur←val| will contain the integer or scaled value in question. @d int←val=0 {integer values} @d dimen←val=1 {dimension values} @d glue←val=2 {glue specifications} @d mu←val=3 {math glue specifications} @d ident←val=4 {font identifier} @d tok←val=5 {token lists} @<Glob...@>= @!cur←val:integer; {value returned by numeric scanners} @!cur←val←level:int←val..tok←val; {the ``level'' of this value} @ The hash table is initialized with `\.{\\count}', `\.{\\dimen}', `\.{\\skip}', and `\.{\\muskip}' all having |register| as their command code; they are distinguished by the |chr←code|, which is either |int←val|, |dimen←val|, |glue←val|, or |mu←val|. @<Put each...@>= primitive("count",register,int←val); @!@:count←}{\.{\\count} primitive@> primitive("dimen",register,dimen←val); @!@:dimen←}{\.{\\dimen} primitive@> primitive("skip",register,glue←val); @!@:skip←}{\.{\\skip} primitive@> primitive("muskip",register,mu←val); @!@:mu←skip←}{\.{\\muskip} primitive@> @ @<Cases of |print←cmd←chr|...@>= register: if chr←code=int←val then print←esc("count") else if chr←code=dimen←val then print←esc("dimen") else if chr←code=glue←val then print←esc("skip") else print←esc("muskip"); @ OK, we're ready for |scan←something←internal| itself. A second parameter, |negative|, is set |true| if the value that is found should be negated. It is assumed that |cur←cmd| and |cur←chr| represent the first token of the internal quantity to be scanned; an error will be signalled if |cur←cmd<min←internal| or |cur←cmd>max←internal|. @d scanned←result←end(#)==cur←val←level:=#;@+end @d scanned←result(#)==@+begin cur←val:=#;scanned←result←end @p procedure scan←something←internal(@!level:small←number;@!negative:boolean); {fetch an internal parameter} var m:halfword; {|chr←code| part of the operand token} @!p:0..nest←size; {index into |nest|} begin m:=cur←chr; case cur←cmd of def←code: @<Fetch a character code from some table@>; toks←register,assign←toks,def←family,set←font,def←font: @<Fetch a token list or font identifier, provided that |level=tok←val|@>; assign←int: scanned←result(eqtb[m].int)(int←val); assign←dimen: scanned←result(eqtb[m].sc)(dimen←val); assign←glue: scanned←result(equiv(m))(glue←val); assign←mu←glue: scanned←result(equiv(m))(mu←val); set←aux: @<Fetch the |space←factor| or the |prev←depth|@>; set←prev←graf: @<Fetch the |prev←graf|@>; set←page←int:@<Fetch the |dead←cycles| or the |insert←penalties|@>; set←page←dimen: @<Fetch something on the |page←so←far|@>; set←shape: @<Fetch the |par←shape| size@>; set←box←dimen: @<Fetch a box dimension@>; char←given,math←given: scanned←result(cur←chr)(int←val); assign←font←dimen: @<Fetch a font dimension@>; assign←font←int: @<Fetch a font integer@>; register: @<Fetch a register@>; last←item: @<Fetch an item in the current node, if appropriate@>; othercases @<Complain that \.{\\the} can't do this; give zero result@> endcases;@/ while cur←val←level>level do @<Convert \(c)|cur←val| to a lower level@>; @<Fix the reference count, if any, and negate |cur←val| if |negative|@>; end; @ @<Fetch a character code from some table@>= begin scan←seven←bit←int; if m=math←code←base then scanned←result(ho(math←code(cur←val)))(int←val) else if m<math←code←base then scanned←result(equiv(m+cur←val))(int←val) else scanned←result(eqtb[m+cur←val].int)(int←val); end @ @<Fetch a token list...@>= if level<>tok←val then begin print←err("Missing number, treated as zero"); @.Missing number...@> help3("A number should have been here; I inserted `0'.")@/ ("(If you can't figure out why I needed to see a number,")@/ ("look up `weird error' in the index to The TeXbook.)"); @:TeXbook}{\sl The \TeX book@> back←error; scanned←result(0)(dimen←val); end else if cur←cmd<=assign←toks then begin if cur←cmd<assign←toks then {|cur←cmd=toks←register|} begin scan←eight←bit←int; m:=toks←base+cur←val; end; scanned←result(equiv(m))(tok←val); end else begin back←input; scan←font←ident; scanned←result(font←id←base+cur←val)(ident←val); end @ A user is allowed to refer to `\.{\\the\\spacefactor}' only in horizontal mode, and to `\.{\\the\\prevdepth}' only in vertical mode; so we put the associated mode in the modifier part of the |set←aux| command. The |set←page←int| command has modifier 0 or 1, for `\.{\\deadcycles}' and `\.{\\insertpenalties}', respectively. The |set←box←dimen| command is modified by either |width←offset|, |height←offset|, or |depth←offset|. The |last←item| command is modified by either |int←val|, |dimen←val|, or |glue←val|. @<Put each...@>= primitive("spacefactor",set←aux,hmode); @!@:space←factor←}{\.{\\spacefactor} primitive@> primitive("prevdepth",set←aux,vmode);@/ @!@:prev←depth←}{\.{\\prevdepth} primitive@> primitive("deadcycles",set←page←int,0); @!@:dead←cycles←}{\.{\\deadcycles} primitive@> primitive("insertpenalties",set←page←int,1); @!@:insert←penalties←}{\.{\\insertpenalties} primitive@> primitive("wd",set←box←dimen,width←offset); @!@:wd←}{\.{\\wd} primitive@> primitive("ht",set←box←dimen,height←offset); @!@:ht←}{\.{\\ht} primitive@> primitive("dp",set←box←dimen,depth←offset); @!@:dp←}{\.{\\dp} primitive@> primitive("lastpenalty",last←item,int←val); @!@:last←penalty←}{\.{\\lastpenalty} primitive@> primitive("lastkern",last←item,dimen←val); @!@:last←kern←}{\.{\\lastkern} primitive@> primitive("lastskip",last←item,glue←val); @!@:last←skip←}{\.{\\lastskip} primitive@> @ @<Cases of |print←cmd←chr|...@>= set←aux: if chr←code=vmode then print←esc("prevdepth") else print←esc("spacefactor"); set←page←int: if chr←code=0 then print←esc("deadcycles") else print←esc("insertpenalties"); set←box←dimen: if chr←code=width←offset then print←esc("wd") else if chr←code=height←offset then print←esc("ht") else print←esc("dp"); last←item: if chr←code=int←val then print←esc("lastpenalty") else if chr←code=dimen←val then print←esc("lastkern") else print←esc("lastskip"); @ @<Fetch the |space←factor| or the |prev←depth|@>= if abs(mode)<>m then begin print←err("Improper "); print←cmd←chr(set←aux,m); @.Improper \\spacefactor@> @.Improper \\prevdepth@> help4("You can refer to \spacefactor only in horizontal mode;")@/ ("you can refer to \prevdepth only in vertical mode; and")@/ ("neither of these is meaningful inside \write. So")@/ ("I'm forgetting what you said and using zero instead."); error; if level<>tok←val then scanned←result(0)(dimen←val) else scanned←result(0)(int←val); end else begin cur←val:=aux; if m=vmode then cur←val←level:=dimen←val@+else cur←val←level:=int←val; end @ @<Fetch the |dead←cycles| or the |insert←penalties|@>= begin if m=0 then cur←val:=dead←cycles@+else cur←val:=insert←penalties; cur←val←level:=int←val; end @ @<Fetch a box dimension@>= begin scan←eight←bit←int; if box(cur←val)=null then cur←val:=0 @+else cur←val:=mem[box(cur←val)+m].sc; cur←val←level:=dimen←val; end @ @d max←dimen==@'7777777777 {$2↑{30}-1$} @<Fetch something on the |page←so←far|@>= begin if page←contents=empty then if m=0 then cur←val:=max←dimen@+else cur←val:=0 else cur←val:=page←so←far[m]; cur←val←level:=dimen←val; end @ @<Fetch the |prev←graf|@>= begin nest[nest←ptr]:=cur←list; p:=nest←ptr; while abs(nest[p].mode←field)<>vmode do decr(p); scanned←result(nest[p].pg←field)(int←val); end @ @<Fetch the |par←shape| size@>= begin if par←shape←ptr=null then cur←val:=0 else cur←val:=info(par←shape←ptr); cur←val←level:=int←val; end @ Here is where \.{\\lastpenalty}, \.{\\lastkern}, and \.{\\lastskip} are implemented. The reference count for \.{\\lastskip} will be updated later. @<Fetch an item in the current node...@>= begin if cur←chr=glue←val then cur←val:=zero←glue@+else cur←val:=0; cur←val←level:=cur←chr; if not is←char←node(tail)and(mode<>0) then case cur←chr of int←val: if type(tail)=penalty←node then cur←val:=penalty(tail); dimen←val: if type(tail)=kern←node then cur←val:=width(tail); glue←val: if type(tail)=glue←node then begin cur←val:=glue←ptr(tail); if subtype(tail)=mu←glue then cur←val←level:=mu←val; end; end {there are no other cases} else if (mode=vmode)and(tail=head) then case cur←chr of int←val: cur←val:=last←penalty; dimen←val: cur←val:=last←kern; glue←val: if last←glue<>max←halfword then cur←val:=last←glue; end; {there are no other cases} end @ @<Fetch a font dimension@>= begin find←font←dimen(false); font←info[fmem←ptr].sc:=0; scanned←result(font←info[cur←val].sc)(dimen←val); end @ @<Fetch a font integer@>= begin scan←font←ident; if m=0 then scanned←result(hyphen←char[cur←val])(int←val) else scanned←result(skew←char[cur←val])(int←val); end @ @<Fetch a register@>= begin scan←eight←bit←int; case m of int←val:cur←val:=count(cur←val); dimen←val:cur←val:=dimen(cur←val); glue←val: cur←val:=skip(cur←val); mu←val: cur←val:=mu←skip(cur←val); end; {there are no other cases} cur←val←level:=m; end @ @<Complain that \.{\\the} can't do this; give zero result@>= begin print←err("You can't use `"); print←cmd←chr(cur←cmd,cur←chr); @.You can't use x after ...@> print("' after "); print←esc("the"); help1("I'm forgetting what you said and using zero instead."); error; if level<>tok←val then scanned←result(0)(dimen←val) else scanned←result(0)(int←val); end @ When a |glue←val| changes to a |dimen←val|, we use the width component of the glue; there is no need to decrease the reference count, since it has not yet been increased. When a |dimen←val| changes to an |int←val|, we use scaled points so that the value doesn't actually change. And when a |mu←val| changes to a |glue←val|, the value doesn't change either. @<Convert \(c)|cur←val| to a lower level@>= begin if cur←val←level=glue←val then cur←val:=width(cur←val) else if cur←val←level=mu←val then mu←error; decr(cur←val←level); end @ If |cur←val| points to a glue specification at this point, the reference count for the glue does not yet include the reference by |cur←val|. If |negative| is |true|, |cur←val←level| is known to be |<=mu←val|. @<Fix the reference count, if any, ...@>= if negative then if cur←val←level>=glue←val then begin cur←val:=new←spec(cur←val); @<Negate all three glue components of |cur←val|@>; end else negate(cur←val) else if (cur←val←level>=glue←val)and(cur←val←level<=mu←val) then add←glue←ref(cur←val) @ @<Negate all three...@>= begin negate(width(cur←val)); negate(stretch(cur←val)); negate(shrink(cur←val)); end @ Our next goal is to write the |scan←int| procedure, which scans anything that \TeX\ treats as an integer. But first we might as well look at some simple applications of |scan←int| that have already been made inside of |scan←something←internal|: @<Declare procedures that scan restricted classes of integers@>= procedure scan←seven←bit←int; begin scan←int; if (cur←val<0)or(cur←val>127) then begin print←err("Bad character code"); @.Bad character code@> help2("The numeric code for a character must be between 0 and 127.")@/ ("I changed this one to zero."); int←error(cur←val); cur←val:=0; end; end; @ @<Declare procedures that scan restricted classes of integers@>= procedure scan←eight←bit←int; begin scan←int; if (cur←val<0)or(cur←val>255) then begin print←err("Bad register code"); @.Bad register code@> help2("A register number must be between 0 and 255.")@/ ("I changed this one to zero."); int←error(cur←val); cur←val:=0; end; end; @ @<Declare procedures that scan restricted classes of integers@>= procedure scan←four←bit←int; begin scan←int; if (cur←val<0)or(cur←val>15) then begin print←err("Bad number"); @.Bad number@> help2("Since I expected to read a number between 0 and 15,")@/ ("I changed this one to zero."); int←error(cur←val); cur←val:=0; end; end; @ While we're at it, we might as well deal with similar routines that will be needed later. @<Declare procedures that scan restricted classes of integers@>= procedure scan←char←num; begin scan←int; if (cur←val<0)or(cur←val>255) then begin print←err("Bad character code"); @.Bad character code@> help2("A character number must be between 0 and 255.")@/ ("I changed this one to zero."); int←error(cur←val); cur←val:=0; end; end; @ @<Declare procedures that scan restricted classes of integers@>= procedure scan←fifteen←bit←int; begin scan←int; if (cur←val<0)or(cur←val>@'77777) then begin print←err("Bad math code"); @.Bad math code@> help2("A numeric math code must be between 0 and 32767.")@/ ("I changed this one to zero."); int←error(cur←val); cur←val:=0; end; end; @ @<Declare procedures that scan restricted classes of integers@>= procedure scan←twenty←seven←bit←int; begin scan←int; if (cur←val<0)or(cur←val>@'777777777) then begin print←err("Bad delimiter code"); @.Bad delimiter code@> help2("A numeric delimiter code must be between 0 and 2↑{27}-1.")@/ ("I changed this one to zero."); int←error(cur←val); cur←val:=0; end; end; @ An integer number can be preceded by any number of spaces and `\.+' or `\.-' signs. Then comes either a decimal constant (i.e., radix 10), an octal constant (i.e., radix 8, preceded by~\.\'), a hexadecimal constant (radix 16, preceded by~\."), an alphabetic constant (preceded by~\.\`), or an internal variable. After scanning is complete, |cur←val| will contain the answer, which must be at most $2↑{31}-1=2147483647$ in absolute value. The value of |radix| is set to 10, 8, or 16 in the cases of decimal, octal, or hexadecimal constants, otherwise |radix| is set to zero. An optional space follows a constant. @d octal←token=other←token+"'" {apostrophe, indicates an octal constant} @d hex←token=other←token+"""" {double quote, indicates a hex constant} @d alpha←token=other←token+"`" {reverse apostrophe, precedes alpha constants} @d point←token=other←token+"." {decimal point} @d continental←point←token=other←token+"," {decimal point, Eurostyle} @<Glob...@>= @!radix:small←number; {|scan←int| sets this to 8, 10, 16, or zero} @ We initialize the following global variables just in case |expand| comes into action before any of the basic scanning routines has assigned them a value. @<Set init...@>= cur←val:=0; cur←val←level:=int←val; radix:=0; @ The |scan←int| routine is used also to scan the integer part of a fraction; for example, the `\.3' in `\.{3.14159}' will be found by |scan←int|. The |scan←dimen| routine assumes that |cur←tok=point←token| after the integer part of such a fraction has been scanned by |scan←int|, and that the decimal point has been backed up to be scanned again. @p procedure scan←int; {sets |cur←val| to an integer} label done; var negative:boolean; {should the answer be negated?} @!m:integer; {|@t$2↑{31}$@> div radix|, the threshold of danger} @!d:small←number; {the digit just scanned} @!vacuous:boolean; {have no digits appeared?} @!OK←so←far:boolean; {has an error message been issued?} begin radix:=0; OK←so←far:=true;@/ @<Get the next non-blank non-sign token; set |negative| appropriately@>; if cur←tok=alpha←token then @<Scan an alphabetic character code into |cur←val|@> else if (cur←cmd>=min←internal)and(cur←cmd<=max←internal) then scan←something←internal(int←val,false) else @<Scan a numeric constant@>; if negative then negate(cur←val); end; @ @<Get the next non-blank non-sign token...@>= negative:=false; repeat @<Get the next non-blank non-call token@>; if cur←tok=other←token+"-" then begin negative := not negative; cur←tok:=other←token+"+"; end; until cur←tok<>other←token+"+" @ A space is ignored after an alphabetic character constant, so that such constants behave like numeric ones. @<Scan an alphabetic character code into |cur←val|@>= begin get←token; {suppress macro expansion} if cur←tok<cs←token←flag then begin cur←val:=cur←chr; if cur←cmd<=right←brace then if cur←cmd=right←brace then incr(align←state) else decr(align←state); end else if cur←tok<cs←token←flag+single←base then cur←val:=cur←tok-cs←token←flag-active←base else cur←val:=cur←tok-cs←token←flag-single←base; if cur←val>127 then begin print←err("Improper alphabetic constant"); @.Improper alphabetic constant@> help2("A one-character control sequence belongs after a ` mark.")@/ ("So I'm essentially inserting \0 here."); cur←val:="0"; back←error; end else @<Scan an optional space@>; end @ @<Scan an optional space@>= begin get←x←token; if cur←cmd<>spacer then back←input; end @ @<Scan a numeric constant@>= begin radix:=10; m:=214748364; if cur←tok=octal←token then begin radix:=8; m:=@'2000000000; get←x←token; end else if cur←tok=hex←token then begin radix:=16; m:=@'1000000000; get←x←token; end; vacuous:=true; cur←val:=0;@/ @<Accumulate the constant until |cur←tok| is not a suitable digit@>; if vacuous then @<Express astonishment that no number was here@> else if cur←cmd<>spacer then back←input; end @ @d infinity==@'17777777777 {the largest positive value that \TeX\ knows} @d zero←token=other←token+"0" {zero, the smallest digit} @d A←token=letter←token+"A" {the smallest special hex digit} @d other←A←token=other←token+"A" {special hex digit of type |other←char|} @<Accumulate the constant...@>= loop@+ begin if (cur←tok<zero←token+radix)and(cur←tok>=zero←token)and (cur←tok<=zero←token+9) then d:=cur←tok-zero←token else if radix=16 then if (cur←tok<=A←token+5)and(cur←tok>=A←token) then d:=cur←tok-A←token+10 else if (cur←tok<=other←A←token+5)and(cur←tok>=other←A←token) then d:=cur←tok-other←A←token+10 else goto done else goto done; vacuous:=false; if (cur←val>=m)and((cur←val>m)or(d>7)or(radix<>10)) then begin if OK←so←far then begin print←err("Number too big"); @.Number too big@> help2("I can only go up to 2147483647='17777777777=""7FFFFFFF,")@/ ("so I'm using that number instead of yours."); error; cur←val:=infinity; OK←so←far:=false; end; end else cur←val:=cur←val*radix+d; get←x←token; end; done: @ @<Express astonishment...@>= begin print←err("Missing number, treated as zero"); @.Missing number...@> help3("A number should have been here; I inserted `0'.")@/ ("(If you can't figure out why I needed to see a number,")@/ ("look up `weird error' in the index to The TeXbook.)"); @:TeXbook}{\sl The \TeX book@> back←error; end @ The |scan←dimen| routine is similar to |scan←int|, but it sets |cur←val| to a |scaled| value, i.e., an integral number of sp. One of its main tasks is therefore to interpret the abbreviations for various kinds of units and to convert measurements to scaled points. There are three parameters: |mu| is |true| if the finite units must be `\.{mu}', while |mu| is |false| if `\.{mu}' units are disallowed; |inf| is |true| if the infinite units `\.{fil}', `\.{fill}', `\.{filll}' are permitted; and |shortcut| is |true| if |cur←val| already contains an integer and only the units need to be considered. The order of infinity that was found in the case of infinite glue is returned in the global variable |cur←order|. @<Glob...@>= @!cur←order:glue←ord; {order of infinity found by |scan←dimen|} @ Constructions like `\.{-\'77 pt}' are legal dimensions, so |scan←dimen| may begin with |scan←int|. This explains why it is convenient to use |scan←int| also for the integer part of a decimal fraction. Several branches of |scan←dimen| work with |cur←val| as an integer and with an auxiliary fraction |f|, so that the actual quantity of interest is $|cur←val|+|f|/2↑{16}$. At the end of the routine, this ``unpacked'' representation is put into the single word |cur←val|, which suddenly switches significance from |integer| to |scaled|. @d attach←fraction=88 {go here to pack |cur←val| and |f| into |cur←val|} @d attach←sign=89 {go here when |cur←val| is correct except perhaps for sign} @d scan←normal←dimen==scan←dimen(false,false,false) @p procedure scan←dimen(@!mu,@!inf,@!shortcut:boolean); {sets |cur←val| to a dimension} label done, done1, done2, found, not←found, attach←fraction, attach←sign; var negative:boolean; {should the answer be negated?} @!f:integer; {numerator of a fraction whose denominator is $2↑{16}$} @<Local variables for dimension calculations@>@; begin f:=0; arith←error:=false; cur←order:=normal; negative:=false; if not shortcut then begin @<Get the next non-blank non-sign...@>; if (cur←cmd>=min←internal)and(cur←cmd<=max←internal) then @<Fetch an internal dimension and |goto attach←sign|, or fetch an internal integer@> else begin back←input; if cur←tok=continental←point←token then cur←tok:=point←token; if cur←tok<>point←token then scan←int else begin radix:=10; cur←val:=0; end; if cur←tok=continental←point←token then cur←tok:=point←token; if (radix=10)and(cur←tok=point←token) then @<Scan decimal fraction@>; end; end; if cur←val<0 then {in this case |f=0|} begin negative := not negative; negate(cur←val); end; @<Scan units and set |cur←val| to $x\cdot(|cur←val|+f/2↑{16})$, where there are |x| units per sp; |goto attach←sign| if the units are internal@>; @<Scan an optional space@>; attach←sign: if arith←error or(abs(cur←val)>=@'10000000000) then @<Report that this dimension is out of range@>; if negative then negate(cur←val); end; @ @<Fetch an internal dimension and |goto attach←sign|...@>= if mu then begin scan←something←internal(mu←val,false); @<Coerce glue to a dimension@>; if cur←val←level=mu←val then goto attach←sign; if cur←val←level<>int←val then mu←error; end else begin scan←something←internal(dimen←val,false); if cur←val←level=dimen←val then goto attach←sign; end @ @<Local variables for dimension calculations@>= @!num,@!denom:1..65536; {conversion ratio for the scanned units} @!k:small←number; {number of digits in a decimal fraction} @!v:scaled; {an internal dimension} @!save←cur←val:integer; {temporary storage of |cur←val|} @ The following code is executed when |scan←something←internal| was called asking for |mu←val|, when we really wanted a ``mudimen'' instead of ``muglue.'' @<Coerce glue to a dimension@>= if cur←val←level>=glue←val then begin v:=width(cur←val); delete←glue←ref(cur←val); cur←val:=v; end @ When the following code is executed, we have |cur←tok=point←token|, but this token has been backed up using |back←input|; we must first discard it. It turns out that a decimal point all by itself is equivalent to `\.{0.0}'. Let's hope people don't use that fact. @<Scan decimal fraction@>= begin k:=0; get←token; {|point←token| is being re-scanned} loop@+ begin get←x←token; if (cur←tok>zero←token+9)or(cur←tok<zero←token) then goto done1; if k<17 then {digits for |k>=17| cannot affect the result} begin dig[k]:=cur←tok-zero←token; incr(k); end; end; done1: f:=round←decimals(k); if cur←cmd<>spacer then back←input; end @ Now comes the harder part: At this point in the program, |cur←val| is a nonnegative integer and $f/2↑{16}$ is a nonnegative fraction less than 1; we want to multiply the sum of these two quantities by the appropriate factor, based on the specified units, in order to produce a |scaled| result, and we want to do the calculation with fixed point arithmetic that does not overflow. @<Scan units and set |cur←val| to $x\cdot(|cur←val|+f/2↑{16})$...@>= if inf then @<Scan for \(f)\.{fil} units; |goto attach←fraction| if found@>; @<Scan for \(u)units that are internal dimensions; |goto attach←sign| with |cur←val| set if found@>; if mu then @<Scan for \(m)\.{mu} units and |goto attach←fraction|@>; if scan←keyword("true") then @<Adjust \(f)for the magnification ratio@>; @.true@> if scan←keyword("pt") then goto attach←fraction; {the easy case} @.pt@> @<Scan for \(a)all other units and adjust |cur←val| and |f| accordingly; |goto done| in the case of scaled points@>; attach←fraction: if cur←val>=@'40000 then arith←error:=true else cur←val:=cur←val*unity+f; done: @ @<Scan for \(f)\.{fil} units...@>= if scan←keyword("fil") then @.fil@> begin cur←order:=fil; while scan←keyword("l") do begin if cur←order=filll then begin print←err("Illegal unit of measure ("); @.Illegal unit of measure@> print("replaced by filll)"); help1("I dddon't go any higher than filll."); error; end else incr(cur←order); end; goto attach←fraction; end @ @<Scan for \(u)units that are internal dimensions...@>= save←cur←val:=cur←val; @<Get the next non-blank non-call...@>; if (cur←cmd<min←internal)or(cur←cmd>max←internal) then back←input else begin if mu then begin scan←something←internal(mu←val,false); @<Coerce glue...@>; if cur←val←level<>mu←val then mu←error; end else scan←something←internal(dimen←val,false); v:=cur←val; goto found; end; if mu then goto not←found; if scan←keyword("em") then v:=(@<The em width for |cur←font|@>) @.em@> else if scan←keyword("ex") then v:=(@<The x-height for |cur←font|@>) @.ex@> else goto not←found; @<Scan an optional space@>; found:cur←val:=nx←plus←y(save←cur←val,v,xn←over←d(v,f,@'200000)); goto attach←sign; not←found: @ @<Scan for \(m)\.{mu} units and |goto attach←fraction|@>= if scan←keyword("mu") then goto attach←fraction @.mu@> else begin print←err("Illegal unit of measure ("); print("mu inserted)"); @.Illegal unit of measure@> help4("The unit of measurement in math glue must be mu.")@/ ("To recover gracefully from this error, it's best to")@/ ("delete the erroneous units; e.g., type `2' to delete")@/ ("two letters. (See Chapter 27 of The TeXbook.)"); @:TeXbook}{\sl The \TeX book@> error; goto attach←fraction; end @ @<Adjust \(f)for the magnification ratio@>= begin prepare←mag; if mag<>1000 then begin cur←val:=xn←over←d(cur←val,1000,mag); f:=(1000*f+@'200000*remainder) div mag; cur←val:=cur←val+(f div @'200000); f:=f mod @'200000; end; end @ All of the necessary conversion factors can be specified exactly as fractions whose numerator and denominator are 32768 or less. According to the definitions here, $\rm2660\,dd\approx1000.33297\,mm$; this agrees well with the value $\rm1000.333\,mm$ cited by Bosshard @↑Bosshard, Hans Rudolf@> in {\sl Technische Grundlagen zur Satzherstellung\/} (Bern, 1980). @d set←conversion←end(#)== denom:=#; end @d set←conversion(#)==@+begin num:=#; set←conversion←end @<Scan for \(a)all other units and adjust |cur←val| and |f|...@>= if scan←keyword("in") then set←conversion(7227)(100) @.in@> else if scan←keyword("pc") then set←conversion(12)(1) @.pc@> else if scan←keyword("cm") then set←conversion(7227)(254) @.cm@> else if scan←keyword("mm") then set←conversion(7227)(2540) @.mm@> else if scan←keyword("bp") then set←conversion(7227)(7200) @.bp@> else if scan←keyword("dd") then set←conversion(1238)(1157) @.dd@> else if scan←keyword("cc") then set←conversion(14856)(1157) @.cc@> else if scan←keyword("sp") then goto done @.sp@> else @<Complain about unknown unit and |goto done2|@>; cur←val:=xn←over←d(cur←val,num,denom); f:=(num*f+@'200000*remainder) div denom;@/ cur←val:=cur←val+(f div @'200000); f:=f mod @'200000; done2: @ @<Complain about unknown unit...@>= begin print←err("Illegal unit of measure ("); print("pt inserted)"); @.Illegal unit of measure@> help6("Dimensions can be in units of em, ex, in, pt, pc,")@/ ("cm, mm, dd, cc, bp, or sp; but yours is a new one!")@/ ("I'll assume that you meant to say pt, for printers' points.")@/ ("To recover gracefully from this error, it's best to")@/ ("delete the erroneous units; e.g., type `2' to delete")@/ ("two letters. (See Chapter 27 of The TeXbook.)"); @:TeXbook}{\sl The \TeX book@> error; goto done2; end @ @<Report that this dimension is out of range@>= begin print←err("Dimension too large"); @.Dimension too large@> help2("I can't work with sizes bigger than about 19 feet.")@/ ("Continue and I'll use the largest value I can.");@/ error; cur←val:=max←dimen; arith←error:=false; end @ The final member of \TeX's value-scanning trio is |scan←glue|, which makes |cur←val| point to a glue specification. The reference count of that glue spec will take account of the fact that |cur←val| is pointing to~it. The |level| parameter should be either |glue←val| or |mu←val|. Since |scan←dimen| was so much more complex than |scan←int|, we might expect |scan←glue| to be even worse. But fortunately, it is very simple, since most of the work has already been done for us. @p procedure scan←glue(@!level:small←number); {sets |cur←val| to a glue spec pointer} label exit; var negative:boolean; {should the answer be negated?} @!q:pointer; {new glue specification} @!mu:boolean; {does |level=mu←val|?} begin mu:=(level=mu←val); @<Get the next non-blank non-sign...@>; if (cur←cmd>=min←internal)and(cur←cmd<=max←internal) then begin scan←something←internal(level,negative); if cur←val←level>=glue←val then begin if cur←val←level<>level then mu←error; return; end; if cur←val←level=int←val then scan←dimen(mu,false,true) else if level=mu←val then mu←error; end else begin back←input; scan←dimen(mu,false,false); if negative then negate(cur←val); end; @<Create a new glue specification whose width is |cur←val|; scan for its stretch and shrink components@>; exit:end; @ @<Create a new glue specification whose width is |cur←val|...@>= q:=new←spec(zero←glue); width(q):=cur←val; if scan←keyword("plus") then @.plus@> begin scan←dimen(mu,true,false); stretch(q):=cur←val; stretch←order(q):=cur←order; end; if scan←keyword("minus") then @.minus@> begin scan←dimen(mu,true,false); shrink(q):=cur←val; shrink←order(q):=cur←order; end; cur←val:=q @ Here's a similar procedure that returns a pointer to a rule node. This routine is called just after \TeX\ has seen \.{\\hrule} or \.{\\vrule}; therefore |cur←cmd| will be either |hrule| or |vrule|. The idea is to store the default rule dimensions in the node, then to override them if `\.{height}' or `\.{width}' or `\.{depth}' specifications are found (in any order). @d default←rule=26214 {0.4\thinspace pt} @p function scan←rule←spec:pointer; label reswitch; var q:pointer; {the rule node being created} begin q:=new←rule; {|width|, |depth|, and |height| all equal |null←flag| now} if cur←cmd=vrule then width(q):=default←rule else begin height(q):=default←rule; depth(q):=0; end; reswitch: if scan←keyword("width") then @.width@> begin scan←normal←dimen; width(q):=cur←val; goto reswitch; end; if scan←keyword("height") then @.height@> begin scan←normal←dimen; height(q):=cur←val; goto reswitch; end; if scan←keyword("depth") then @.depth@> begin scan←normal←dimen; depth(q):=cur←val; goto reswitch; end; scan←rule←spec:=q; end; @* \[27] Building token lists. The token lists for macros and for other things like \.{\\mark} and \.{\\output} and \.{\\write} are produced by a procedure called |scan←toks|. Before we get into the details of |scan←toks|, let's consider a much simpler task, that of converting the current string into a token list. The |str←toks| function does this; it classifies spaces as type |spacer| and everything else as type |other←char|. The token list created by |str←toks| begins at |link(temp←head)| and ends at the value |p| that is returned. (If |p=temp←head|, the list is empty.) @p function str←toks:pointer; {changes the current string to a token list} var p:pointer; {tail of the token list} @!q:pointer; {new node being added to the token list via |store←new←token|} @!t:halfword; {token being appended} @!k:pool←pointer; {index into |str←pool|} begin str←room(1); p:=temp←head; link(p):=null; k:=str←start[str←ptr]; while k<pool←ptr do begin t:=str←pool[k]; if t=" " then t:=space←token else t:=other←token+t; fast←store←new←token(t); incr(k); end; pool←ptr:=str←start[str←ptr]; str←toks:=p; end; @ The main reason for wanting |str←toks| is the next function, |the←toks|, which has similar input/output characteristics. This procedure is supposed to scan something like `\.{\\skip\\count12}', i.e., whatever can follow `\.{\\the}', and it constructs a token list containing something like `\.{-3.0pt minus 0.5fill}'. @p function the←toks:pointer; var old←setting:0..max←selector; {holds |selector| setting} @!p,@!q,@!r:pointer; {used for copying a token list} begin get←x←token; scan←something←internal(tok←val,false); if cur←val←level>=ident←val then @<Copy the token list@> else begin old←setting:=selector; selector:=new←string; case cur←val←level of int←val:print←int(cur←val); dimen←val:begin print←scaled(cur←val); print("pt"); end; glue←val: begin print←spec(cur←val,"pt"); delete←glue←ref(cur←val); end; mu←val: begin print←spec(cur←val,"mu"); delete←glue←ref(cur←val); end; end; {there are no other cases} selector:=old←setting; the←toks:=str←toks; end; end; @ @<Copy the token list@>= begin p:=temp←head; link(p):=null; if cur←val←level=ident←val then store←new←token(cs←token←flag+cur←val) else if cur←val<>null then begin r:=link(cur←val); {do not copy the reference count} while r<>null do begin fast←store←new←token(info(r)); r:=link(r); end; end; the←toks:=p; end @ Here's part of the |expand| subroutine that we are now ready to complete: @p procedure ins←the←toks; begin link(garbage):=the←toks; ins←list(link(temp←head)); end; @ The primitives \.{\\number}, \.{\\romannumeral}, \.{\\string}, \.{\\meaning}, \.{\\fontname}, and \.{\\jobname} are defined as follows. @d number←code=0 {command code for \.{\\number}} @d roman←numeral←code=1 {command code for \.{\\romannumeral}} @d string←code=2 {command code for \.{\\string}} @d meaning←code=3 {command code for \.{\\meaning}} @d font←name←code=4 {command code for \.{\\fontname}} @d job←name←code=5 {command code for \.{\\jobname}} @<Put each...@>= primitive("number",convert,number←code);@/ @!@:number←}{\.{\\number} primitive@> primitive("romannumeral",convert,roman←numeral←code);@/ @!@:roman←numeral←}{\.{\\romannumeral} primitive@> primitive("string",convert,string←code);@/ @!@:string←}{\.{\\string} primitive@> primitive("meaning",convert,meaning←code);@/ @!@:meaning←}{\.{\\meaning} primitive@> primitive("fontname",convert,font←name←code);@/ @!@:font←name←}{\.{\\fontname} primitive@> primitive("jobname",convert,job←name←code);@/ @!@:job←name←}{\.{\\jobname} primitive@> @ @<Cases of |print←cmd←chr|...@>= convert: case chr←code of number←code: print←esc("number"); roman←numeral←code: print←esc("romannumeral"); string←code: print←esc("string"); meaning←code: print←esc("meaning"); font←name←code: print←esc("fontname"); othercases print←esc("jobname") endcases; @ The procedure |conv←toks| uses |str←toks| to insert the token list for |convert| functions into the scanner; `\.{\\outer}' control sequences are allowed to follow `\.{\\string}' and `\.{\\meaning}'. @p procedure conv←toks; var old←setting:0..max←selector; {holds |selector| setting} @!c:number←code..job←name←code; {desired type of conversion} @!save←scanner←status:small←number; {|scanner←status| upon entry} begin c:=cur←chr; @<Scan the argument for command |c|@>; old←setting:=selector; selector:=new←string; @<Print the result of command |c|@>; selector:=old←setting; link(garbage):=str←toks; ins←list(link(temp←head)); end; @ @<Scan the argument for command |c|@>= case c of number←code,roman←numeral←code: scan←int; string←code, meaning←code: begin save←scanner←status:=scanner←status; scanner←status:=normal; get←token; scanner←status:=save←scanner←status; end; font←name←code: scan←font←ident; job←name←code: if job←name=0 then open←log←file; end {there are no other cases} @ @<Print the result of command |c|@>= case c of number←code: print←int(cur←val); roman←numeral←code: print←roman←int(cur←val); string←code:if cur←cs<>0 then sprint←cs(cur←cs) else print←char(cur←chr); meaning←code: print←meaning; font←name←code: begin print(font←name[cur←val]); if font←size[cur←val]<>font←dsize[cur←val] then begin print(" at "); print←scaled(font←size[cur←val]); print("pt"); end; end; job←name←code: print(job←name); end {there are no other cases} @ Now we can't postpone the difficulties any longer; we must bravely tackle |scan←toks|. This function returns a pointer to the tail of a new token list, and it also makes |def←ref| point to the reference count at the head of that list. There are two boolean parameters, |macro←def| and |xpand|. If |macro←def| is true, the goal is to create the token list for a macro definition; otherwise the goal is to create the token list for some other \TeX\ primitive: \.{\\mark}, \.{\\output}, \.{\\everypar}, \.{\\lowercase}, \.{\\uppercase}, \.{\\message}, \.{\\errmessage}, \.{\\write}, or \.{\\special}. In the latter cases a left brace must be scanned next; this left brace will not be part of the token list, nor will the matching right brace that comes at the end. If |xpand| is false, the token list will simply be copied from the input using |get←token|. Otherwise all expandable tokens will be expanded until unexpandable tokens are left, except that the results of expanding `\.{\\the}' are not expanded further. If both |macro←def| and |xpand| are true, the expansion applies only to the macro body (i.e., to the material following the first |left←brace| character). The value of |cur←cs| when |scan←toks| begins should be the |eqtb| address of the control sequence to display in ``runaway'' error messages. @p function scan←toks(@!macro←def,@!xpand:boolean):pointer; label found,done,done1,done2; var t:halfword; {token representing the highest parameter number} @!s:halfword; {saved token} @!p:pointer; {tail of the token list being built} @!q:pointer; {new node being added to the token list via |store←new←token|} @!unbalance:halfword; {number of unmatched left braces} @!hash←brace:halfword; {possible `\.{\#\{}' token} begin if macro←def then scanner←status:=defining @+else scanner←status:=absorbing; warning←index:=cur←cs; def←ref:=get←avail; token←ref←count(def←ref):=null; p:=def←ref; hash←brace:=0; t:=zero←token; if macro←def then @<Scan and build the parameter part of the macro definition@> else scan←left←brace; {remove the compulsory left brace} @<Scan and build the body of the token list; |goto found| when finished@>; found: scanner←status:=normal; if hash←brace<>0 then store←new←token(hash←brace); scan←toks:=p; end; @ @<Scan and build the parameter part...@>= begin loop begin get←token; {set |cur←cmd|, |cur←chr|, |cur←tok|} if cur←tok<right←brace←limit then goto done1; if cur←cmd=mac←param then @<If the next character is a parameter number, make |cur←tok| a |match| token; but if it is a left brace, store `|left←brace|, |end←match|', set |hash←brace|, and |goto done|@>; store←new←token(cur←tok); end; done1: store←new←token(end←match←token); if cur←cmd=right←brace then @<Express shock at the missing left brace; |goto found|@>; done: end @ @<Express shock...@>= begin print←err("Missing { inserted"); incr(align←state); @.Missing \{ inserted@> help2("Where was the left brace? You said something like `\def\a}',")@/ ("which I'm going to interpret as `\def\a{}'."); error; goto found; end @ @<If the next character is a parameter number...@>= begin s:=match←token+cur←chr; get←token; if cur←cmd=left←brace then begin hash←brace:=cur←tok; store←new←token(cur←tok); store←new←token(end←match←token); goto done; end; if t=zero←token+9 then begin print←err("You already have nine parameters"); @.You already have nine...@> help1("I'm going to ignore the # sign you just used."); error; end else begin incr(t); if cur←tok<>t then begin print←err("Parameters must be numbered consecutively"); @.Parameters...consecutively@> help2("I've inserted the digit you should have used after the #.")@/ ("Type `1' to delete what you did use."); back←error; end; cur←tok:=s; end; end @ @<Scan and build the body of the token list; |goto found| when finished@>= unbalance:=1; loop@+ begin if xpand then @<Expand the next part of the input@> else get←token; if cur←tok<right←brace←limit then if cur←cmd<right←brace then incr(unbalance) else begin decr(unbalance); if unbalance=0 then goto found; end else if cur←cmd=mac←param then if macro←def then @<Look for parameter number or \.{\#\#}@>; store←new←token(cur←tok); end @ Here we insert an entire token list created by |the←toks| without expanding it further. @<Expand the next part of the input@>= begin loop begin get←next; if cur←cmd<>the then goto done2; q:=the←toks; if link(temp←head)<>null then begin link(p):=link(temp←head); p:=q; end; end; done2: x←token end @ @<Look for parameter number...@>= begin s:=cur←tok; if xpand then get←x←token else get←token; if cur←cmd<>mac←param then if (cur←tok<=zero←token)or(cur←tok>t) then begin print←err("Illegal parameter number in definition of "); @.Illegal parameter number...@> sprint←cs(warning←index); help3("You meant to type ## instead of #, right?")@/ ("Or maybe a } was forgotten somewhere earlier, and things")@/ ("are all screwed up? I'm going to assume that you meant ##."); back←error; cur←tok:=s; end else cur←tok:=out←param←token-"0"+cur←chr; end @ Another way to create a token list is via the \.{\\read} command. The sixteen files potentially usable for reading appear in the following global variables. The value of |read←open[n]| will be |closed| if stream number |n| has not been opened or if it has been fully read; |just←open| if an \.{\\openin} but not a \.{\\read} has been done; and |normal| if it is open and ready to ready the next line. @d closed=2 {not open, or at end of file} @d just←open=1 {newly opened, first line not yet read} @<Glob...@>= @!read←file:array[0..15] of alpha←file; {used for \.{\\read}} @!read←open:array[0..16] of normal..closed; {state of |read←file[n]|} @ @<Set init...@>= for k:=0 to 16 do read←open[k]:=closed; @ The |read←toks| procedure constructs a token list like that for any macro definition, and makes |cur←val| point to it. Parameter |r| points to the control sequence that will receive this token list. @p procedure read←toks(@!n:integer;@!r:pointer); label done; var p:pointer; {tail of the token list} @!q:pointer; {new node being added to the token list via |store←new←token|} @!s:integer; {saved value of |align←state|} @!m:small←number; {stream number} begin scanner←status:=defining; warning←index:=r; def←ref:=get←avail; token←ref←count(def←ref):=null; p:=def←ref; {the reference count} store←new←token(end←match←token); if (n<0)or(n>15) then m:=16@+else m:=n; s:=align←state; align←state:=1000000; {disable tab marks, etc.} repeat @<Input and store tokens from the next line of the file@>; until align←state=1000000; cur←val:=def←ref; scanner←status:=normal; align←state:=s; end; @ @<Input and store tokens from the next line of the file@>= begin←file←reading; name:=m+1; if read←open[m]=closed then @<Input for \.{\\read} from the terminal@> else if read←open[m]=just←open then @<Input the first line of |read←file[m]|@> else @<Input the next line of |read←file[m]|@>; limit:=last; if (end←line←char<0)or(end←line←char>127) then decr(limit) else buffer[limit]:=end←line←char; first:=limit+1; loc:=start; state:=new←line;@/ loop@+ begin get←token; if cur←tok=0 then goto done; {|cur←cmd=cur←chr=0| will occur at the end of the line} store←new←token(cur←tok); end; done: end←file←reading @ Here we input on-line into the |buffer| array, prompting the user explicitly if |n>=0|. The value of |n| is set negative so that additional prompts will not be given in the case of multi-line input. @<Input for \.{\\read} from the terminal@>= if interaction>nonstop←mode then if n<0 then prompt←input("") else begin wake←up←terminal; print←ln; sprint←cs(r); prompt←input("="); n:=-1; end else fatal←error("*** (cannot \read from terminal in nonstop modes)") @.cannot \\read@> @ The first line of a file must be treated specially, since |input←ln| must be told not to start with |get|. @↑system dependencies@> @<Input the first line of |read←file[m]|@>= if input←ln(read←file[m],false) then read←open[m]:=normal else begin a←close(read←file[m]); read←open[m]:=closed; end @ An empty line is appended at the end of a |read←file|. @↑empty line at end of file@> @<Input the next line of |read←file[m]|@>= begin if not input←ln(read←file[m],true) then begin a←close(read←file[m]); read←open[m]:=closed; if align←state<>1000000 then begin runaway; print←err("File ended within "); print←esc("read"); @.File ended within \\read@> help1("This \read has unbalanced braces."); align←state:=1000000; error; end; end; end @* \[28] Conditional processing. We consider now the way \TeX\ handles various kinds of \.{\\if} commands. @d if←char←code=0 { `\.{\\if}' } @d if←cat←code=1 { `\.{\\ifcat}' } @d if←int←code=2 { `\.{\\ifnum}' } @d if←dim←code=3 { `\.{\\ifdim}' } @d if←odd←code=4 { `\.{\\ifodd}' } @d if←vmode←code=5 { `\.{\\ifvmode}' } @d if←hmode←code=6 { `\.{\\ifhmode}' } @d if←mmode←code=7 { `\.{\\ifmmode}' } @d if←inner←code=8 { `\.{\\ifinner}' } @d if←void←code=9 { `\.{\\ifvoid}' } @d if←hbox←code=10 { `\.{\\ifhbox}' } @d if←vbox←code=11 { `\.{\\ifvbox}' } @d ifx←code=12 { `\.{\\ifx}' } @d if←eof←code=13 { `\.{\\ifeof}' } @d if←true←code=14 { `\.{\\iftrue}' } @d if←false←code=15 { `\.{\\iffalse}' } @d if←case←code=16 { `\.{\\ifcase}' } @<Put each...@>= primitive("if",if←test,if←char←code); @!@:if←char←}{\.{\\if} primitive@> primitive("ifcat",if←test,if←cat←code); @!@:if←cat←code←}{\.{\\ifcat} primitive@> primitive("ifnum",if←test,if←int←code); @!@:if←int←}{\.{\\ifnum} primitive@> primitive("ifdim",if←test,if←dim←code); @!@:if←dim←}{\.{\\ifdim} primitive@> primitive("ifodd",if←test,if←odd←code); @!@:if←odd←}{\.{\\ifodd} primitive@> primitive("ifvmode",if←test,if←vmode←code); @!@:if←vmode←}{\.{\\ifvmode} primitive@> primitive("ifhmode",if←test,if←hmode←code); @!@:if←hmode←}{\.{\\ifhmode} primitive@> primitive("ifmmode",if←test,if←mmode←code); @!@:if←mmode←}{\.{\\ifmmode} primitive@> primitive("ifinner",if←test,if←inner←code); @!@:if←inner←}{\.{\\ifinner} primitive@> primitive("ifvoid",if←test,if←void←code); @!@:if←void←}{\.{\\ifvoid} primitive@> primitive("ifhbox",if←test,if←hbox←code); @!@:if←hbox←}{\.{\\ifhbox} primitive@> primitive("ifvbox",if←test,if←vbox←code); @!@:if←vbox←}{\.{\\ifvbox} primitive@> primitive("ifx",if←test,ifx←code); @!@:ifx←}{\.{\\ifx} primitive@> primitive("ifeof",if←test,if←eof←code); @!@:if←eof←}{\.{\\ifeof} primitive@> primitive("iftrue",if←test,if←true←code); @!@:if←true←}{\.{\\iftrue} primitive@> primitive("iffalse",if←test,if←false←code); @!@:if←false←}{\.{\\iffalse} primitive@> primitive("ifcase",if←test,if←case←code); @!@:if←case←}{\.{\\ifcase} primitive@> @ @<Cases of |print←cmd←chr|...@>= if←test: case chr←code of if←cat←code:print←esc("ifcat"); if←int←code:print←esc("ifnum"); if←dim←code:print←esc("ifdim"); if←odd←code:print←esc("ifodd"); if←vmode←code:print←esc("ifvmode"); if←hmode←code:print←esc("ifhmode"); if←mmode←code:print←esc("ifmmode"); if←inner←code:print←esc("ifinner"); if←void←code:print←esc("ifvoid"); if←hbox←code:print←esc("ifhbox"); if←vbox←code:print←esc("ifvbox"); ifx←code:print←esc("ifx"); if←eof←code:print←esc("ifeof"); if←true←code:print←esc("iftrue"); if←false←code:print←esc("iffalse"); if←case←code:print←esc("ifcase"); othercases print←esc("if") endcases; @ Conditions can be inside conditions, and this nesting has a stack that is independent of the |save←stack|. Four global variables represent the top of the condition stack: |cond←ptr| points to pushed-down entries, if any; |if←limit| specifies the largest code of a |fi←or←else| command that is syntactically legal; |cur←if| is the name of the current type of conditional; and |if←line| is the line number at which it began. If no conditions are currently in progress, the condition stack has the special state |cond←ptr=null|, |if←limit=normal|, |cur←if=0|, |if←line=0|. Otherwise |cond←ptr| points to a two-word node; the |type|, |subtype|, and |link| fields of the first word contain |if←limit|, |cur←if|, and |cond←ptr| at the next level, and the second word contains the corresponding |if←line|. @d if←node←size=2 {number of words in stack entry for conditionals} @d if←line←field(#)==mem[#+1].int @d if←code=1 {code for \.{\\if...} being evaluated} @d fi←code=2 {code for \.{\\fi}} @d else←code=3 {code for \.{\\else}} @d or←code=4 {code for \.{\\or}} @<Glob...@>= @!cond←ptr:pointer; {top of the condition stack} @!if←limit:normal..or←code; {upper bound on |fi←or←else| codes} @!cur←if:small←number; {type of conditional being worked on} @!if←line:integer; {line where that conditional began} @ @<Set init...@>= cond←ptr:=null; if←limit:=normal; cur←if:=0; if←line:=0; @ @<Put each...@>= primitive("fi",fi←or←else,fi←code); @!@:fi←}{\.{\\fi} primitive@> text(frozen←fi):="fi"; eqtb[frozen←fi]:=eqtb[cur←val]; primitive("or",fi←or←else,or←code); @!@:or←}{\.{\\or} primitive@> primitive("else",fi←or←else,else←code); @!@:else←}{\.{\\else} primitive@> @ @<Cases of |print←cmd←chr|...@>= fi←or←else: if chr←code=fi←code then print←esc("fi") else if chr←code=or←code then print←esc("or") else print←esc("else"); @ When we skip conditional text, we keep track of the line number where skipping began, for use in error messages. @<Glob...@>= @!skip←line:integer; {skipping began here} @ Here is a procedure that ignores text until coming to an \.{\\or}, \.{\\else}, or \.{\\fi} at level zero of $\.{\\if}\ldots\.{\\fi}$ nesting. After it has acted, |cur←chr| will indicate the token that was found, but |cur←tok| will not be set (because this makes the procedure run faster). @p procedure pass←text; label done; var l:integer; {level of $\.{\\if}\ldots\.{\\fi}$ nesting} @!save←scanner←status:small←number; {|scanner←status| upon entry} begin save←scanner←status:=scanner←status; scanner←status:=skipping; l:=0; skip←line:=line; loop@+ begin get←next; if cur←cmd=fi←or←else then begin if l=0 then goto done; if cur←chr=fi←code then decr(l); end else if cur←cmd=if←test then incr(l); end; done: scanner←status:=save←scanner←status; end; @ When we begin to process a new \.{\\if}, we set |if←limit:=if←code|; then if \.{\\or} or \.{\\else} or \.{\\fi} occurs before the current \.{\\if} condition has been evaluated, \.{\\relax} will be inserted. For example, a sequence of commands like `\.{\\ifvoid1\\else...\\fi}' would otherwise require something after the `\.1'. @<Push the condition stack@>= begin p:=get←node(if←node←size); link(p):=cond←ptr; type(p):=if←limit; subtype(p):=cur←if; if←line←field(p):=if←line; cond←ptr:=p; cur←if:=cur←chr; if←limit:=if←code; if←line:=line; end @ @<Pop the condition stack@>= begin p:=cond←ptr; if←line:=if←line←field(p); cur←if:=subtype(p); if←limit:=type(p); cond←ptr:=link(p); free←node(p,if←node←size); end @ Here's a procedure that changes the |if←limit| code corresponding to a given value of |cond←ptr|. @p procedure change←if←limit(@!l:small←number;@!p:pointer); label exit; var q:pointer; begin if p=cond←ptr then if←limit:=l {that's the easy case} else begin q:=cond←ptr; loop@+ begin if q=null then confusion("if"); @:this can't happen if}{\quad if@> if link(q)=p then begin type(q):=l; return; end; q:=link(q); end; end; exit:end; @ A condition is started when the |expand| procedure encounters an |if←test| command; in that case |expand| reduces to |conditional|, which is a recursive procedure. @p procedure conditional; label exit,common←ending; var b:boolean; {is the condition true?} @!r:"<"..">"; {relation to be evaluated} @!m,@!n:integer; {to be tested against the second operand} @!p,@!q:pointer; {for traversing token lists in \.{\\ifx} tests} @!save←scanner←status:small←number; {|scanner←status| upon entry} @!save←cond←ptr:pointer; {|cond←ptr| corresponding to this conditional} @!this←if:small←number; {type of this conditional} begin @<Push the condition stack@>;@+save←cond←ptr:=cond←ptr;this←if:=cur←chr;@/ @<Either process \.{\\ifcase} or set |b| to the value of a boolean condition@>; if tracing←commands>1 then @<Display the value of |b|@>; if b then begin change←if←limit(else←code,save←cond←ptr); return; {wait for \.{\\else} or \.{\\fi}} end; @<Skip to \.{\\else} or \.{\\fi}, then |goto common←ending|@>; common←ending: if cur←chr=fi←code then @<Pop the condition stack@> else if←limit:=fi←code; {wait for \.{\\fi}} exit:end; @ In a construction like `\.{\\if\\iftrue abc\\else d\\fi}', the first \.{\\else} that we come to after learning that the \.{\\if} is false is not the \.{\\else} we're looking for. Hence the following curious logic is needed. @ @<Skip to \.{\\else} or \.{\\fi}...@>= loop@+ begin pass←text; if cond←ptr=save←cond←ptr then begin if cur←chr<>or←code then goto common←ending; print←err("Extra "); print←esc("or"); @.Extra \\or@> help1("I'm ignoring this; it doesn't match any \if."); error; end else if cur←chr=fi←code then @<Pop the condition stack@>; end @ @<Either process \.{\\ifcase} or set |b|...@>= case this←if of if←char←code, if←cat←code: @<Test if two characters match@>; if←int←code, if←dim←code: @<Test relation between integers or dimensions@>; if←odd←code: @<Test if an integer is odd@>; if←vmode←code: b:=(abs(mode)=vmode); if←hmode←code: b:=(abs(mode)=hmode); if←mmode←code: b:=(abs(mode)=mmode); if←inner←code: b:=(mode<0); if←void←code, if←hbox←code, if←vbox←code: @<Test box register status@>; ifx←code: @<Test if two tokens match@>; if←eof←code: begin scan←four←bit←int; b:=(read←open[cur←val]=closed); end; if←true←code: b:=true; if←false←code: b:=false; if←case←code: @<Select the appropriate case and |return| or |goto common←ending|@>; end {there are no other cases} @ @<Display the value of |b|@>= begin begin←diagnostic; if b then print("{true}")@+else print("{false}"); end←diagnostic(false); end @ Here we use the fact that |"<"|, |"="|, and |">"| are consecutive ASCII codes. @↑ASCII code@> @<Test relation between integers or dimensions@>= begin if this←if=if←int←code then scan←int@+else scan←normal←dimen; n:=cur←val; @<Get the next non-blank non-call...@>; if (cur←tok>=other←token+"<")and(cur←tok<=other←token+">") then r:=cur←tok-other←token else begin print←err("Missing = inserted for "); @.Missing = inserted@> print←cmd←chr(if←test,this←if); help1("I was expecting to see `<', `=', or `>'. Didn't."); back←error; r:="="; end; if this←if=if←int←code then scan←int@+else scan←normal←dimen; case r of "<": b:=(n<cur←val); "=": b:=(n=cur←val); ">": b:=(n>cur←val); end; end @ @<Test if an integer is odd@>= begin scan←int; b:=odd(cur←val); end @ @<Test box register status@>= begin scan←eight←bit←int; p:=box(cur←val); if this←if=if←void←code then b:=(p=null) else if p=null then b:=false else if this←if=if←hbox←code then b:=(type(p)=hlist←node) else b:=(type(p)=vlist←node); end @ An active character will be treated as category 13 following \.{\\if\\noexpand} or \.{\\ifcat\\noexpand}. We use the fact that active characters have the smallest tokens, among all control sequences. @d get←x←token←or←active←char==@t@>@; begin get←x←token; if cur←cmd=relax then if cur←chr=no←expand←flag then begin cur←cmd:=active←char; cur←chr:=cur←tok-cs←token←flag-active←base; end; end @<Test if two characters match@>= begin get←x←token←or←active←char; if (cur←cmd>active←char)or(cur←chr>127) then begin m:=relax; n:=256; end else begin m:=cur←cmd; n:=cur←chr; end; get←x←token←or←active←char; if (cur←cmd>active←char)or(cur←chr>127) then begin cur←cmd:=relax; cur←chr:=256; end; if this←if=if←char←code then b:=(n=cur←chr)@+else b:=(m=cur←cmd); end @ Note that `\.{\\ifx}' will declare two macros different if one is \\{long} or \\{outer} and the other isn't, even though the texts of the macros are the same. We need to reset |scanner←status|, since \.{\\outer} control sequences are allowed, but we might be scanning a macro definition or preamble. @<Test if two tokens match@>= begin save←scanner←status:=scanner←status; scanner←status:=normal; get←next; n:=cur←cs; p:=cur←cmd; q:=cur←chr; get←next; if cur←cmd<>p then b:=false else if cur←cmd<call then b:=(cur←chr=q) else @<Test if two macro texts match@>; scanner←status:=save←scanner←status; end @ Note also that `\.{\\ifx}' decides that macros \.{\\a} and \.{\\b} are different in examples like this: $$\vbox{\halign{\.{#}\hfil\cr {}\\def\\a\{\\c\}\cr {}\\def\\b\{\\d\}\cr {}\\def\\c\{\}\cr {}\\def\\d\{\}\cr}}$$ @<Test if two macro texts match@>= begin p:=link(cur←chr); q:=link(equiv(n)); {omit reference counts} while (p<>null)and(q<>null) do if info(p)<>info(q) then p:=null else begin p:=link(p); q:=link(q); end; b:=((p=null)and(q=null)); end @ @<Select the appropriate case and |return| or |goto common←ending|@>= begin scan←int; n:=cur←val; {|n| is the number of cases to pass} if tracing←commands>1 then begin begin←diagnostic; print("{case "); print←int(n); print←char("}"); end←diagnostic(false); end; while n<>0 do begin pass←text; if cond←ptr=save←cond←ptr then if cur←chr=or←code then decr(n) else goto common←ending else if cur←chr=fi←code then @<Pop the condition stack@>; end; change←if←limit(or←code,save←cond←ptr); return; {wait for \.{\\or}, \.{\\else}, or \.{\\fi}} end @ The processing of conditionals is complete except for the following code, which is actually part of |expand|. It comes into play when \.{\\or}, \.{\\else}, or \.{\\fi} is scanned. @<Terminate the current conditional and skip to \.{\\fi}@>= if cur←chr>if←limit then if if←limit=if←code then insert←relax {condition not yet evaluated} else begin print←err("Extra "); print←cmd←chr(fi←or←else,cur←chr); @.Extra \\or@> @.Extra \\else@> @.Extra \\fi@> help1("I'm ignoring this; it doesn't match any \if."); error; end else begin while cur←chr<>fi←code do pass←text; {skip to \.{\\fi}} @<Pop the condition stack@>; end @* \[29] File names. It's time now to fret about file names. Besides the fact that different operating systems treat files in different ways, we must cope with the fact that completely different naming conventions are used by different groups of people. The following programs show what is required for one particular operating system; similar routines for other systems are not difficult to devise. @↑fingers@> @↑system dependencies@> \TeX\ assumes that a file name has three parts: the name proper; its ``extension''; and a ``file area'' where it is found in an external file system. The extension of an input file or a write file is assumed to be `\.{.tex}' unless otherwise specified; it is `\.{.log}' on the transcript file that records each run of \TeX; it is `\.{.tfm}' on the font metric files that describe characters in the fonts \TeX\ uses; it is `\.{.dvi}' on the output files that specify typesetting information; and it is `\.{.fmt}' on the format files written by \.{INITEX} to initialize \TeX. The file area can be arbitrary on input files, but files are usually output to the user's current area. If an input file cannot be found on the specified area, \TeX\ will look for it on a special system area; this special area is intended for commonly used input files like \.{webhdr.tex}. Simple uses of \TeX\ refer only to file names that have no explicit extension or area. For example, a person usually says `\.{\\input} \.{paper}' or `\.{\\font\\tenrm} \.= \.{helvetica}' instead of `\.{\\input} \.{paper.new}' or `\.{\\font\\tenrm} \.= \.{<csd.knuth>test}'. Simple file names are best, because they make the \TeX\ source files portable; whenever a file name consists entirely of letters and digits, it should be treated in the same way by all implementations of \TeX. However, users need the ability to refer to other files in their environment, especially when responding to error messages concerning unopenable files; therefore we want to let them use the syntax that appears in their favorite operating system. @ In order to isolate the system-dependent aspects of file names, the @↑system dependencies@> system-independent parts of \TeX\ make use of three system-dependent procedures that are called |begin←name|, |more←name|, and |end←name|. In essence, if the user-specified characters of the file name are $c←1\ldots c←n$, the system-independent driver program does the operations $$|begin←name|;\,|more←name|(c←1);\,\ldots\,;|more←name|(c←n); \,|end←name|.$$ These three procedures communicate with each other via global variables. Afterwards the file name will appear in the string pool as three strings called |cur←name|\penalty10000\hskip-.05em, |cur←area|, and |cur←ext|; the latter two are null (i.e., |""|), unless they were explicitly specified by the user. Actually the situation is slightly more complicated, because \TeX\ needs to know when the file name ends. The |more←name| routine is a function (with side effects) that returns |true| on the calls |more←name|$(c←1)$, \dots, |more←name|$(c←{n-1})$. The final call |more←name|$(c←n)$ returns |false|; or, it returns |true| and the token following $c←n$ is something like `\.{\\hbox}' (i.e., not a character). In other words, |more←name| is supposed to return |true| unless it is sure that the file name has been completely scanned; and |end←name| is supposed to be able to finish the assembly of |cur←name|, |cur←area|, and |cur←ext| regardless of whether $|more←name|(c←n)$ returned |true| or |false|. @<Glob...@>= @!cur←name:str←number; {name of file just scanned} @!cur←area:str←number; {file area just scanned, or \.{""}} @!cur←ext:str←number; {file extension just scanned, or \.{""}} @ The file names we shall deal with for illustrative purposes have the following structure: If the name contains `\.>' or `\.:', the file area consists of all characters up to and including the final such character; otherwise the file area is null. If the remaining file name contains `\..', the file extension consists of all such characters from the first remaining `\..' to the end, otherwise the file extension is null. @↑system dependencies@> We can scan such file names easily by using two global variables that keep track of the occurrences of area and extension delimiters: @<Glob...@>= @!area←delimiter:pool←pointer; {the most recent `\.>' or `\.:', if any} @!ext←delimiter:pool←pointer; {the relevant `\..', if any} @ Input files that can't be found in the user's area may appear in a standard system area called |TEX←area|. Font metric files whose areas are not given explicitly are assumed to appear in a standard system area called |TEX←font←area|. These system area names will, of course, vary from place to place. @↑system dependencies@> @d TEX←area=="TeXinputs:" @.TeXinputs@> @d TEX←font←area=="TeXfonts:" @.TeXfonts@> @ Here now is the first of the system-dependent routines for file name scanning. @↑system dependencies@> @p procedure begin←name; begin area←delimiter:=0; ext←delimiter:=0; end; @ And here's the second. @↑system dependencies@> @p function more←name(@!c:ASCII←code):boolean; begin if c=" " then more←name:=false else begin if (c=">")or(c=":") then begin area←delimiter:=pool←ptr; ext←delimiter:=0; end else if (c=".")and(ext←delimiter=0) then ext←delimiter:=pool←ptr; str←room(1); append←char(c); {contribute |c| to the current string} more←name:=true; end; end; @ The third. @↑system dependencies@> @p procedure end←name; begin if str←ptr+3>max←strings then overflow("number of strings",max←strings-init←str←ptr); @:TeX capacity exceeded number of strings}{\quad number of strings@> if area←delimiter=0 then cur←area:="" else begin cur←area:=str←ptr; incr(str←ptr); str←start[str←ptr]:=area←delimiter+1; end; if ext←delimiter=0 then begin cur←ext:=""; cur←name:=make←string; end else begin cur←name:=str←ptr; incr(str←ptr); str←start[str←ptr]:=ext←delimiter; cur←ext:=make←string; end; end; @ Conversely, here is a routine that takes three strings and prints a file name that might have produced them. (The routine is system dependent, because some operating systems put the file area last instead of first.) @↑system dependencies@> @<Basic printing...@>= procedure print←file←name(@!n,@!a,@!e:integer); begin print(a); print(n); print(e); end; @ Another system-dependent routine is needed to convert three \TeX\ strings into the |name←of←file| value that is used to open files. The present code allows both lowercase and uppercase letters in the file name. @↑system dependencies@> @d append←to←name(#)==begin c:=#; incr(k); if k<=file←name←size then name←of←file[k]:=xchr[c]; end @p procedure pack←file←name(@!n,@!a,@!e:str←number); var k:integer; {number of positions filled in |name←of←file|} @!c: ASCII←code; {character being packed} @!j:pool←pointer; {index into |str←pool|} begin k:=0; for j:=str←start[a] to str←start[a+1]-1 do append←to←name(str←pool[j]); for j:=str←start[n] to str←start[n+1]-1 do append←to←name(str←pool[j]); for j:=str←start[e] to str←start[e+1]-1 do append←to←name(str←pool[j]); if k<=file←name←size then name←length:=k@+else name←length:=file←name←size; for k:=name←length+1 to file←name←size do name←of←file[k]:=' '; end; @ A messier routine is also needed, since format file names must be scanned before \TeX's string mechanism has been initialized. We shall use the global variable |TEX←format←default| to supply the text for default system areas and extensions related to format files. @↑system dependencies@> @d format←default←length=20 {length of the |TEX←format←default| string} @d format←area←length=11 {length of its area part} @d format←ext←length=4 {length of its `\.{.fmt}' part} @<Glob...@>= @!TEX←format←default:packed array[1..format←default←length] of char; @ @<Set init...@>= TEX←format←default:='TeXformats:PLAIN.fmt'; @.TeXformats@> @.PLAIN@> @↑system dependencies@> @ @<Check the ``constant'' values for consistency@>= if format←default←length>file←name←size then bad:=31; @ Here is the messy routine that was just mentioned. It sets |name←of←file| from the first |n| characters of |TEX←format←default|, followed by |buffer[a..b]|, followed by the last |format←ext←length| characters of |TEX←format←default|. We dare not give error messages here, since \TeX\ calls this routine before the |error| routine is ready to roll. Instead, we simply drop excess characters, since the error will be detected in another way when a strange file name isn't found. @↑system dependencies@> @p procedure pack←buffered←name(@!n:small←number;@!a,@!b:integer); var k:integer; {number of positions filled in |name←of←file|} @!c: ASCII←code; {character being packed} @!j:integer; {index into |buffer| or |TEX←format←default|} begin if n+b-a+1+format←ext←length>file←name←size then b:=a+file←name←size-n-1-format←ext←length; k:=0; for j:=1 to n do append←to←name(xord[TEX←format←default[j]]); for j:=a to b do append←to←name(buffer[j]); for j:=format←default←length-format←ext←length+1 to format←default←length do append←to←name(xord[TEX←format←default[j]]); if k<=file←name←size then name←length:=k@+else name←length:=file←name←size; for k:=name←length+1 to file←name←size do name←of←file[k]:=' '; end; @ Here is the only place we use |pack←buffered←name|. This part of the program becomes active when a ``virgin'' \TeX\ is trying to get going, just after the preliminary initialization, or when the user is substituting another format file by typing `\.\&' after the initial `\.{**}' prompt. The buffer contains the first line of input in |buffer[loc..(last-1)]|, where |loc<last| and |buffer[loc]<>" "|. @<Declare the function called |open←fmt←file|@>= function open←fmt←file:boolean; label found,exit; var j:0..buf←size; {the first space after the file name} begin if buffer[loc]="&" then begin incr(loc); j:=loc; buffer[last]:=" "; while buffer[j]<>" " do incr(j); pack←buffered←name(0,loc,j-1); {try first without the system file area} if w←open←in(fmt←file) then begin loc:=j; goto found; end;@/ pack←buffered←name(format←area←length,loc,j-1); {now try the system format file area} if w←open←in(fmt←file) then begin loc:=j; goto found; end; wake←up←terminal; wterm←ln('Sorry, I can''t find that format;',' will try PLAIN.'); @.Sorry, I can't find...@> end; {now pull out all the stops: try for the system \.{PLAIN} file} pack←buffered←name(format←default←length-format←ext←length,1,0); if not w←open←in(fmt←file) then begin wake←up←terminal; wterm←ln('I can''t find the PLAIN format file!'); @.I can't find PLAIN...@> @.PLAIN@> open←fmt←file:=false; return; end; found:open←fmt←file:=true; exit:end; @ Operating systems often make it possible to determine the exact name (and possible version number) of a file that has been opened. The following routine, which simply makes a \TeX\ string from the value of |name←of←file|, should ideally be changed to deduce the full name of file~|f|, if it is possible to do this in a \PASCAL\ program. @↑system dependencies@> @p function make←name←string:str←number; var k:1..file←name←size; {index into |name←of←file|} begin str←room(name←length); for k:=1 to name←length do append←char(xord[name←of←file[k]]); make←name←string:=make←string; end; function a←make←name←string(var f:alpha←file):str←number; begin a←make←name←string:=make←name←string; end; function b←make←name←string(var f:byte←file):str←number; begin b←make←name←string:=make←name←string; end; function w←make←name←string(var f:word←file):str←number; begin w←make←name←string:=make←name←string; end; @ Now let's consider the routines by which \TeX\ deals with file names in a system-independent manner. First comes a procedure that looks for a file name in the input by calling |get←x←token| for the information. @p procedure scan←file←name; label done; begin name←in←progress:=true; begin←name; @<Get the next non-blank non-call...@>; loop@+begin if (cur←cmd>other←char)or(cur←chr>127) then {not a character} begin back←input; goto done; end; if not more←name(cur←chr) then goto done; get←x←token; end; done: end←name; name←in←progress:=false; end; @ The global variable |name←in←progress| is used to prevent recursive use of |scan←file←name|, since the |begin←name| and other procedures communicate via global variables. Recursion would arise only by devious tricks like `\.{\\input\\input f}'; such attempts at sabotage must be thwarted. Furthermore, |name←in←progress| prevents \.{\\input} @↑recursion@> from being initiated when a font size specification is being scanned. Another global variable, |job←name|, contains the file name that was first \.{\\input} by the user. This name is extended by `\.{log}' and `\.{dvi}' and `\.{fmt}' in order to make the names of \TeX's output files. @<Glob...@>= @!name←in←progress:boolean; {is a file name being scanned?} @!job←name:str←number; {principal file name} @ Initially |job←name=0|; it becomes nonzero as soon as the true name is known. We have |job←name=0| if and only if the `\.{log}' file has not been opened, except of course for a short time just after |job←name| has become nonzero. @<Initialize the output...@>=job←name:=0; name←in←progress:=false; @ Here is a routine that manufactures the output file names, assuming that |job←name<>0|. It ignores and changes the current settings of |cur←area| and |cur←ext|. @d pack←cur←name==pack←file←name(cur←name,cur←area,cur←ext) @p procedure pack←job←name(@!s:str←number); {|s = ".log"|, |".dvi"|, or |".fmt"|} begin cur←area:=""; cur←ext:=s; cur←name:=job←name; pack←cur←name; end; @ If some trouble arises when \TeX\ tries to open a file, the following routine calls upon the user to supply another file name. Parameter~|s| is used in the error message to identify the type of file; parameter~|e| is the default extension if none is given. Upon exit from the routine, variables |cur←name|, |cur←area|, |cur←ext|, and |name←of←file| are ready for another attempt at file opening. @p procedure prompt←file←name(@!s,@!e:str←number); label done; var k:0..buf←size; {index into |buffer|} begin if interaction=scroll←mode then wake←up←terminal; if s="input file name" then print←err("I can't find file `") @.I can't find file x@> else print←err("I can't write on file `"); @.I can't write on file x@> print←file←name(cur←name,cur←area,cur←ext); print("'."); if e=".tex" then show←context; print←nl("Please type another "); print(s); @.Please type...@> if interaction<scroll←mode then fatal←error("*** (job aborted, file error in nonstop mode)"); @.job aborted, file error...@> clear←terminal; prompt←input(": "); @<Scan file name in the buffer@>; if cur←ext="" then cur←ext:=e; pack←cur←name; end; @ @<Scan file name in the buffer@>= begin begin←name; k:=first; while (buffer[k]=" ")and(k<last) do incr(k); loop@+ begin if k=last then goto done; if not more←name(buffer[k]) then goto done; incr(k); end; done:end←name; end @ Here's an example of how these conventions are used. We shall use the macro |ensure←dvi←open| when it is time to ship out a box of stuff. @d ensure←dvi←open==if output←file←name=0 then begin if job←name=0 then open←log←file; pack←job←name(".dvi"); while not b←open←out(dvi←file) do prompt←file←name("file name for output",".dvi"); output←file←name:=b←make←name←string(dvi←file); end @<Glob...@>= @!dvi←file: byte←file; {the device-independent output goes here} @!output←file←name: str←number; {full name of the output file} @!log←name:str←number; {full name of the log file} @ @<Initialize the output...@>=output←file←name:=0; @ The |open←log←file| routine is used to open the transcript file and to help it catch up to what has previously been printed on the terminal. @p procedure open←log←file; var old←setting:0..max←selector; {previous |selector| setting} @!k:0..buf←size; {index into |months| and |buffer|} @!l:0..buf←size; {end of first input line} @!months:packed array [1..36] of char; {abbreviations of month names} begin old←setting:=selector; if job←name=0 then job←name:="texput"; @.texput@> pack←job←name(".log"); while not a←open←out(log←file) do @<Try to get a different log file name@>; log←name:=a←make←name←string(log←file); selector:=log←only; @<Print the banner line, including the date and time@>; input←stack[input←ptr]:=cur←input; {make sure bottom level is in memory} print←nl("**"); @.**@> l:=input←stack[0].limit←field; {last position of first line} if buffer[l]=end←line←char then decr(l); for k:=1 to l do print(buffer[k]); print←ln; {now the transcript file contains the first line of input} selector:=old←setting+2; {|log←only| or |term←and←log|} end; @ Sometimes |open←log←file| is called at awkward moments when \TeX\ is unable to print error messages or even to |show←context|. Therefore the program is careful not to call |prompt←file←name| if a fatal error could result. Incidentally, the program always refers to the log file as a `\.{transcript file}', because some systems cannot use the extension `\.{.log}' for this file. @<Try to get a different log file name@>= begin if interaction<scroll←mode then {bypass |fatal←error|} begin print←err("I can't write on file `"); @.I can't write on file x@> print←file←name(cur←name,cur←area,cur←ext); print("'.");@/ job←name:=0; history:=fatal←error←stop; jump←out; end; {abort the program without a log file} prompt←file←name("transcript file name",".log"); end @ @<Print the banner...@>= begin wlog(banner); print(format←ident); print(" "); print←int(day); print←char(" "); months:='JANFEBMARAPRMAYJUNJULAUGSEPOCTNOVDEC'; for k:=3*month-2 to 3*month do wlog(months[k]); print←char(" "); print←int(year); print←char(" "); print←two(time div 60); print←char(":"); print←two(time mod 60); end @ Let's turn now to the procedure that is used to initiate file reading when an `\.{\\input}' command is being processed. @p procedure start←input; {\TeX\ will \.{\\input} something} label done; begin scan←file←name; {set |cur←name| to desired file name} if cur←ext="" then cur←ext:=".tex"; pack←cur←name; loop@+ begin begin←file←reading; {set up |cur←file| and new level of input} if a←open←in(cur←file) then goto done; pack←file←name(cur←name,TEX←area,cur←ext); if a←open←in(cur←file) then goto done; end←file←reading; {remove the level that didn't work} prompt←file←name("input file name",".tex"); end; done: name:=a←make←name←string(cur←file); if job←name=0 then begin job←name:=cur←name; open←log←file; end; {|open←log←file| doesn't |show←context|, so |limit| and |loc| needn't be set to meaningful values yet} if term←offset+length(name)>max←print←line-2 then print←ln else if (term←offset>0)or(file←offset>0) then print←char(" "); print←char("("); print(name); update←terminal; state:=new←line; if name=str←ptr-1 then {we can conserve string pool space now} begin flush←string; name:=cur←name; end; @<Read the first line of the new file@>; end; @ Here we have to remember to tell the |input←ln| routine not to start with a |get|. If the file is empty, it is considered to contain a single blank line. @↑system dependencies@> @↑empty line at end of file@> @<Read the first line...@>= begin if not input←ln(cur←file,false) then do←nothing; firm←up←the←line; if (end←line←char<0)or(end←line←char>127) then decr(limit) else buffer[limit]:=end←line←char; first:=limit+1; loc:=start; line:=1; end @* \[30] Font metric data. \TeX\ gets its knowledge about fonts from font metric files, also called \.{TFM} files; the `\.T' in `\.{TFM}' stands for \TeX, but other programs know about them too. @:TFM files}{\.{TFM} files@> @↑font metric files@> The information in a \.{TFM} file appears in a sequence of 8-bit bytes. Since the number of bytes is always a multiple of 4, we could also regard the file as a sequence of 32-bit words, but \TeX\ uses the byte interpretation. The format of \.{TFM} files was designed by Lyle Ramshaw in 1980. The intent is to convey a lot of different kinds @↑Ramshaw, Lyle Harold@> of information in a compact but useful form. @<Glob...@>= @!tfm←file:byte←file; @ The first 24 bytes (6 words) of a \.{TFM} file contain twelve 16-bit integers that give the lengths of the various subsequent portions of the file. These twelve integers are, in order: $$\vbox{\halign{\hfil#&$\null=\null$#\hfil\cr |lf|&length of the entire file, in words;\cr |lh|&length of the header data, in words;\cr |bc|&smallest character code in the font;\cr |ec|&largest character code in the font;\cr |nw|&number of words in the width table;\cr |nh|&number of words in the height table;\cr |nd|&number of words in the depth table;\cr |ni|&number of words in the italic correction table;\cr |nl|&number of words in the lig/kern table;\cr |nk|&number of words in the kern table;\cr |ne|&number of words in the extensible character table;\cr |np|&number of font parameter words.\cr}}$$ They are all nonnegative and less than $2↑{15}$. We must have |bc-1<=ec<=255|, and $$\hbox{|lf=6+lh+(ec-bc+1)+nw+nh+nd+ni+nl+nk+ne+np|.}$$ Note that a font may contain as many as 256 characters (if |bc=0| and |ec=255|), and as few as 0 characters (if |bc=ec+1|). Incidentally, when two or more 8-bit bytes are combined to form an integer of 16 or more bits, the most significant bytes appear first in the file. This is called BigEndian order. @!@↑BigEndian order@> @ The rest of the \.{TFM} file may be regarded as a sequence of ten data arrays having the informal specification $$\def\arr$[#1]#2${\&{array} $[#1]$ \&{of} #2} \vbox{\halign{\hfil\\{#}&$\,:\,$\arr#\hfil\cr header&|[0..lh-1]@t\\{stuff}@>|\cr char\←info&|[bc..ec]char←info←word|\cr width&|[0..nw-1]fix←word|\cr height&|[0..nh-1]fix←word|\cr depth&|[0..nd-1]fix←word|\cr italic&|[0..ni-1]fix←word|\cr lig\←kern&|[0..nl-1]lig←kern←command|\cr kern&|[0..nk-1]fix←word|\cr exten&|[0..ne-1]extensible←recipe|\cr param&|[1..np]fix←word|\cr}}$$ The most important data type used here is a |@!fix←word|, which is a 32-bit representation of a binary fraction. A |fix←word| is a signed quantity, with the two's complement of the entire word used to represent negation. Of the 32 bits in a |fix←word|, exactly 12 are to the left of the binary point; thus, the largest |fix←word| value is $2048-2↑{-20}$, and the smallest is $-2048$. We will see below, however, that all but two of the |fix←word| values must lie between $-16$ and $+16$. @ The first data array is a block of header information, which contains general facts about the font. The header must contain at least two words, |header[0]| and |header[1]|, whose meaning is explained below. Additional header information of use to other software routines might also be included, but \TeX82 does not need to know about such details. For example, 16 more words of header information are in use at the Xerox Palo Alto Research Center; the first ten specify the character coding scheme used (e.g., `\.{XEROX TEXT}' or `\.{TeX MATHSY}'), the next five give the font family name (e.g., `\.{HELVETICA}' or `\.{CMSY}'), and the last gives the ``face byte.'' The program that converts \.{DVI} files to Xerox printing format gets this information by looking at the \.{TFM} file, which it needs to read anyway because of other information that is not explicitly repeated in \.{DVI} format. Extensions of \TeX\ for oriental languages should be able to identify oriental fonts by means of this additional header information. @↑oriental characters@>@↑Chinese characters@>@↑Japanese characters@> \yskip\hang|header[0]| is a 32-bit check sum that \TeX\ will copy into the \.{DVI} output file. Later on when the \.{DVI} file is printed, possibly on another computer, the actual font that gets used is supposed to have a check sum that agrees with the one in the \.{TFM} file used by \TeX. In this way, users will be warned about potential incompatibilities. (However, if the check sum is zero in either the font file or the \.{TFM} file, no check is made.) The actual relation between this check sum and the rest of the \.{TFM} file is not important; the check sum is simply an identification number with the property that incompatible fonts almost always have distinct check sums. @↑check sum@> \yskip\hang|header[1]| is a |fix←word| containing the design size of the font, in units of \TeX\ points. This number must be at least 1.0; it is fairly arbitrary, but usually the design size is 10.0 for a ``10 point'' font, i.e., a font that was designed to look best at a 10-point size, whatever that really means. When a \TeX\ user asks for a font `\.{at} $\delta$ \.{pt}', the effect is to override the design size and replace it by $\delta$, and to multiply the $x$ and~$y$ coordinates of the points in the font image by a factor of $\delta$ divided by the design size. {\sl All other dimensions in the\/ \.{TFM} file are |fix←word|\kern-1pt\ numbers in design-size units}, with the exception of |param[1]| (which denotes the slant ratio). Thus, for example, the value of |param[6]|, which defines the \.{em} unit, is often the |fix←word| value $2↑{20}=1.0$, since many fonts have a design size equal to one em. The other dimensions must be less than 16 design-size units in absolute value; thus, |header[1]| and |param[1]| are the only |fix←word| entries in the whole \.{TFM} file whose first byte might be something besides 0 or 255. @ Next comes the |char←info| array, which contains one |@!char←info←word| per character. Each word in this part of the file contains six fields packed into four bytes as follows. \yskip\hang first byte: |@!width←index| (8 bits)\par \hang second byte: |@!height←index| (4 bits) times 16, plus |@!depth←index| (4~bits)\par \hang third byte: |@!italic←index| (6 bits) times 4, plus |@!tag| (2~bits)\par \hang fourth byte: |@!remainder| (8 bits)\par \yskip\noindent The actual width of a character is \\{width}|[width←index]|, in design-size units; this is a device for compressing information, since many characters have the same width. Since it is quite common for many characters to have the same height, depth, or italic correction, the \.{TFM} format imposes a limit of 16 different heights, 16 different depths, and 64 different italic corrections. @!@↑italic correction@> The italic correction of a character has two different uses. (a)~In ordinary text, the italic correction is added to the width only if the \TeX\ user specifies `\.{\\/}' after the character. (b)~In math formulas, the italic correction is always added to the width, except with respect to the positioning of subscripts. Incidentally, the relation $\\{width}[0]=\\{height}[0]=\\{depth}[0]= \\{italic}[0]=0$ should always hold, so that an index of zero implies a value of zero. The |width←index| should never be zero unless the character does not exist in the font, since a character is valid if and only if it lies between |bc| and |ec| and has a nonzero |width←index|. @ The |tag| field in a |char←info←word| has four values that explain how to interpret the |remainder| field. \yskip\hang|tag=0| (|no←tag|) means that |remainder| is unused.\par \hang|tag=1| (|lig←tag|) means that this character has a ligature/kerning program starting at |lig←kern[remainder]|.\par \hang|tag=2| (|list←tag|) means that this character is part of a chain of characters of ascending sizes, and not the largest in the chain. The |remainder| field gives the character code of the next larger character.\par \hang|tag=3| (|ext←tag|) means that this character code represents an extensible character, i.e., a character that is built up of smaller pieces so that it can be made arbitrarily large. The pieces are specified in |@!exten[remainder]|.\par \yskip\noindent Characters with |tag=2| and |tag=3| are treated as characters with |tag=0| unless they are used in special circumstances in math formulas. For example, the \.{\\sum} operation looks for a |list←tag|, and the \.{\\left} operation looks for both |list←tag| and |ext←tag|. @d no←tag=0 {vanilla character} @d lig←tag=1 {character has a ligature/kerning program} @d list←tag=2 {character has a successor in a charlist} @d ext←tag=3 {character is extensible} @ The |lig←kern| array contains instructions in a simple programming language that explains what to do for special letter pairs. Each word in this array is a |@!lig←kern←command| of four bytes. \yskip\hang first byte: |stop←bit|, indicates that this is the final program step if the byte is 128 or more.\par \hang second byte: |next←char|, ``if |next←char| follows the current character, then perform the operation and stop, otherwise continue.''\par \hang third byte: |op←bit|, indicates a ligature step if less than~128, a kern step otherwise.\par \hang fourth byte: |remainder|.\par \yskip\noindent In a ligature step the current character and |next←char| are replaced by the single character whose code is |remainder|. In a kern step, an additional space equal to |@!kern[remainder]| is inserted between the current character and |next←char|. (The value of |kern[remainder]| is often negative, so that the characters are brought closer together by kerning; but it might be positive.) @d stop←flag=128+min←quarterword {value indicating `\.{STOP}' in a lig/kern program} @d kern←flag=128+min←quarterword {op code for a kern step} @d stop←bit(#)==#.b0 @d next←char(#)==#.b1 @d op←bit(#)==#.b2 @d rem←byte(#)==#.b3 @ Extensible characters are specified by an |@!extensible←recipe|, which consists of four bytes called |@!top|, |@!mid|, |@!bot|, and |@!rep| (in this order). These bytes are the character codes of individual pieces used to build up a large symbol. If |top|, |mid|, or |bot| are zero, they are not present in the built-up result. For example, an extensible vertical line is like an extensible bracket, except that the top and bottom pieces are missing. Let $T$, $M$, $B$, and $R$ denote the respective pieces, or an empty box if the piece isn't present. Then the extensible characters have the form $TR↑kMR↑kB$ from top to bottom, for some |k>=0|, unless $M$ is absent; in the latter case we can have $TR↑kB$ for both even and odd values of~|k|. The width of the extensible character is the width of $R$; and the height-plus-depth is the sum of the individual height-plus-depths of the components used, since the pieces are butted together in a vertical list. @d ext←top(#)==#.b0 {|top| piece in a recipe} @d ext←mid(#)==#.b1 {|mid| piece in a recipe} @d ext←bot(#)==#.b2 {|bot| piece in a recipe} @d ext←rep(#)==#.b3 {|rep| piece in a recipe} @ The final portion of a \.{TFM} file is the |param| array, which is another sequence of |fix←word| values. \yskip\hang|param[1]=slant| is the amount of italic slant, which is used to help position accents. For example, |slant=.25| means that when you go up one unit, you also go .25 units to the right. The |slant| is a pure number; it's the only |fix←word| other than the design size itself that is not scaled by the design size. \hang|param[2]=space| is the normal spacing between words in text. Note that character |" "| in the font need not have anything to do with blank spaces. \hang|param[3]=space←stretch| is the amount of glue stretching between words. \hang|param[4]=space←shrink| is the amount of glue shrinking between words. \hang|param[5]=x←height| is the size of one ex in the font; it is also the height of letters for which accents don't have to be raised or lowered. \hang|param[6]=quad| is the size of one em in the font. \hang|param[7]=extra←space| is the amount added to |param[2]| at the ends of sentences. \yskip\noindent If fewer than seven parameters are present, \TeX\ sets the missing parameters to zero. Fonts used for math symbols are required to have additional parameter information, which is explained later. @d slant←code=1 @d space←code=2 @d space←stretch←code=3 @d space←shrink←code=4 @d x←height←code=5 @d quad←code=6 @d extra←space←code=7 @ So that is what \.{TFM} files hold. Since \TeX\ has to absorb such information about lots of fonts, it stores most of the data in a large array called |font←info|. Each item of |font←info| is a |memory←word|; the |fix←word| data gets converted into |scaled| entries, while everything else goes into words of type |four←quarters|. When the user defines \.{\\font\\f}, say, \TeX\ assigns an internal number to the user's font~\.{\\f}. Adding this number to |font←id←base| gives the |eqtb| location of a ``frozen'' control sequence that will always select the font. @<Types...@>= @!internal←font←number=font←base..font←max; {|font| in a |char←node|} @ Here now is the (rather formidable) array of font arrays. @<Glob...@>= @!font←info:array[0..font←mem←size] of memory←word; {the big collection of font data} @!fmem←ptr:0..font←mem←size; {first unused word of |font←info|} @!font←ptr:internal←font←number; {largest internal font number in use} @!font←check:array[internal←font←number] of four←quarters; {check sum} @!font←size:array[internal←font←number] of scaled; {``at'' size} @!font←dsize:array[internal←font←number] of scaled; {``design'' size} @!font←params:array[internal←font←number] of halfword; {how many font parameters are present} @!font←name:array[internal←font←number] of str←number; {name of the font} @!font←area:array[internal←font←number] of str←number; {area of the font} @!font←bc:array[internal←font←number] of eight←bits; {beginning (smallest) character code} @!font←ec:array[internal←font←number] of eight←bits; {ending (largest) character code} @!font←glue:array[internal←font←number] of pointer; {glue specification for interword space, |null| if not allocated} @!font←used:array[internal←font←number] of boolean; {has a character from this font actually appeared in the output?} @!hyphen←char:array[internal←font←number] of integer; {current \.{\\hyphenchar} values} @!skew←char:array[internal←font←number] of integer; {current \.{\\skewchar} values} @ Besides the arrays just enumerated, we have directory arrays that make it easy to get at the individual entries in |font←info|. For example, the |char←info| data for character |c| in font |f| will be in |font←info[char←base[f]+c].qqqq|; and if |w| is the |width←index| part of this word (the |b0| field), the width of the character is |font←info[width←base[f]+w].sc|. (These formulas assume that |min←quarterword| has already been added to |c| and to |w|, since \TeX\ stores its quarterwords that way.) @<Glob...@>= @!char←base:array[internal←font←number] of integer; {base addresses for |char←info|} @!width←base:array[internal←font←number] of integer; {base addresses for widths} @!height←base:array[internal←font←number] of integer; {base addresses for heights} @!depth←base:array[internal←font←number] of integer; {base addresses for depths} @!italic←base:array[internal←font←number] of integer; {base addresses for italic corrections} @!lig←kern←base:array[internal←font←number] of integer; {base addresses for ligature/kerning programs} @!kern←base:array[internal←font←number] of integer; {base addresses for kerns} @!exten←base:array[internal←font←number] of integer; {base addresses for extensible recipes} @!param←base:array[internal←font←number] of integer; {base addresses for font parameters} @ @<Set init...@>= for k:=font←base to font←max do font←used[k]:=false; @ \TeX\ always knows at least one font, namely the null font. It has no characters, and its seven parameters are all equal to zero. @<Initialize table...@>= font←ptr:=null←font; fmem←ptr:=7; font←name[null←font]:="nullfont"; font←area[null←font]:=""; hyphen←char[null←font]:="-"; skew←char[null←font]:=-1; font←bc[null←font]:=1; font←ec[null←font]:=0; font←size[null←font]:=0; font←dsize[null←font]:=0; char←base[null←font]:=0; width←base[null←font]:=0; height←base[null←font]:=0; depth←base[null←font]:=0; italic←base[null←font]:=0; lig←kern←base[null←font]:=0; kern←base[null←font]:=0; exten←base[null←font]:=0; font←glue[null←font]:=null; font←params[null←font]:=7; param←base[null←font]:=-1; for k:=0 to 6 do font←info[k].sc:=0; @ @<Put each...@>= primitive("nullfont",set←font,null←font); @!@:null←font←}{\.{\\nullfont} primitive@> text(frozen←null←font):="nullfont"; eqtb[frozen←null←font]:=eqtb[cur←val]; @ Of course we want to define macros that suppress the detail of how font information is actually packed, so that we don't have to write things like $$\hbox{|font←info[width←base[f]+font←info[char←base[f]+c].qqqq.b0].sc|}$$ too often. The \.{WEB} definitions here make |char←info(f)(c)| the |four←quarters| word of font information corresponding to character |c| of font |f|. If |q| is such a word, |char←width(f)(q)| will be the character's width; hence the long formula above is at least abbreviated to $$\hbox{|char←width(f)(char←info(f)(c))|.}$$ Usually, of course, we will fetch |q| first and look at several of its fields at the same time. The italic correction of a character will be denoted by |char←italic(f)(q)|, so it is analogous to |char←width|. But we will get at the height and depth in a slightly different way, since we usually want to compute both height and depth if we want either one. The value of |height←depth(q)| will be the 8-bit quantity $$b=|height←index|\times16+|depth←index|,$$ and if |b| is such a byte we will write |char←height(f)(b)| and |char←depth(f)(b)| for the height and depth of the character |c| for which |q=char←info(f)(c)|. Got that? The tag field will be called |char←tag(q)|; the remainder byte will be called |rem←byte(q)|, using a macro that we have already defined above. Access to a character's |height|, |depth|, and |tag| fields is part of \TeX's inner loop, so we want these macros to produce code that is as fast as possible under the circumstances. @↑inner loop@> @d char←info←end(#)==#].qqqq @d char←info(#)==font←info[char←base[#]+char←info←end @d char←width←end(#)==#.b0].sc @d char←width(#)==font←info[width←base[#]+char←width←end @d char←exists(#)==(#.b0>min←quarterword) @d char←italic←end(#)==(qo(#.b2)) div 4].sc @d char←italic(#)==font←info[italic←base[#]+char←italic←end @d height←depth(#)==qo(#.b1) @d char←height←end(#)==(#) div 16].sc @d char←height(#)==font←info[height←base[#]+char←height←end @d char←depth←end(#)==(#) mod 16].sc @d char←depth(#)==font←info[depth←base[#]+char←depth←end @d char←tag(#)==((qo(#.b2)) mod 4) @ The global variable |null←character| is set up to be a word of |char←info| for a character that doesn't exist. Such a word provides a convenient way to deal with erroneous situations. @<Glob...@>= @!null←character:four←quarters; {nonexistent character information} @ @<Set init...@>= null←character.b0:=min←quarterword; null←character.b1:=min←quarterword; null←character.b2:=min←quarterword; null←character.b3:=min←quarterword; @ Here are some macros that help process ligatures and kerns. We write |char←kern(f)(j)| to find the amount of kerning specified by kerning command~|j| in font~|f|. @d lig←kern←start(#)==lig←kern←base[#]+rem←byte {beginning of lig/kern program} @d char←kern←end(#)==rem←byte(#)].sc @d char←kern(#)==font←info[kern←base[#]+char←kern←end @ Font parameters are referred to as |slant(f)|, |space(f)|, etc. @d param←end(#)==param←base[#]].sc @d param(#)==font←info[#+param←end @d slant==param(slant←code) {slant to the right, per unit distance upward} @d space==param(space←code) {normal space between words} @d space←stretch==param(space←stretch←code) {stretch between words} @d space←shrink==param(space←shrink←code) {shrink between words} @d x←height==param(x←height←code) {one ex} @d quad==param(quad←code) {one em} @d extra←space==param(extra←space←code) {additional space at end of sentence} @<The em width for |cur←font|@>=quad(cur←font) @ @<The x-height for |cur←font|@>=x←height(cur←font) @ \TeX\ checks the information of a \.{TFM} file for validity as the file is being read in, so that no further checks will be needed when typesetting is going on. The somewhat tedious subroutine that does this is called |read←font←info|. It has four parameters: the user font identifier~|u|, the file name and area strings |nom| and |aire|, and the ``at'' size~|s|. If |s|~is negative, its the negative of a scale factor to be applied to the design size; |s=-1000| is the normal case. Otherwise |s| will be substituted for the design size; in this case, |s| must be positive and less than $2048\rm\,pt$ (i.e., it must be less than $2↑{27}$ when considered as an integer). The subroutine opens and closes a global file variable called |tfm←file|. It returns the value of the internal font number that was just loaded. If an error is detected, an error message is issued and no font information is stored; |null←font| is returned in this case. @d bad←tfm=11 {label for |read←font←info|} @d abort==goto bad←tfm {do this when the \.{TFM} data is wrong} @p function read←font←info(@!u:pointer;@!nom,@!aire:str←number; @!s:scaled):internal←font←number; {input a \.{TFM} file} label done,bad←tfm,not←found; var k:0..font←mem←size; {index into |font←info|} @!file←opened:boolean; {was |tfm←file| successfully opened?} @!lf,@!lh,@!bc,@!ec,@!nw,@!nh,@!nd,@!ni,@!nl,@!nk,@!ne,@!np:halfword; {sizes of subfiles} @!f:internal←font←number; {the new font's number} @!g:internal←font←number; {the number to return} @!a,@!b,@!c,@!d:eight←bits; {byte variables} @!qw:four←quarters;@!sw:scaled; {accumulators} @!z:scaled; {the design size or the ``at'' size} @!alpha:integer;@!beta:1..16; {auxiliary quantities used in fixed-point multiplication} begin g:=null←font;@/ @<Read and check the font data; |abort| if the \.{TFM} file is malformed; if there's no room for this font, say so and |goto done|; otherwise |incr(font←ptr)| and |goto done|@>; bad←tfm: @<Report that the font won't be loaded@>; done: b←close(tfm←file); read←font←info:=g; end; @ There are programs called \.{TFtoPL} and \.{PLtoTF} that convert between the \.{TFM} format and a symbolic property-list format that can be easily edited. These programs contain extensive diagnostic information, so \TeX\ does not have to bother giving precise details about why it rejects a particular \.{TFM} file. @.TFtoPL@> @.PLtoTF@> @d start←font←error←message==print←err("Font "); sprint←cs(u); print←char("="); print←file←name(nom,aire,""); if s>=0 then begin print(" at "); print←scaled(s); print("pt"); end else if s<>-1000 then begin print(" scaled "); print←int(-s); end @<Report that the font won't be loaded@>= start←font←error←message; @.Font x=xx not loadable...@> if file←opened then print(" not loadable: Bad metric (TFM) file") else print(" not loadable: Metric (TFM) file not found"); help5("I wasn't able to read the size data for this font,")@/ ("so I will ignore the font specification.")@/ ("[Wizards can fix TFM files using TFtoPL/PLtoTF.]")@/ ("You might try inserting a different font spec;")@/ ("e.g., type `I\font<same font id>=<substitute font name>'."); error @ @<Read and check...@>= @<Open |tfm←file| for input@>; @<Read the {\.{TFM}} size fields@>; @<Use size fields to allocate font information@>; @<Read the {\.{TFM}} header@>; @<Read character data@>; @<Read box dimensions@>; @<Read ligature/kern program@>; @<Read extensible character recipes@>; @<Read font parameters@>; @<Make final adjustments and |goto done|@> @ @<Open |tfm←file| for input@>= file←opened:=false; if aire="" then pack←file←name(nom,TEX←font←area,".tfm") else pack←file←name(nom,aire,".tfm"); if not b←open←in(tfm←file) then abort; file←opened:=true @ Note: A malformed \.{TFM} file might be shorter than it claims to be; thus |eof(tfm←file)| might be true when |read←font←info| refers to |tfm←file↑| or when it says |get(tfm←file)|. If such circumstances cause system error messages, you will have to defeat them somehow, for example by defining |fget| to be `\ignorespaces|begin get(tfm←file);| |if eof(tfm←file) then abort; end|\unskip'. @↑system dependencies@> @d fget==get(tfm←file) @d fbyte==tfm←file↑ @d read←sixteen(#)==begin #:=fbyte; if #>127 then abort; fget; #:=#*@'400+fbyte; end @d store←four←quarters(#)==begin fget; a:=fbyte; qw.b0:=qi(a); fget; b:=fbyte; qw.b1:=qi(b); fget; c:=fbyte; qw.b2:=qi(c); fget; d:=fbyte; qw.b3:=qi(d); #:=qw; end @ @<Read the {\.{TFM}} size fields@>= begin read←sixteen(lf); fget; read←sixteen(lh); fget; read←sixteen(bc); fget; read←sixteen(ec); if (bc>ec+1)or(ec>255) then abort; fget; read←sixteen(nw); fget; read←sixteen(nh); fget; read←sixteen(nd); fget; read←sixteen(ni); fget; read←sixteen(nl); fget; read←sixteen(nk); fget; read←sixteen(ne); fget; read←sixteen(np); if lf<>6+lh+(ec-bc+1)+nw+nh+nd+ni+nl+nk+ne+np then abort; end @ The preliminary settings of the index variables |char←base|, |width←base|, |lig←kern←base|, |kern←base|, and |exten←base| will be corrected later by subtracting |min←quarterword| from them; and we will subtract 1 from |param←base| too. It's best to forget about such anomalies until later. @<Use size fields to allocate font information@>= lf:=lf-6-lh; {|lf| words should be loaded into |font←info|} if np<7 then lf:=lf+7-np; {at least seven parameters will appear} if (font←ptr=font←max)or(fmem←ptr+lf>font←mem←size) then @<Apologize for not loading the font, |goto done|@>; f:=font←ptr+1; char←base[f]:=fmem←ptr-bc; width←base[f]:=char←base[f]+ec+1; height←base[f]:=width←base[f]+nw; depth←base[f]:=height←base[f]+nh; italic←base[f]:=depth←base[f]+nd; lig←kern←base[f]:=italic←base[f]+ni; kern←base[f]:=lig←kern←base[f]+nl; exten←base[f]:=kern←base[f]+nk; param←base[f]:=exten←base[f]+ne @ @<Apologize for not loading...@>= begin start←font←error←message; print(" not loaded: Not enough room left"); @.Font x=xx not loaded...@> help4("I'm afraid I won't be able to make use of this font,")@/ ("because my memory for character-size data is too small.")@/ ("If you're really stuck, ask a wizard to enlarge me.")@/ ("Or maybe try `I\font<same font id>=<name of loaded font>'."); error; goto done; end @ Only the first two words of the header are needed by \TeX82. @<Read the {\.{TFM}} header@>= begin if lh<2 then abort; store←four←quarters(font←check[f]); fget; read←sixteen(z); {this rejects a negative design size} fget; z:=z*@'400+fbyte; fget; z:=(z*@'20)+(fbyte div@'20); if z<unity then abort; while lh>2 do begin fget;fget;fget;fget;decr(lh); {ignore the rest of the header} end; font←dsize[f]:=z; if s<>-1000 then if s>=0 then z:=s else z:=xn←over←d(z,-s,1000); font←size[f]:=z; end @ @<Read character data@>= for k:=fmem←ptr to width←base[f]-1 do begin store←four←quarters(font←info[k].qqqq); if (a>=nw)or(b div @'20>=nh)or(b mod @'20>=nd)or (c div 4>=ni) then abort; case c mod 4 of lig←tag: if d>=nl then abort; ext←tag: if d>=ne then abort; list←tag: @<Check for charlist cycle@>; othercases do←nothing {|no←tag|} endcases; end @ We want to make sure that there is no cycle of characters linked together by |list←tag| entries, since such a cycle would get \TeX\ into an endless loop. If such a cycle exists, the routine here detects it when processing the largest character code in the cycle. @d check←byte←range(#)==begin if (#<bc)or(#>ec) then abort@+end @d current←character←being←worked←on==k+bc-fmem←ptr @<Check for charlist cycle@>= begin check←byte←range(d); while d<current←character←being←worked←on do begin qw:=char←info(f)(d); {N.B.: not |qi(d)|, since |char←base[f]| hasn't been adjusted yet} if char←tag(qw)<>list←tag then goto not←found; d:=qo(rem←byte(qw)); {next character on the list} end; if d=current←character←being←worked←on then abort; {yes, there's a cycle} not←found:end @ A |fix←word| whose four bytes are $(a,b,c,d)$ from left to right represents the number $$x=\left\{\vcenter{\halign{$#$,\hfil\qquad&if $#$\hfil\cr b\cdot2↑{-4}+c\cdot2↑{-12}+d\cdot2↑{-20}&a=0;\cr -16+b\cdot2↑{-4}+c\cdot2↑{-12}+d\cdot2↑{-20}&a=255.\cr}}\right.$$ (No other choices of |a| are allowed, since the magnitude of a number in design-size units must be less than 16.) We want to multiply this quantity by the integer~|z|, which is known to be less than $2↑{27}$. Let $\alpha=16z$. If $|z|<2↑{23}$, the individual multiplications $b\cdot z$, $c\cdot z$, $d\cdot z$ cannot overflow; otherwise we will divide |z| by 2, 4, 8, or 16, to obtain a multiplier less than $2↑{23}$, and we can compensate for this later. If |z| has thereby been replaced by $|z|↑\prime=|z|/2↑e$, let $\beta=2↑{4-e}$; we shall compute $$\lfloor(b+c\cdot2↑{-8}+d\cdot2↑{-16})\,z↑\prime/\beta\rfloor$$ if $a=0$, or the same quantity minus $\alpha$ if $a=255$. This calculation must be done exactly, in order to guarantee portability of \TeX\ between computers. @d store←scaled(#)==begin fget; a:=fbyte; fget; b:=fbyte; fget; c:=fbyte; fget; d:=fbyte;@/ sw:=(((((d*z)div@'400)+(c*z))div@'400)+(b*z))div beta; if a=0 then #:=sw@+else if a=255 then #:=sw-alpha@+else abort; end @<Read box dimensions@>= begin @<Replace |z| by $|z|↑\prime$ and compute $\alpha,\beta$@>; for k:=width←base[f] to lig←kern←base[f]-1 do store←scaled(font←info[k].sc); if font←info[width←base[f]].sc<>0 then abort; {\\{width}[0] must be zero} if font←info[height←base[f]].sc<>0 then abort; {\\{height}[0] must be zero} if font←info[depth←base[f]].sc<>0 then abort; {\\{depth}[0] must be zero} if font←info[italic←base[f]].sc<>0 then abort; {\\{italic}[0] must be zero} end @ @<Replace |z|...@>= begin alpha:=16*z; beta:=16; while z>=@'40000000 do begin z:=z div 2; beta:=beta div 2; end; end @ @<Read ligature/kern program@>= begin for k:=lig←kern←base[f] to kern←base[f]-1 do begin store←four←quarters(font←info[k].qqqq); check←byte←range(b); if c<128 then check←byte←range(d) {check ligature} else if d>=nk then abort; {check kern} end; if (nl>0)and(a<128) then abort; {check for stop bit on last command} for k:=kern←base[f] to exten←base[f]-1 do store←scaled(font←info[k].sc); end @ @<Read extensible character recipes@>= for k:=exten←base[f] to param←base[f]-1 do begin store←four←quarters(font←info[k].qqqq); if a<>0 then check←byte←range(a); if b<>0 then check←byte←range(b); if c<>0 then check←byte←range(c); check←byte←range(d); end @ We check to see that the \.{TFM} file doesn't end prematurely; but no error message is given for files having more than |lf| words. @<Read font parameters@>= begin for k:=1 to np do if k=1 then {the |slant| parameter is a pure number} begin fget; sw:=fbyte; if sw>127 then sw:=sw-256; fget; sw:=sw*@'400+fbyte; fget; sw:=sw*@'400+fbyte; fget; font←info[param←base[f]].sc:= (sw*@'20)+(fbyte div@'20); end else store←scaled(font←info[param←base[f]+k-1].sc); if eof(tfm←file) then abort; for k:=np+1 to 7 do font←info[param←base[f]+k-1].sc:=0; end @ Now to wrap it up, we have checked all the necessary things about the \.{TFM} file, and all we need to do is put the finishing touches on the data for the new font. @d adjust(#)==#[f]:=qo(#[f]) {correct for the excess |min←quarterword| that was added} @<Make final adjustments...@>= if np>=7 then font←params[f]:=np@+else font←params[f]:=7; hyphen←char[f]:=default←hyphen←char; skew←char[f]:=default←skew←char; font←name[f]:=nom; font←area[f]:=aire; font←bc[f]:=bc; font←ec[f]:=ec; font←glue[f]:=null; adjust(char←base); adjust(width←base); adjust(lig←kern←base); adjust(kern←base); adjust(exten←base); decr(param←base[f]); fmem←ptr:=fmem←ptr+lf; font←ptr:=f; g:=f; goto done @ Before we forget about the format of these tables, let's deal with two of \TeX's basic scanning routines related to font information. @<Declare procedures that scan font-related stuff@>= procedure scan←font←ident; var f:internal←font←number; @!m:halfword; begin @<Get the next non-blank non-call...@>; if cur←cmd=def←font then f:=cur←font else if cur←cmd=set←font then f:=cur←chr else if cur←cmd=def←family then begin m:=cur←chr; scan←four←bit←int; f:=equiv(m+cur←val); end else begin print←err("Missing font identifier"); @.Undefined font code@> help2("I was looking for a control sequence whose")@/ ("current meaning has been defined by \font."); back←error; f:=null←font; end; cur←val:=f; end; @ The following routine is used to implement `\.{\\fontdimen} |n| |f|'. The boolean parameter |writing| is set |true| if the calling program intends to change the parameter value. @<Declare procedures that scan font-related stuff@>= procedure find←font←dimen(@!writing:boolean); {sets |cur←val| to |font←info| location} var f:internal←font←number; @!n:integer; {the parameter number} begin scan←int; n:=cur←val; scan←font←ident; f:=cur←val; if n<=0 then cur←val:=fmem←ptr else begin if writing and(n<=space←shrink←code)and@| (n>=space←code)and(font←glue[f]<>null) then begin delete←glue←ref(font←glue[f]); font←glue[f]:=null; end; if n>font←params[f] then if f<font←ptr then cur←val:=fmem←ptr else @<Increase the number of parameters in the last font@> else cur←val:=n+param←base[f]; end; @<Issue an error message if |cur←val=fmem←ptr|@>; end; @ @<Issue an error message if |cur←val=fmem←ptr|@>= if cur←val=fmem←ptr then begin print←err("Font "); print←esc(font←id←text(f)); print(" has only "); print←int(font←params[f]); print(" fontdimen parameters"); @.Font x has only...@> help2("To increase the number of font parameters, you must")@/ ("use \fontdimen immediately after the \font is loaded."); error; end @ @<Increase the number of parameters...@>= begin repeat if fmem←ptr=font←mem←size then overflow("font memory",font←mem←size); @:TeX capacity exceeded font memory}{\quad font memory@> font←info[fmem←ptr].sc:=0; incr(fmem←ptr); incr(font←params[f]); until n=font←params[f]; cur←val:=fmem←ptr-1; {this equals |param←base[f]+font←params[f]|} end @ When \TeX\ wants to typeset a character that doesn't exist, the character node is not created; thus the output routine can assume that characters exist when it sees them. The following procedure prints a warning message unless the user has suppressed it. @p procedure char←warning(@!f:internal←font←number;@!c:eight←bits); begin if tracing←lost←chars>0 then begin begin←diagnostic; print←nl("Missing character: There is no "); @.Missing character@> print←ASCII(c); print(" in font "); print(font←name[f]); print←char("!"); end←diagnostic(false); end; end; @ Here is a function that returns a pointer to a character node for a given character in a given font. If that character doesn't exist, |null| is returned instead. @p function new←character(@!f:internal←font←number;@!c:eight←bits):pointer; label exit; var p:pointer; {newly allocated node} begin if (font←bc[f]<=c)and(font←ec[f]>=c) then if char←exists(char←info(f)(qi(c))) then begin p:=get←avail; font(p):=f; character(p):=qi(c); new←character:=p; return; end; char←warning(f,c); new←character:=null; exit:end; @* \[31] Device-independent file format. The most important output produced by a run of \TeX\ is the ``device independent'' (\.{DVI}) file that specifies where characters and rules are to appear on printed pages. The form of these files was designed by David R. Fuchs in 1979. Almost any reasonable typesetting device can be @↑Fuchs, David Raymond@> @:DVI←files}{\.{DVI} files@> driven by a program that takes \.{DVI} files as input, and dozens of such \.{DVI}-to-whatever programs have been written. Thus, it is possible to print the output of \TeX\ on many different kinds of equipment, using \TeX\ as a device-independent ``front end.'' A \.{DVI} file is a stream of 8-bit bytes, which may be regarded as a series of commands in a machine-like language. The first byte of each command is the operation code, and this code is followed by zero or more bytes that provide parameters to the command. The parameters themselves may consist of several consecutive bytes; for example, the `|set←rule|' command has two parameters, each of which is four bytes long. Parameters are usually regarded as nonnegative integers; but four-byte-long parameters, and shorter parameters that denote distances, can be either positive or negative. Such parameters are given in two's complement notation. For example, a two-byte-long distance parameter has a value between $-2↑{15}$ and $2↑{15}-1$. As in \.{TFM} files, numbers that occupy more than one byte position appear in BigEndian order. A \.{DVI} file consists of a ``preamble,'' followed by a sequence of one or more ``pages,'' followed by a ``postamble.'' The preamble is simply a |pre| command, with its parameters that define the dimensions used in the file; this must come first. Each ``page'' consists of a |bop| command, followed by any number of other commands that tell where characters are to be placed on a physical page, followed by an |eop| command. The pages appear in the order that \TeX\ generated them. If we ignore |nop| commands and \\{fnt\←def} commands (which are allowed between any two commands in the file), each |eop| command is immediately followed by a |bop| command, or by a |post| command; in the latter case, there are no more pages in the file, and the remaining bytes form the postamble. Further details about the postamble will be explained later. Some parameters in \.{DVI} commands are ``pointers.'' These are four-byte quantities that give the location number of some other byte in the file; the first byte is number~0, then comes number~1, and so on. For example, one of the parameters of a |bop| command points to the previous |bop|; this makes it feasible to read the pages in backwards order, in case the results are being directed to a device that stacks its output face up. Suppose the preamble of a \.{DVI} file occupies bytes 0 to 99. Now if the first page occupies bytes 100 to 999, say, and if the second page occupies bytes 1000 to 1999, then the |bop| that starts in byte 1000 points to 100 and the |bop| that starts in byte 2000 points to 1000. (The very first |bop|, i.e., the one that starts in byte 100, has a pointer of $-1$.) @ The \.{DVI} format is intended to be both compact and easily interpreted by a machine. Compactness is achieved by making most of the information implicit instead of explicit. When a \.{DVI}-reading program reads the commands for a page, it keeps track of several quantities: (a)~The current font |f| is an integer; this value is changed only by \\{fnt} and \\{fnt\←num} commands. (b)~The current position on the page is given by two numbers called the horizontal and vertical coordinates, |h| and |v|. Both coordinates are zero at the upper left corner of the page; moving to the right corresponds to increasing the horizontal coordinate, and moving down corresponds to increasing the vertical coordinate. Thus, the coordinates are essentially Cartesian, except that vertical directions are flipped; the Cartesian version of |(h,v)| would be |(h,-v)|. (c)~The current spacing amounts are given by four numbers |w|, |x|, |y|, and |z|, where |w| and~|x| are used for horizontal spacing and where |y| and~|z| are used for vertical spacing. (d)~There is a stack containing |(h,v,w,x,y,z)| values; the \.{DVI} commands |push| and |pop| are used to change the current level of operation. Note that the current font~|f| is not pushed and popped; the stack contains only information about positioning. The values of |h|, |v|, |w|, |x|, |y|, and |z| are signed integers having up to 32 bits, including the sign. Since they represent physical distances, there is a small unit of measurement such that increasing |h| by~1 means moving a certain tiny distance to the right. The actual unit of measurement is variable, as explained below; \TeX\ sets things up so that its \.{DVI} output is in sp units, i.e., scaled points, in agreement with all the |scaled| dimensions in \TeX's data structures. @ Here is a list of all the commands that may appear in a \.{DVI} file. Each command is specified by its symbolic name (e.g., |bop|), its opcode byte (e.g., 139), and its parameters (if any). The parameters are followed by a bracketed number telling how many bytes they occupy; for example, `|p[4]|' means that parameter |p| is four bytes long. \yskip\hang|set←char←0| 0. Typeset character number~0 from font~|f| such that the reference point of the character is at |(h,v)|. Then increase |h| by the width of that character. Note that a character may have zero or negative width, so one cannot be sure that |h| will advance after this command; but |h| usually does increase. \yskip\hang\\{set\←char\←1} through \\{set\←char\←127} (opcodes 1 to 127). Do the operations of |set←char←0|; but use the character whose number matches the opcode, instead of character~0. \yskip\hang|set1| 128 |c[1]|. Same as |set←char←0|, except that character number~|c| is typeset. \TeX82 uses this command for characters in the range |128<=c<256|. \yskip\hang|@!set2| 129 |c[2]|. Same as |set1|, except that |c|~is two bytes long, so it is in the range |0<=c<65536|. \TeX82 never uses this command, but it should come in handy for extensions of \TeX\ that deal with oriental languages. @↑oriental characters@>@↑Chinese characters@>@↑Japanese characters@> \yskip\hang|@!set3| 130 |c[3]|. Same as |set1|, except that |c|~is three bytes long, so it can be as large as $2↑{24}-1$. Not even the Chinese language has this many characters, but this command might prove useful in some yet unforeseen extension. \yskip\hang|@!set4| 131 |c[4]|. Same as |set1|, except that |c|~is four bytes long. Imagine that. \yskip\hang|set←rule| 132 |a[4]| |b[4]|. Typeset a solid black rectangle of height~|a| and width~|b|, with its bottom left corner at |(h,v)|. Then set |h:=h+b|. If either |a<=0| or |b<=0|, nothing should be typeset. Note that if |b<0|, the value of |h| will decrease even though nothing else happens. See below for details about how to typeset rules so that consistency with \MF\ is guaranteed. \yskip\hang|put1| 133 |c[1]|. Typeset character number~|c| from font~|f| such that the reference point of the character is at |(h,v)|. (The `put' commands are exactly like the `set' commands, except that they simply put out a character or a rule without moving the reference point afterwards.) \yskip\hang|@!put2| 134 |c[2]|. Same as |set2|, except that |h| is not changed. \yskip\hang|@!put3| 135 |c[3]|. Same as |set3|, except that |h| is not changed. \yskip\hang|@!put4| 136 |c[4]|. Same as |set4|, except that |h| is not changed. \yskip\hang|put←rule| 137 |a[4]| |b[4]|. Same as |set←rule|, except that |h| is not changed. \yskip\hang|nop| 138. No operation, do nothing. Any number of |nop|'s may occur between \.{DVI} commands, but a |nop| cannot be inserted between a command and its parameters or between two parameters. \yskip\hang|bop| 139 $c←0[4]$ $c←1[4]$ $\ldots$ $c←9[4]$ $p[4]$. Beginning of a page: Set |(h,v,w,x,y,z):=(0,0,0,0,0,0)| and set the stack empty. Set the current font |f| to an undefined value. The ten $c←i$ parameters hold the values of \.{\\count0} $\ldots$ \.{\\count9} in \TeX\ at the time \.{\\shipout} was invoked for this page; they can be used to identify pages, if a user wants to print only part of a \.{DVI} file. The parameter |p| points to the previous |bop| command in the file, where the first |bop| has $p=-1$. \yskip\hang|eop| 140. End of page: Print what you have read since the previous |bop|. At this point the stack should be empty. (The \.{DVI}-reading programs that drive most output devices will have kept a buffer of the material that appears on the page that has just ended. This material is largely, but not entirely, in order by |v| coordinate and (for fixed |v|) by |h|~coordinate; so it usually needs to be sorted into some order that is appropriate for the device in question.) \yskip\hang|push| 141. Push the current values of |(h,v,w,x,y,z)| onto the top of the stack; do not change any of these values. Note that |f| is not pushed. \yskip\hang|pop| 142. Pop the top six values off of the stack and assign them respectively to |(h,v,w,x,y,z)|. The number of pops should never exceed the number of pushes, since it would be highly embarrassing if the stack were empty at the time of a |pop| command. \yskip\hang|right1| 143 |b[1]|. Set |h:=h+b|, i.e., move right |b| units. The parameter is a signed number in two's complement notation, |-128<=b<128|; if |b<0|, the reference point actually moves left. \yskip\hang|right2| 144 |b[2]|. Same as |right1|, except that |b| is a two-byte quantity in the range |-32768<=b<32768|. \yskip\hang|right3| 145 |b[3]|. Same as |right1|, except that |b| is a three-byte quantity in the range |@t$-2↑{23}$@><=b<@t$2↑{23}$@>|. \yskip\hang|right4| 146 |b[4]|. Same as |right1|, except that |b| is a four-byte quantity in the range |@t$-2↑{31}$@><=b<@t$2↑{31}$@>|. \yskip\hang|w0| 147. Set |h:=h+w|; i.e., move right |w| units. With luck, this parameterless command will usually suffice, because the same kind of motion will occur several times in succession; the following commands explain how |w| gets particular values. \yskip\hang|w1| 148 |b[1]|. Set |w:=b| and |h:=h+b|. The value of |b| is a signed quantity in two's complement notation, |-128<=b<128|. This command changes the current |w|~spacing and moves right by |b|. \yskip\hang|@!w2| 149 |b[2]|. Same as |w1|, but |b| is two bytes long, |-32768<=b<32768|. \yskip\hang|@!w3| 150 |b[3]|. Same as |w1|, but |b| is three bytes long, |@t$-2↑{23}$@><=b<@t$2↑{23}$@>|. \yskip\hang|@!w4| 151 |b[4]|. Same as |w1|, but |b| is four bytes long, |@t$-2↑{31}$@><=b<@t$2↑{31}$@>|. \yskip\hang|x0| 152. Set |h:=h+x|; i.e., move right |x| units. The `|x|' commands are like the `|w|' commands except that they involve |x| instead of |w|. \yskip\hang|x1| 153 |b[1]|. Set |x:=b| and |h:=h+b|. The value of |b| is a signed quantity in two's complement notation, |-128<=b<128|. This command changes the current |x|~spacing and moves right by |b|. \yskip\hang|@!x2| 154 |b[2]|. Same as |x1|, but |b| is two bytes long, |-32768<=b<32768|. \yskip\hang|@!x3| 155 |b[3]|. Same as |x1|, but |b| is three bytes long, |@t$-2↑{23}$@><=b<@t$2↑{23}$@>|. \yskip\hang|@!x4| 156 |b[4]|. Same as |x1|, but |b| is four bytes long, |@t$-2↑{31}$@><=b<@t$2↑{31}$@>|. \yskip\hang|down1| 157 |a[1]|. Set |v:=v+a|, i.e., move down |a| units. The parameter is a signed number in two's complement notation, |-128<=a<128|; if |a<0|, the reference point actually moves up. \yskip\hang|@!down2| 158 |a[2]|. Same as |down1|, except that |a| is a two-byte quantity in the range |-32768<=a<32768|. \yskip\hang|@!down3| 159 |a[3]|. Same as |down1|, except that |a| is a three-byte quantity in the range |@t$-2↑{23}$@><=a<@t$2↑{23}$@>|. \yskip\hang|@!down4| 160 |a[4]|. Same as |down1|, except that |a| is a four-byte quantity in the range |@t$-2↑{31}$@><=a<@t$2↑{31}$@>|. \yskip\hang|y0| 161. Set |v:=v+y|; i.e., move down |y| units. With luck, this parameterless command will usually suffice, because the same kind of motion will occur several times in succession; the following commands explain how |y| gets particular values. \yskip\hang|y1| 162 |a[1]|. Set |y:=a| and |v:=v+a|. The value of |a| is a signed quantity in two's complement notation, |-128<=a<128|. This command changes the current |y|~spacing and moves down by |a|. \yskip\hang|@!y2| 163 |a[2]|. Same as |y1|, but |a| is two bytes long, |-32768<=a<32768|. \yskip\hang|@!y3| 164 |a[3]|. Same as |y1|, but |a| is three bytes long, |@t$-2↑{23}$@><=a<@t$2↑{23}$@>|. \yskip\hang|@!y4| 165 |a[4]|. Same as |y1|, but |a| is four bytes long, |@t$-2↑{31}$@><=a<@t$2↑{31}$@>|. \yskip\hang|z0| 166. Set |v:=v+z|; i.e., move down |z| units. The `|z|' commands are like the `|y|' commands except that they involve |z| instead of |y|. \yskip\hang|z1| 167 |a[1]|. Set |z:=a| and |v:=v+a|. The value of |a| is a signed quantity in two's complement notation, |-128<=a<128|. This command changes the current |z|~spacing and moves down by |a|. \yskip\hang|@!z2| 168 |a[2]|. Same as |z1|, but |a| is two bytes long, |-32768<=a<32768|. \yskip\hang|@!z3| 169 |a[3]|. Same as |z1|, but |a| is three bytes long, |@t$-2↑{23}$@><=a<@t$2↑{23}$@>|. \yskip\hang|@!z4| 170 |a[4]|. Same as |z1|, but |a| is four bytes long, |@t$-2↑{31}$@><=a<@t$2↑{31}$@>|. \yskip\hang|fnt←num←0| 171. Set |f:=0|. Font 0 must previously have been defined by a \\{fnt\←def} instruction, as explained below. \yskip\hang\\{fnt\←num\←1} through \\{fnt\←num\←63} (opcodes 172 to 234). Set |f:=1|, \dots, \hbox{|f:=63|}, respectively. \yskip\hang|fnt1| 235 |k[1]|. Set |f:=k|. \TeX82 uses this command for font numbers in the range |64<=k<256|. \yskip\hang|@!fnt2| 236 |k[2]|. Same as |fnt1|, except that |k|~is two bytes long, so it is in the range |0<=k<65536|. \TeX82 never generates this command, but large font numbers may prove useful for specifications of color or texture, or they may be used for special fonts that have fixed numbers in some external coding scheme. \yskip\hang|@!fnt3| 237 |k[3]|. Same as |fnt1|, except that |k|~is three bytes long, so it can be as large as $2↑{24}-1$. \yskip\hang|@!fnt4| 238 |k[4]|. Same as |fnt1|, except that |k|~is four bytes long; this is for the really big font numbers (and for the negative ones). \yskip\hang|xxx1| 239 |k[1]| |x[k]|. This command is undefined in general; it functions as a $(k+2)$-byte |nop| unless special \.{DVI}-reading programs are being used. \TeX82 generates |xxx1| when a short enough \.{\\special} appears, setting |k| to the number of bytes being sent. It is recommended that |x| be a string having the form of a keyword followed by possible parameters relevant to that keyword. \yskip\hang|@!xxx2| 240 |k[2]| |x[k]|. Like |xxx1|, but |0<=k<65536|. \yskip\hang|@!xxx3| 241 |k[3]| |x[k]|. Like |xxx1|, but |0<=k<@t$2↑{24}$@>|. \yskip\hang|xxx4| 242 |k[4]| |x[k]|. Like |xxx1|, but |k| can be ridiculously large. \TeX82 uses |xxx4| when sending a string of length 256 or more. \yskip\hang|fnt←def1| 243 |k[1]| |c[4]| |s[4]| |d[4]| |a[1]| |l[1]| |n[a+l]|. Define font |k|, where |0<=k<256|; font definitions will be explained shortly. \yskip\hang|@!fnt←def2| 244 |k[2]| |c[4]| |s[4]| |d[4]| |a[1]| |l[1]| |n[a+l]|. Define font |k|, where |0<=k<65536|. \yskip\hang|@!fnt←def3| 245 |k[3]| |c[4]| |s[4]| |d[4]| |a[1]| |l[1]| |n[a+l]|. Define font |k|, where |0<=k<@t$2↑{24}$@>|. \yskip\hang|@!fnt←def4| 246 |k[4]| |c[4]| |s[4]| |d[4]| |a[1]| |l[1]| |n[a+l]|. Define font |k|, where |@t$-2↑{31}$@><=k<@t$2↑{31}$@>|. \yskip\hang|pre| 247 |i[1]| |num[4]| |den[4]| |mag[4]| |k[1]| |x[k]|. Beginning of the preamble; this must come at the very beginning of the file. Parameters |i|, |num|, |den|, |mag|, |k|, and |x| are explained below. \yskip\hang|post| 248. Beginning of the postamble, see below. \yskip\hang|post←post| 249. Ending of the postamble, see below. \yskip\noindent Commands 250--255 are undefined at the present time. @ @d set←char←0=0 {typeset character 0 and move right} @d set1=128 {typeset a character and move right} @d set←rule=132 {typeset a rule and move right} @d put1=133 {typeset a character} @d put←rule=137 {typeset a rule} @d nop=138 {no operation} @d bop=139 {beginning of page} @d eop=140 {ending of page} @d push=141 {save the current positions} @d pop=142 {restore previous positions} @d right1=143 {move right} @d w0=147 {move right by |w|} @d w1=148 {move right and set |w|} @d x0=152 {move right by |x|} @d x1=153 {move right and set |x|} @d down1=157 {move down} @d y0=161 {move down by |y|} @d y1=162 {move down and set |y|} @d z0=166 {move down by |z|} @d z1=167 {move down and set |z|} @d fnt←num←0=171 {set current font to 0} @d fnt1=235 {set current font} @d xxx1=239 {extension to \.{DVI} primitives} @d xxx4=242 {potentially long extension to \.{DVI} primitives} @d fnt←def1=243 {define the meaning of a font number} @d pre=247 {preamble} @d post=248 {postamble beginning} @d post←post=249 {postamble ending} @ The preamble contains basic information about the file as a whole. As stated above, there are six parameters: $$\hbox{|@!i[1]| |@!num[4]| |@!den[4]| |@!mag[4]| |@!k[1]| |@!x[k]|.}$$ The |i| byte identifies \.{DVI} format; currently this byte is always set to~2. (Some day we will set |i=3|, when \.{DVI} format makes another incompatible change---perhaps in 1992.) The next two parameters, |num| and |den|, are positive integers that define the units of measurement; they are the numerator and denominator of a fraction by which all dimensions in the \.{DVI} file could be multiplied in order to get lengths in units of $10↑{-7}$ meters. Since $\rm 7227{pt} = 254{cm}$, and since \TeX\ works with scaled points where there are $2↑{16}$ sp in a point, \TeX\ sets $|num|/|den|=(254\cdot10↑5)/(7227\cdot2↑{16})=25400000/473628672$. @↑sp@> The |mag| parameter is what \TeX\ calls \.{\\mag}, i.e., 1000 times the desired magnification. The actual fraction by which dimensions are multiplied is therefore $|mag|\cdot|num|/1000|den|$. Note that if a \TeX\ source document does not call for any `\.{true}' dimensions, and if you change it only by specifying a different \.{\\mag} setting, the \.{DVI} file that \TeX\ creates will be completely unchanged except for the value of |mag| in the preamble and postamble. (Fancy \.{DVI}-reading programs allow users to override the |mag|~setting when a \.{DVI} file is being printed.) Finally, |k| and |x| allow the \.{DVI} writer to include a comment, which is not interpreted further. The length of comment |x| is |k|, where |0<=k<256|. @d id←byte=2 {identifies the kind of \.{DVI} files described here} @ Font definitions for a given font number |k| contain further parameters $$\hbox{|c[4]| |s[4]| |d[4]| |a[1]| |l[1]| |n[a+l]|.}$$ The four-byte value |c| is the check sum that \TeX\ found in the \.{TFM} file for this font; |c| should match the check sum of the font found by programs that read this \.{DVI} file. @↑check sum@> Parameter |s| contains a fixed-point scale factor that is applied to the character widths in font |k|; font dimensions in \.{TFM} files and other font files are relative to this quantity, which is called the ``at size'' elsewhere in this documentation. The value of |s| is always positive and less than $2↑{27}$. It is given in the same units as the other \.{DVI} dimensions, i.e., in sp when \TeX82 has made the file. Parameter |d| is similar to |s|; it is the ``design size,'' and it is given in \.{DVI} units that have not been corrected for the magnification~|mag| found in the preamble. Thus, font |k| is to be used at $|mag|\cdot s/1000d$ times its normal size. The remaining part of a font definition gives the external name of the font, which is an ASCII string of length |a+l|. The number |a| is the length of the ``area'' or directory, and |l| is the length of the font name itself; the standard local system font area is supposed to be used when |a=0|. The |n| field contains the area in its first |a| bytes. Font definitions must appear before the first use of a particular font number. Once font |k| is defined, it must not be defined again; however, we shall see below that font definitions appear in the postamble as well as in the pages, so in this sense each font number is defined exactly twice, if at all. Like |nop| commands and \\{xxx} commands, font definitions can appear before the first |bop|, or between an |eop| and a |bop|. @ Sometimes it is desirable to make horizontal or vertical rules line up precisely with certain features in characters of a font. It is possible to guarantee the correct matching between \.{DVI} output and the characters generated by \MF\ by adhering to the following principles: (1)~The \MF\ characters should be positioned so that a bottom edge or left edge that is supposed to line up with the bottom or left edge of a rule appears at the reference point, i.e., in row~0 and column~0 of the \MF\ raster. This ensures that the position of the rule will not be rounded differently when the pixel size is not a perfect multiple of the units of measurement in the \.{DVI} file. (2)~A typeset rule of height $a>0$ and width $b>0$ should be equivalent to a \MF-generated character having black pixels in precisely those raster positions whose \MF\ coordinates satisfy |0<=x<@t$\alpha$@>b| and |0<=y<@t$\alpha$@>a|, where $\alpha$ is the number of pixels per \.{DVI} unit. @:METAFONT}{\MF@> @↑alignment of rules with characters@> @↑rules aligning with characters@> @ The last page in a \.{DVI} file is followed by `|post|'; this command introduces the postamble, which summarizes important facts that \TeX\ has accumulated about the file, making it possible to print subsets of the data with reasonable efficiency. The postamble has the form $$\vbox{\halign{\hbox{#\hfil}\cr |post| |p[4]| |num[4]| |den[4]| |mag[4]| |l[4]| |u[4]| |s[2]| |t[2]|\cr $\langle\,$font definitions$\,\rangle$\cr |post←post| |q[4]| |i[1]| 223's$[{\G}4]$\cr}}$$ Here |p| is a pointer to the final |bop| in the file. The next three parameters, |num|, |den|, and |mag|, are duplicates of the quantities that appeared in the preamble. Parameters |l| and |u| give respectively the height-plus-depth of the tallest page and the width of the widest page, in the same units as other dimensions of the file. These numbers might be used by a \.{DVI}-reading program to position individual ``pages'' on large sheets of film or paper; however, the standard convention for output on normal size paper is to position each page so that the upper left-hand corner is exactly one inch from the left and the top. Experience has shown that it is unwise to design \.{DVI}-to-printer software that attempts cleverly to center the output; a fixed position of the upper left corner is easiest for users to understand and to work with. Therefore |l| and~|u| are often ignored. Parameter |s| is the maximum stack depth (i.e., the largest excess of |push| commands over |pop| commands) needed to process this file. Then comes |t|, the total number of pages (|bop| commands) present. The postamble continues with font definitions, which are any number of \\{fnt\←def} commands as described above, possibly interspersed with |nop| commands. Each font number that is used in the \.{DVI} file must be defined exactly twice: Once before it is first selected by a \\{fnt} command, and once in the postamble. @ The last part of the postamble, following the |post←post| byte that signifies the end of the font definitions, contains |q|, a pointer to the |post| command that started the postamble. An identification byte, |i|, comes next; this currently equals~2, as in the preamble. The |i| byte is followed by four or more bytes that are all equal to the decimal number 223 (i.e., @'337 in octal). \TeX\ puts out four to seven of these trailing bytes, until the total length of the file is a multiple of four bytes, since this works out best on machines that pack four bytes per word; but any number of 223's is allowed, as long as there are at least four of them. In effect, 223 is a sort of signature that is added at the very end. @↑Fuchs, David Raymond@> This curious way to finish off a \.{DVI} file makes it feasible for \.{DVI}-reading programs to find the postamble first, on most computers, even though \TeX\ wants to write the postamble last. Most operating systems permit random access to individual words or bytes of a file, so the \.{DVI} reader can start at the end and skip backwards over the 223's until finding the identification byte. Then it can back up four bytes, read |q|, and move to byte |q| of the file. This byte should, of course, contain the value 248 (|post|); now the postamble can be read, so the \.{DVI} reader can discover all the information needed for typesetting the pages. Note that it is also possible to skip through the \.{DVI} file at reasonably high speed to locate a particular page, if that proves desirable. This saves a lot of time, since \.{DVI} files used in production jobs tend to be large. Unfortunately, however, standard \PASCAL\ does not include the ability to @↑system dependencies@> access a random position in a file, or even to determine the length of a file. Almost all systems nowadays provide the necessary capabilities, so \.{DVI} format has been designed to work most efficiently with modern operating systems. But if \.{DVI} files have to be processed under the restrictions of standard \PASCAL, one can simply read them from front to back, since the necessary header information is present in the preamble and in the font definitions. (The |l| and |u| and |s| and |t| parameters, which appear only in the postamble, are ``frills'' that are handy but not absolutely necessary.) @* \[32] Shipping pages out. After considering \TeX's eyes and stomach, we come now to the bowels. @↑bowels@> The |ship←out| procedure is given a pointer to a box; its mission is to describe that box in \.{DVI} form, outputting a ``page'' to |dvi←file|. The \.{DVI} coordinates $(h,v)=(0,0)$ should correspond to the upper left corner of the box being shipped. Since boxes can be inside of boxes inside of boxes, the main work of |ship←out| is done by two mutually recursive routines, |hlist←out| and |vlist←out|, which traverse the hlists and vlists inside of horizontal and vertical boxes. As individual pages are being processed, we need to accumulate information about the entire set of pages, since such statistics must be reported in the postamble. The global variables |total←pages|, |max←v|, |max←h|, |max←push|, and |last←bop| are used to record this information. The variable |doing←leaders| is |true| while leaders are being output. The variable |dead←cycles| contains the number of times an output routine has been initiated since the last |ship←out|. A few additional global variables are also defined here for use in |vlist←out| and |hlist←out|. They could have been local variables, but that would waste stack space when boxes are deeply nested, since the values of these variables are not needed during recursive calls. @↑recursion@> @<Glob...@>= @!total←pages:integer; {the number of pages that have been shipped out} @!max←v:scaled; {maximum height-plus-depth of pages shipped so far} @!max←h:scaled; {maximum width of pages shipped so far} @!max←push:integer; {deepest nesting of |push| commands encountered so far} @!last←bop:integer; {location of previous |bop| in the \.{DVI} output} @!dead←cycles:integer; {recent outputs that didn't ship anything out} @!doing←leaders:boolean; {are we inside a leader box?} @# @!c,@!f:quarterword; {character and font in current |char←node|} @!rule←ht,@!rule←dp,@!rule←wd:scaled; {size of current rule being output} @!g:pointer; {current glue specification} @!lq,@!lr:integer; {quantities used in calculations for leaders} @ @<Set init...@>= total←pages:=0; max←v:=0; max←h:=0; max←push:=0; last←bop:=-1; doing←leaders:=false; dead←cycles:=0; @ The \.{DVI} bytes are output to a buffer instead of being written directly to the output file. This makes it possible to reduce the overhead of subroutine calls, thereby measurably speeding up the computation, since output of \.{DVI} bytes is part of \TeX's inner loop. And it has another advantage as well, since we can change instructions in the buffer in order to make the output more compact. For example, a `|down2|' command can be changed to a `|y2|', thereby making a subsequent `|y0|' command possible, saving two bytes. The output buffer is divided into two parts of equal size; the bytes found in |dvi←buf[0..half←buf-1]| constitute the first half, and those in |dvi←buf[half←buf..dvi←buf←size-1]| constitute the second. The global variable |dvi←ptr| points to the position that will receive the next output byte. When |dvi←ptr| reaches |dvi←limit|, which is always equal to one of the two values |half←buf| or |dvi←buf←size|, the half buffer that is about to be invaded next is sent to the output and |dvi←limit| is changed to its other value. Thus, there is always at least a half buffer's worth of information present, except at the very beginning of the job. Bytes of the \.{DVI} file are numbered sequentially starting with 0; the next byte to be generated will be number |dvi←offset+dvi←ptr|. A byte is present in the buffer only if its number is |>=dvi←gone|. @<Types...@>= @!dvi←index=0..dvi←buf←size; {an index into the output buffer} @ Some systems may find it more efficient to make |dvi←buf| a |packed| array, since output of four bytes at once may be facilitated. @↑system dependencies@> @<Glob...@>= @!dvi←buf:array[dvi←index] of eight←bits; {buffer for \.{DVI} output} @!half←buf:dvi←index; {half of |dvi←buf←size|} @!dvi←limit:dvi←index; {end of the current half buffer} @!dvi←ptr:dvi←index; {the next available buffer address} @!dvi←offset:integer; {|dvi←buf←size| times the number of times the output buffer has been fully emptied} @!dvi←gone:integer; {the number of bytes already output to |dvi←file|} @ Initially the buffer is all in one piece; we will output half of it only after it first fills up. @<Set init...@>= half←buf:=dvi←buf←size div 2; dvi←limit:=dvi←buf←size; dvi←ptr:=0; dvi←offset:=0; dvi←gone:=0; @ The actual output of |dvi←buf[a..b]| to |dvi←file| is performed by calling |write←dvi(a,b)|. For best results, this procedure should be optimized to run as fast as possible on each particular system, since it is part of \TeX's inner loop. It is safe to assume that |a| and |b+1| will both be multiples of 4 when |write←dvi(a,b)| is called; therefore it is possible on many machines to use efficient methods to pack four bytes per word and to output an array of words with one system call. @↑system dependencies@> @↑inner loop@> @↑defecation@> @p procedure write←dvi(@!a,@!b:dvi←index); var k:dvi←index; begin for k:=a to b do write(dvi←file,dvi←buf[k]); end; @ To put a byte in the buffer without paying the cost of invoking a procedure each time, we use the macro |dvi←out|. @d dvi←out(#)==@+begin dvi←buf[dvi←ptr]:=#; incr(dvi←ptr); if dvi←ptr=dvi←limit then dvi←swap; end @p procedure dvi←swap; {outputs half of the buffer} begin if dvi←limit=dvi←buf←size then begin write←dvi(0,half←buf-1); dvi←limit:=half←buf; dvi←offset:=dvi←offset+dvi←buf←size; dvi←ptr:=0; end else begin write←dvi(half←buf,dvi←buf←size-1); dvi←limit:=dvi←buf←size; end; dvi←gone:=dvi←gone+half←buf; end; @ Here is how we clean out the buffer when \TeX\ is all through; |dvi←ptr| will be a multiple of~4. @<Empty the last bytes out of |dvi←buf|@>= if dvi←limit=half←buf then write←dvi(half←buf,dvi←buf←size-1); if dvi←ptr>0 then write←dvi(0,dvi←ptr-1) @ The |dvi←four| procedure outputs four bytes in two's complement notation, without risking arithmetic overflow. @p procedure dvi←four(@!x:integer); begin if x>=0 then dvi←out(x div @'100000000) else begin x:=x+@'10000000000; x:=x+@'10000000000; dvi←out((x div @'100000000) + 128); end; x:=x mod @'100000000; dvi←out(x div @'200000); x:=x mod @'200000; dvi←out(x div @'400); dvi←out(x mod @'400); end; @ A mild optimization of the output is performed by the |dvi←pop| routine, which issues a |pop| unless it is possible to cancel a `|push| |pop|' pair. The parameter to |dvi←pop| is the byte address following the old |push| that matches the new |pop|. @p procedure dvi←pop(@!l:integer); begin if (l=dvi←offset+dvi←ptr)and(dvi←ptr>0) then decr(dvi←ptr) else dvi←out(pop); end; @ Here's a procedure that outputs a font definition. Since \TeX82 uses at most 256 different fonts per job, |fnt←def1| is always used as the command code. @p procedure dvi←font←def(@!f:internal←font←number); var k:pool←pointer; {index into |str←pool|} begin dvi←out(fnt←def1); dvi←out(f-font←base-1);@/ dvi←out(qo(font←check[f].b0)); dvi←out(qo(font←check[f].b1)); dvi←out(qo(font←check[f].b2)); dvi←out(qo(font←check[f].b3));@/ dvi←four(font←size[f]); dvi←four(font←dsize[f]);@/ dvi←out(length(font←area[f])); dvi←out(length(font←name[f])); @<Output the font name whose internal number is |f|@>; end; @ @<Output the font name whose internal number is |f|@>= for k:=str←start[font←area[f]] to str←start[font←area[f]+1]-1 do dvi←out(str←pool[k]); for k:=str←start[font←name[f]] to str←start[font←name[f]+1]-1 do dvi←out(str←pool[k]) @ Versions of \TeX\ intended for small computers might well choose to omit the ideas in the next few parts of this program, since it is not really necessary to optimize the \.{DVI} code by making use of the |w0|, |x0|, |y0|, and |z0| commands. Furthermore, the algorithm that we are about to describe does not pretend to give an optimum reduction in the length of the \.{DVI} code; after all, speed is more important than compactness. But the method is surprisingly effective, and it takes comparatively little time. We can best understand the basic idea by first considering a simpler problem that has the same essential characteristics. Given a sequence of digits, say $3\,1\,4\,1\,5\,9\,2\,6\,5\,3\,5\,8\,9$, we want to assign subscripts $d$, $y$, or $z$ to each digit so as to maximize the number of ``$y$-hits'' and ``$z$-hits''; a $y$-hit is an instance of two appearances of the same digit with the subscript $y$, where no $y$'s intervene between the two appearances, and a $z$-hit is defined similarly. For example, the sequence above could be decorated with subscripts as follows: $$3←z\,1←y\,4←d\,1←y\,5←y\,9←d\,2←d\,6←d\,5←y\,3←z\,5←y\,8←d\,9←d.$$ There are three $y$-hits ($1←y\ldots1←y$ and $5←y\ldots5←y\ldots5←y$) and one $z$-hit ($3←z\ldots3←z$); there are no $d$-hits, since the two appearances of $9←d$ have $d$'s between them, but we don't count $d$-hits so it doesn't matter how many there are. These subscripts are analogous to the \.{DVI} commands called \\{down}, $y$, and $z$, and the digits are analogous to different amounts of vertical motion; a $y$-hit or $z$-hit corresponds to the opportunity to use the one-byte commands |y0| or |z0| in a \.{DVI} file. \TeX's method of assigning subscripts works like this: Append a new digit, say $\delta$, to the right of the sequence. Now look back through the sequence until one of the following things happens: (a)~You see $\delta←y$ or $\delta←z$, and this was the first time you encountered a $y$ or $z$ subscript, respectively. Then assign $y$ or $z$ to the new $\delta$; you have scored a hit. (b)~You see $\delta←d$, and no $y$ subscripts have been encountered so far during this search. Then change the previous $\delta←d$ to $\delta←y$ (this corresponds to changing a command in the output buffer), and assign $y$ to the new $\delta$; it's another hit. (c)~You see $\delta←d$, and a $y$ subscript has been seen but not a $z$. Change the previous $\delta←d$ to $\delta←z$ and assign $z$ to the new $\delta$. (d)~You encounter both $y$ and $z$ subscripts before encountering a suitable $\delta$, or you scan all the way to the front of the sequence. Assign $d$ to the new $\delta$; this assignment may be changed later. The subscripts $3←z\,1←y\,4←d\ldots\,$ in the example above were, in fact, produced by this procedure, as the reader can verify. (Go ahead and try it.) @ In order to implement such an idea, \TeX\ maintains a stack of pointers to the \\{down}, $y$, and $z$ commands that have been generated for the current page. And there is a similar stack for \\{right}, |w|, and |x| commands. These stacks are called the down stack and right stack, and their top elements are maintained in the variables |down←ptr| and |right←ptr|. Each entry in these stacks contains four fields: The |width| field is the amount of motion down or to the right; the |location| field is the byte number of the \.{DVI} command in question (including the appropriate |dvi←offset|); the |link| field points to the next item below this one on the stack; and the |info| field encodes the options for possible change in the \.{DVI} command. @d movement←node←size=3 {number of words per entry in the down and right stacks} @d location(#)==mem[#+2].int {\.{DVI} byte number for a movement command} @<Glob...@>= @!down←ptr,@!right←ptr:pointer; {heads of the down and right stacks} @ @<Set init...@>= down←ptr:=null; right←ptr:=null; @ Here is a subroutine that produces a \.{DVI} command for some specified downward or rightward motion. It has two parameters: |w| is the amount of motion, and |o| is either |down1| or |right1|. We use the fact that the command codes have convenient arithmetic properties: |y1-down1=w1-right1| and |z1-down1=x1-right1|. @p procedure movement(@!w:scaled;@!o:eight←bits); label exit,found,not←found,2,1; var mstate:small←number; {have we seen a |y| or |z|?} @!p,@!q:pointer; {current and top nodes on the stack} @!k:integer; {index into |dvi←buf|, modulo |dvi←buf←size|} begin q:=get←node(movement←node←size); {new node for the top of the stack} width(q):=w; location(q):=dvi←offset+dvi←ptr; if o=down1 then begin link(q):=down←ptr; down←ptr:=q; end else begin link(q):=right←ptr; right←ptr:=q; end; @<Look at the other stack entries until deciding what sort of \.{DVI} command to generate; |goto found| if node |p| is a ``hit''@>; @<Generate a |down| or |right| command for |w| and |return|@>; found: @<Generate a |y0| or |z0| command in order to reuse a previous appearance of~|w|@>; exit:end; @ The |info| fields in the entries of the down stack or the right stack have six possible settings: |y←here| or |z←here| mean that the \.{DVI} command refers to |y| or |z|, respectively (or to |w| or |x|, in the case of horizontal motion); |yz←OK| means that the \.{DVI} command is \\{down} (or \\{right}) but can be changed to either |y| or |z| (or to either |w| or |x|); |y←OK| means that it is \\{down} and can be changed to |y| but not |z|; |z←OK| is similar; and |d←fixed| means it must stay \\{down}. The four settings |yz←OK|, |y←OK|, |z←OK|, |d←fixed| would not need to be distinguished from each other if we were simply solving the digit-subscripting problem mentioned above. But in \TeX's case there is a complication because of the nested structure of |push| and |pop| commands. Suppose we add parentheses to the digit-subscripting problem, redefining hits so that $\delta←y\ldots \delta←y$ is a hit if all $y$'s between the $\delta$'s are enclosed in properly nested parentheses, and if the parenthesis level of the right-hand $\delta←y$ is deeper than or equal to that of the left-hand one. Thus, `(' and `)' correspond to `|push|' and `|pop|'. Now if we want to assign a subscript to the final 1 in the sequence $$2←y\,7←d\,1←d\,(\,8←z\,2←y\,8←z\,)\,1$$ we cannot change the previous $1←d$ to $1←y$, since that would invalidate the $2←y\ldots2←y$ hit. But we can change it to $1←z$, scoring a hit since the intervening $8←z$'s are enclosed in parentheses. The program below removes movement nodes that are introduced after a |push|, before it outputs the corresponding |pop|. @d y←here=1 {|info| when the movement entry points to a |y| command} @d z←here=2 {|info| when the movement entry points to a |z| command} @d yz←OK=3 {|info| corresponding to an unconstrained \\{down} command} @d y←OK=4 {|info| corresponding to a \\{down} that can't become a |z|} @d z←OK=5 {|info| corresponding to a \\{down} that can't become a |y|} @d d←fixed=6 {|info| corresponding to a \\{down} that can't change} @ When the |movement| procedure gets to the label |found|, the value of |info(p)| will be either |y←here| or |z←here|. If it is, say, |y←here|, the procedure generates a |y0| command (or a |w0| command), and marks all |info| fields between |q| and |p| so that |y| is not OK in that range. @<Generate a |y0| or |z0| command...@>= info(q):=info(p); if info(q)=y←here then begin dvi←out(o+y0-down1); {|y0| or |w0|} while link(q)<>p do begin q:=link(q); case info(q) of yz←OK: info(q):=z←OK; y←OK: info(q):=d←fixed; othercases do←nothing endcases; end; end else begin dvi←out(o+z0-down1); {|z0| or |x0|} while link(q)<>p do begin q:=link(q); case info(q) of yz←OK: info(q):=y←OK; z←OK: info(q):=d←fixed; othercases do←nothing endcases; end; end @ @<Generate a |down| or |right|...@>= info(q):=yz←OK; if abs(w)>=@'40000000 then begin dvi←out(o+3); {|down4| or |right4|} dvi←four(w); return; end; if abs(w)>=@'100000 then begin dvi←out(o+2); {|down3| or |right3|} if w<0 then w:=w+@'100000000; dvi←out(w div @'200000); w:=w mod @'200000; goto 2; end; if abs(w)>=@'200 then begin dvi←out(o+1); {|down2| or |right2|} if w<0 then w:=w+@'200000; goto 2; end; dvi←out(o); {|down1| or |right1|} if w<0 then w:=w+@'400; goto 1; 2: dvi←out(w div @'400); 1: dvi←out(w mod @'400); return @ As we search through the stack, we are in one of three states, |y←seen|, |z←seen|, or |none←seen|, depending on whether we have encountered |y←here| or |z←here| nodes. These states are encoded as multiples of 6, so that they can be added to the |info| fields for quick decision-making. @↑inner loop@> @d none←seen=0 {no |y←here| or |z←here| nodes have been encountered yet} @d y←seen=6 {we have seen |y←here| but not |z←here|} @d z←seen=12 {we have seen |z←here| but not |y←here|} @<Look at the other stack entries until deciding...@>= p:=link(q); mstate:=none←seen; while p<>null do begin if width(p)=w then @<Consider a node with matching width; |goto found| if it's a hit@> else case mstate+info(p) of none←seen+y←here: mstate:=y←seen; none←seen+z←here: mstate:=z←seen; y←seen+z←here,z←seen+y←here: goto not←found; othercases do←nothing endcases; p:=link(p); end; not←found: @ We might find a valid hit in a |y| or |z| byte that is already gone from the buffer. But we can't change bytes that are gone forever; ``the moving finger writes, $\ldots\,\,$.'' @<Consider a node with matching width...@>= case mstate+info(p) of none←seen+yz←OK,none←seen+y←OK,z←seen+yz←OK,z←seen+y←OK:@t@>@;@/ if location(p)<dvi←gone then goto not←found else @<Change buffered instruction to |y| or |w| and |goto found|@>; none←seen+z←OK,y←seen+yz←OK,y←seen+z←OK:@t@>@;@/ if location(p)<dvi←gone then goto not←found else @<Change buffered instruction to |z| or |x| and |goto found|@>; none←seen+y←here,none←seen+z←here,y←seen+z←here,z←seen+y←here: goto found; othercases do←nothing endcases @ @<Change buffered instruction to |y| or |w| and |goto found|@>= begin k:=location(p)-dvi←offset; if k<0 then k:=k+dvi←buf←size; dvi←buf[k]:=dvi←buf[k]+y1-down1; info(p):=y←here; goto found; end @ @<Change buffered instruction to |z| or |x| and |goto found|@>= begin k:=location(p)-dvi←offset; if k<0 then k:=k+dvi←buf←size; dvi←buf[k]:=dvi←buf[k]+z1-down1; info(p):=z←here; goto found; end @ In case you are wondering when all the movement nodes are removed from \TeX's memory, the answer is that they are recycled just before |hlist←out| and |vlist←out| finish outputting a box. This restores the down and right stacks to the state they were in before the box was output, except that some |info|'s may have become more restrictive. @p procedure prune←movements(@!l:integer); {delete movement nodes with |location>=l|} label done,exit; var p:pointer; {node being deleted} begin while down←ptr<>null do begin if location(down←ptr)<l then goto done; p:=down←ptr; down←ptr:=link(p); free←node(p,movement←node←size); end; done: while right←ptr<>null do begin if location(right←ptr)<l then return; p:=right←ptr; right←ptr:=link(p); free←node(p,movement←node←size); end; exit:end; @ The actual distances by which we want to move might be computed as the sum of several separate movements. For example, there might be several glue nodes in succession, or we might want to move right by the width of some box plus some amount of glue. More importantly, the baselineskip distances are computed in terms of glue together with the depth and height of adjacent boxes, and we want the \.{DVI} file to lump these three quantities together into a single motion. Therefore \TeX\ maintains two pairs of global variables: |dvi←h| and |dvi←v| are the |h| and |v| coordinates corresponding to the commands actually output to the \.{DVI} file, while |cur←h| and |cur←v| are the coordinates corresponding to the current state of the output routines. Coordinate changes will accumulate in |cur←h| and |cur←v| without being reflected in the output, until such a change becomes necessary or desirable; we can call the |movement| procedure whenever we want to make |dvi←h=cur←h| or |dvi←v=cur←v|. The current font reflected in the \.{DVI} output is called |dvi←f|; there is no need for a `\\{cur\←f}' variable. The depth of nesting of |hlist←out| and |vlist←out| is called |cur←s|; this is essentially the depth of |push| commands in the \.{DVI} output. @d synch←h==if cur←h<>dvi←h then begin movement(cur←h-dvi←h,right1); dvi←h:=cur←h; end @d synch←v==if cur←v<>dvi←v then begin movement(cur←v-dvi←v,down1); dvi←v:=cur←v; end @<Glob...@>= @!dvi←h,@!dvi←v:scaled; {a \.{DVI} reader program thinks we are here} @!cur←h,@!cur←v:scaled; {\TeX\ thinks we are here} @!dvi←f:internal←font←number; {the current font} @!cur←s:integer; {current depth of output box nesting} @ @<Initialize variables as |ship←out| begins@>= dvi←h:=0; dvi←v:=0; cur←h:=h←offset; dvi←f:=null←font; cur←s:=-1; ensure←dvi←open; if total←pages=0 then begin dvi←out(pre); dvi←out(id←byte); {output the preamble} @↑preamble of \.{DVI} file@> dvi←four(25400000); dvi←four(473628672); {conversion ratio for sp} prepare←mag; dvi←four(mag); {magnification factor is frozen} old←setting:=selector; selector:=new←string; print(" TeX output "); print←int(year); print←char("."); print←two(month); print←char("."); print←two(day); print←char(":"); print←two(time div 60); print←two(time mod 60); selector:=old←setting; dvi←out(cur←length); for s:=str←start[str←ptr] to pool←ptr-1 do dvi←out(str←pool[s]); pool←ptr:=str←start[str←ptr]; {flush the current string} end @ When |hlist←out| is called, its duty is to output the box represented by the |hlist←node| pointed to by |temp←ptr|. The reference point of that box has coordinates |(cur←h,cur←v)|. Similarly, when |vlist←out| is called, its duty is to output the box represented by the |vlist←node| pointed to by |temp←ptr|. The reference point of that box has coordinates |(cur←h,cur←v)|. @↑recursion@> @p procedure@?vlist←out; forward; {|hlist←out| and |vlist←out| are mutually recursive} @ The recursive procedures |hlist←out| and |vlist←out| each have local variables |save←h| and |save←v| to hold the values of |dvi←h| and |dvi←v| just before entering a new level of recursion. In effect, the values of |save←h| and |save←v| on \TeX's run-time stack correspond to the values of |h| and |v| that a \.{DVI}-reading program will push onto its coordinate stack. @d move←past=13 {go to this label when advancing past glue or a rule} @d fin←rule=14 {go to this label to finish processing a rule} @d next←p=15 {go to this label when finished with node |p|} @p @t\4@>@<Declare procedures needed in |hlist←out|, |vlist←out|@>@t@>@/ procedure hlist←out; {output an |hlist←node| box} label reswitch, move←past, fin←rule, next←p; var base←line: scaled; {the baseline coordinate for this box} @!left←edge: scaled; {the left coordinate for this box} @!save←h,@!save←v: scaled; {what |dvi←h| and |dvi←v| should pop to} @!this←box: pointer; {pointer to containing box} @!g←order: glue←ord; {applicable order of infinity for glue} @!g←sign: normal..shrinking; {selects type of glue} @!p:pointer; {current position in the hlist} @!save←loc:integer; {\.{DVI} byte location upon entry} @!leader←box:pointer; {the leader box being replicated} @!leader←wd:scaled; {width of leader box being replicated} @!lx:scaled; {extra space between leader boxes} @!outer←doing←leaders:boolean; {were we doing leaders?} @!edge:scaled; {left edge of sub-box, or right edge of leader space} begin this←box:=temp←ptr; g←order:=glue←order(this←box); g←sign:=glue←sign(this←box); p:=list←ptr(this←box); incr(cur←s); if cur←s>0 then dvi←out(push); if cur←s>max←push then max←push:=cur←s; save←loc:=dvi←offset+dvi←ptr; base←line:=cur←v; left←edge:=cur←h; while p<>null do @<Output node |p| for |hlist←out| and move to the next node, maintaining the condition |cur←v=base←line|@>; prune←movements(save←loc); if cur←s>0 then dvi←pop(save←loc); decr(cur←s); end; @ We ought to give special care to the efficiency of one part of |hlist←out|, since it belongs to \TeX's inner loop. When a |char←node| is encountered, we save a little time by processing several nodes in succession until reaching a non-|char←node|. The program uses the fact that |set←char←0=0|. @↑inner loop@> @<Output node |p| for |hlist←out|...@>= reswitch: if is←char←node(p) then begin synch←h; synch←v; repeat f:=font(p); c:=character(p); if f<>dvi←f then @<Change font |dvi←f| to |f|@>; if c<qi(128) then dvi←out(qo(c)) else begin dvi←out(set1); dvi←out(qo(c)); end; cur←h:=cur←h+char←width(f)(char←info(f)(c)); p:=link(p); until not is←char←node(p); dvi←h:=cur←h; end else @<Output the non-|char←node| |p| for |hlist←out| and move to the next node@> @ @<Change font |dvi←f| to |f|@>= begin if not font←used[f] then begin dvi←font←def(f); font←used[f]:=true; end; if f<=64+font←base then dvi←out(f-font←base-1+fnt←num←0) else begin dvi←out(fnt1); dvi←out(f-font←base-1); end; dvi←f:=f; end @ @<Output the non-|char←node| |p| for |hlist←out|...@>= begin case type(p) of hlist←node,vlist←node:@<Output a box in an hlist@>; rule←node: begin rule←ht:=height(p); rule←dp:=depth(p); rule←wd:=width(p); goto fin←rule; end; whatsit←node: @<Output the whatsit node |p| in an hlist@>; glue←node: @<Move right or output leaders@>; kern←node,math←node:cur←h:=cur←h+width(p); ligature←node: @<Make node |p| look like a |char←node| and |goto reswitch|@>; othercases do←nothing endcases;@/ goto next←p; fin←rule: @<Output a rule in an hlist@>; move←past: cur←h:=cur←h+rule←wd; next←p:p:=link(p); end @ @<Output a box in an hlist@>= if list←ptr(p)=null then cur←h:=cur←h+width(p) else begin save←h:=dvi←h; save←v:=dvi←v; cur←v:=base←line+shift←amount(p); {shift the box down} temp←ptr:=p; edge:=cur←h; if type(p)=vlist←node then vlist←out@+else hlist←out; dvi←h:=save←h; dvi←v:=save←v; cur←h:=edge+width(p); cur←v:=base←line; end @ @<Output a rule in an hlist@>= if is←running(rule←ht) then rule←ht:=height(this←box); if is←running(rule←dp) then rule←dp:=depth(this←box); rule←ht:=rule←ht+rule←dp; {this is the rule thickness} if (rule←ht>0)and(rule←wd>0) then {we don't output empty rules} begin synch←h; cur←v:=base←line+rule←dp; synch←v; dvi←out(set←rule); dvi←four(rule←ht); dvi←four(rule←wd); cur←v:=base←line; dvi←h:=dvi←h+rule←wd; end @ @<Move right or output leaders@>= begin g:=glue←ptr(p); rule←wd:=width(g); if g←sign<>normal then begin if g←sign=stretching then begin if stretch←order(g)=g←order then rule←wd:=rule←wd+round(float(glue←set(this←box))*stretch(g)); @↑real multiplication@> end else begin if shrink←order(g)=g←order then rule←wd:=rule←wd-round(float(glue←set(this←box))*shrink(g)); end; end; if subtype(p)>=a←leaders then @<Output leaders in an hlist, |goto fin←rule| if a rule or to |next←p| if done@>; goto move←past; end @ @<Output leaders in an hlist...@>= begin leader←box:=leader←ptr(p); if type(leader←box)=rule←node then begin rule←ht:=height(leader←box); rule←dp:=depth(leader←box); goto fin←rule; end; leader←wd:=width(leader←box); if (leader←wd>0)and(rule←wd>0) then begin edge:=cur←h+rule←wd; lx:=0; @<Let |cur←h| be the position of the first box, and set |leader←wd+lx| to the spacing between corresponding parts of boxes@>; while cur←h+leader←wd<=edge do @<Output a leader box at |cur←h|, then advance |cur←h| by |leader←wd+lx|@>; cur←h:=edge; goto next←p; end; end @ The calculations related to leaders require a bit of care. First, in the case of |a←leaders| (aligned leaders), we want to move |cur←h| to |left←edge| plus the smallest multiple of |leader←wd| for which the result is not less than the current value of |cur←h|; i.e., |cur←h| should become $|left←edge|+|leader←wd|\times\lceil (|cur←h|-|left←edge|)/|leader←wd|\rceil$. The program here should work in all cases even though some implementations of \PASCAL\ give nonstandard results for the |div| operation precisely, and even when |cur←h| is less than |left←edge|. In the case of |c←leaders| (centered leaders), we want to increase |cur←h| by half of the excess space not occupied by the leaders; and in the case of case of |x←leaders| (expanded leaders) we increase |cur←h| by $1/(q+1)$ of this excess space, where $q$ is the number of times the leader box will be replicated. Slight inaccuracies in the division might accumulate; half of this rounding error is placed at each end of the leaders. @<Let |cur←h| be the position of the first box, ...@>= if subtype(p)=a←leaders then begin save←h:=cur←h; cur←h:=left←edge+leader←wd*((cur←h-left←edge)@!div leader←wd); if cur←h<save←h then cur←h:=cur←h+leader←wd; end else begin lq:=rule←wd div leader←wd; {the number of box copies} lr:=rule←wd mod leader←wd; {the remaining space} if subtype(p)=c←leaders then cur←h:=cur←h+(lr div 2) else begin lx:=(2*lr+lq+1) div (2*lq+2); {round|(lr/(lq+1))|} cur←h:=cur←h+((lr-(lq-1)*lx) div 2); end; end @ The `\\{synch}' operations here are intended to decrease the number of bytes needed to specify horizontal and vertical motion in the \.{DVI} output. @<Output a leader box at |cur←h|, ...@>= begin cur←v:=base←line+shift←amount(leader←box); synch←v; save←v:=dvi←v;@/ synch←h; save←h:=dvi←h; temp←ptr:=leader←box; outer←doing←leaders:=doing←leaders; doing←leaders:=true; if type(leader←box)=vlist←node then vlist←out@+else hlist←out; doing←leaders:=outer←doing←leaders; dvi←v:=save←v; dvi←h:=save←h; cur←v:=save←v; cur←h:=save←h+leader←wd+lx; end @ The |vlist←out| routine is similar to |hlist←out|, but a bit simpler. @p procedure vlist←out; {output a |vlist←node| box} label move←past, fin←rule, next←p; var left←edge: scaled; {the left coordinate for this box} @!top←edge: scaled; {the top coordinate for this box} @!save←h,@!save←v: scaled; {what |dvi←h| and |dvi←v| should pop to} @!this←box: pointer; {pointer to containing box} @!g←order: glue←ord; {applicable order of infinity for glue} @!g←sign: normal..shrinking; {selects type of glue} @!p:pointer; {current position in the vlist} @!save←loc:integer; {\.{DVI} byte location upon entry} @!leader←box:pointer; {the leader box being replicated} @!leader←ht:scaled; {height of leader box being replicated} @!lx:scaled; {extra space between leader boxes} @!outer←doing←leaders:boolean; {were we doing leaders?} @!edge:scaled; {bottom boundary of leader space} begin this←box:=temp←ptr; g←order:=glue←order(this←box); g←sign:=glue←sign(this←box); p:=list←ptr(this←box); incr(cur←s); if cur←s>0 then dvi←out(push); if cur←s>max←push then max←push:=cur←s; save←loc:=dvi←offset+dvi←ptr; left←edge:=cur←h; cur←v:=cur←v-height(this←box); top←edge:=cur←v; while p<>null do @<Output node |p| for |vlist←out| and move to the next node, maintaining the condition |cur←h=left←edge|@>; prune←movements(save←loc); if cur←s>0 then dvi←pop(save←loc); decr(cur←s); end; @ @<Output node |p| for |vlist←out|...@>= begin if is←char←node(p) then confusion("vlistout") @:this can't happen vlistout}{\quad vlistout@> else @<Output the non-|char←node| |p| for |vlist←out|@>; next←p:p:=link(p); end @ @<Output the non-|char←node| |p| for |vlist←out|@>= begin case type(p) of hlist←node,vlist←node:@<Output a box in a vlist@>; rule←node: begin rule←ht:=height(p); rule←dp:=depth(p); rule←wd:=width(p); goto fin←rule; end; whatsit←node: @<Output the whatsit node |p| in a vlist@>; glue←node: @<Move down or output leaders@>; kern←node:cur←v:=cur←v+width(p); othercases do←nothing endcases;@/ goto next←p; fin←rule: @<Output a rule in a vlist, |goto next←p|@>; move←past: cur←v:=cur←v+rule←ht; end @ The |synch←v| here allows the \.{DVI} output to use one-byte commands for adjusting |v| in most cases, since the baselineskip distance will usually be constant. @<Output a box in a vlist@>= if list←ptr(p)=null then cur←v:=cur←v+height(p)+depth(p) else begin cur←v:=cur←v+height(p); synch←v; save←h:=dvi←h; save←v:=dvi←v; cur←h:=left←edge+shift←amount(p); {shift the box right} temp←ptr:=p; if type(p)=vlist←node then vlist←out@+else hlist←out; dvi←h:=save←h; dvi←v:=save←v; cur←v:=save←v+depth(p); cur←h:=left←edge; end @ @<Output a rule in a vlist...@>= if is←running(rule←wd) then rule←wd:=width(this←box); rule←ht:=rule←ht+rule←dp; {this is the rule thickness} cur←v:=cur←v+rule←ht; if (rule←ht>0)and(rule←wd>0) then {we don't output empty rules} begin synch←h; synch←v; dvi←out(put←rule); dvi←four(rule←ht); dvi←four(rule←wd); end; goto next←p @ @<Move down or output leaders@>= begin g:=glue←ptr(p); rule←ht:=width(g); if g←sign<>normal then begin if g←sign=stretching then begin if stretch←order(g)=g←order then rule←ht:=rule←ht+round(float(glue←set(this←box))*stretch(g)); @↑real multiplication@> end else begin if shrink←order(g)=g←order then rule←ht:=rule←ht-round(float(glue←set(this←box))*shrink(g)); end; end; if subtype(p)>=a←leaders then @<Output leaders in a vlist, |goto fin←rule| if a rule or to |next←p| if done@>; goto move←past; end @ @<Output leaders in a vlist...@>= begin leader←box:=leader←ptr(p); if type(leader←box)=rule←node then begin rule←wd:=width(leader←box); rule←dp:=0; goto fin←rule; end; leader←ht:=height(leader←box)+depth(leader←box); if (leader←ht>0)and(rule←ht>0) then begin edge:=cur←v+rule←ht; lx:=0; @<Let |cur←v| be the position of the first box, and set |leader←ht+lx| to the spacing between corresponding parts of boxes@>; while cur←v+leader←ht<=edge do @<Output a leader box at |cur←v|, then advance |cur←v| by |leader←ht+lx|@>; cur←v:=edge; goto next←p; end; end @ @<Let |cur←v| be the position of the first box, ...@>= if subtype(p)=a←leaders then begin save←v:=cur←v; cur←v:=top←edge+leader←ht*((cur←v-top←edge)@!div leader←ht); if cur←v<save←v then cur←v:=cur←v+leader←ht; end else begin lq:=rule←ht div leader←ht; {the number of box copies} lr:=rule←ht mod leader←ht; {the remaining space} if subtype(p)=c←leaders then cur←v:=cur←v+(lr div 2) else begin lx:=(2*lr+lq+1) div (2*lq+2); {round|(lr/(lq+1))|} cur←v:=cur←v+((lr-(lq-1)*lx) div 2); end; end @ When we reach this part of the program, |cur←v| indicates the top of a leader box, not its baseline. @<Output a leader box at |cur←v|, ...@>= begin cur←h:=left←edge+shift←amount(leader←box); synch←h; save←h:=dvi←h;@/ cur←v:=cur←v+height(leader←box); synch←v; save←v:=dvi←v; temp←ptr:=leader←box; outer←doing←leaders:=doing←leaders; doing←leaders:=true; if type(leader←box)=vlist←node then vlist←out@+else hlist←out; doing←leaders:=outer←doing←leaders; dvi←v:=save←v; dvi←h:=save←h; cur←h:=save←h; cur←v:=save←v-height(leader←box)+leader←ht+lx; end @ The |hlist←out| and |vlist←out| procedures are now complete, so we are ready for the |ship←out| routine that gets them started in the first place. @p procedure ship←out(@!p:pointer); {output the box |p|} label done; var page←loc:integer; {location of the current |bop|} @!j,@!k:0..9; {indices to first ten count registers} @!s:pool←pointer; {index into |str←pool|} @!old←setting:0..max←selector; {saved |selector| setting} begin if tracing←output>0 then begin print←nl(""); print←ln; print("Completed box being shipped out"); @.Completed box...@> end; if term←offset>max←print←line-9 then print←ln else if (term←offset>0)or(file←offset>0) then print←char(" "); print←char("["); j:=9; while (count(j)=0)and(j>0) do decr(j); for k:=0 to j do begin print←int(count(k)); if k<j then print←char("."); end; update←terminal; if tracing←output>0 then begin print←char("]"); begin←diagnostic; show←box(p); end←diagnostic(true); end; @<Ship box |p| out@>; if tracing←output<=0 then print←char("]"); dead←cycles:=0; update←terminal; {progress report} @<Flush the box from memory, showing statistics if requested@>; end; @ @<Flush the box from memory, showing statistics if requested@>= @!stat if tracing←stats>1 then begin print←nl("Memory usage before: "); @.Memory usage...@> print←int(var←used); print←char("&"); print←int(dyn←used); print←char(";"); end; tats@/ flush←node←list(p); @!stat if tracing←stats>1 then begin print(" after: "); print←int(var←used); print←char("&"); print←int(dyn←used); print("; still untouched: "); print←int(hi←mem←min-lo←mem←max-1); print←ln; end; tats @ @<Ship box |p| out@>= @<Update the values of |max←h| and |max←v|; but if the page is too large, |goto done|@>; @<Initialize variables as |ship←out| begins@>; page←loc:=dvi←offset+dvi←ptr; dvi←out(bop); for k:=0 to 9 do dvi←four(count(k)); dvi←four(last←bop); last←bop:=page←loc; cur←v:=height(p)+v←offset; temp←ptr:=p; if type(p)=vlist←node then vlist←out@+else hlist←out; dvi←out(eop); incr(total←pages); done: @ Sometimes the user will generate a huge page because other error messages are being ignored. Such pages are not output to the \.{dvi} file, since they may confuse the printing software. @<Update the values of |max←h| and |max←v|; but if the page is too large...@>= if (height(p)>max←dimen)or@|(depth(p)>max←dimen)or@| (height(p)+depth(p)+v←offset>max←dimen)or@| (width(p)+h←offset>max←dimen) then begin print←err("Huge page cannot be shipped out"); help2("The page just created is more than 18 feet tall or")@/ ("more than 18 feet wide, so I suspect something went wrong."); error; if tracing←output<=0 then begin begin←diagnostic; print←nl("The following box has been deleted:"); show←box(p); end←diagnostic(true); end; goto done; end; if height(p)+depth(p)+v←offset>max←v then max←v:=height(p)+depth(p)+v←offset; if width(p)+h←offset>max←h then max←h:=width(p)+h←offset @ At the end of the program, we must finish things off by writing the post\-amble. If |total←pages=0|, the \.{DVI} file was never opened. An integer variable |k| will be declared for use by this routine. @<Finish the \.{DVI} file@>= if total←pages=0 then print←nl("No pages of output.") @.No pages of output@> else begin dvi←out(post); {beginning of the postamble} dvi←four(last←bop); last←bop:=dvi←offset+dvi←ptr-5; {|post| location} dvi←four(25400000); dvi←four(473628672); {conversion ratio for sp} prepare←mag; dvi←four(mag); {magnification factor} dvi←four(max←v); dvi←four(max←h);@/ dvi←out(max←push div 256); dvi←out(max←push mod 256);@/ dvi←out(total←pages div 256); dvi←out(total←pages mod 256);@/ @<Output the font definitions for all fonts that were used@>; dvi←out(post←post); dvi←four(last←bop); dvi←out(id←byte);@/ k:=4+((dvi←buf←size-dvi←ptr) mod 4); {the number of 223's} while k>0 do begin dvi←out(223); decr(k); end; @<Empty the last bytes out of |dvi←buf|@>; print←nl("Output written on "); print(output←file←name); @.Output written on x@> print(" ("); print←int(total←pages); print(" page"); if total←pages<>1 then print←char("s"); print(", "); print←int(dvi←offset+dvi←ptr); print(" bytes)."); b←close(dvi←file); end @ @<Output the font definitions...@>= while font←ptr>font←base do begin if font←used[font←ptr] then dvi←font←def(font←ptr); decr(font←ptr); end @* \[33] Packaging. We're essentially done with the parts of \TeX\ that are concerned with the input (|get←next|) and the output (|ship←out|). So it's time to get heavily into the remaining part, which does the real work of typesetting. After lists are constructed, \TeX\ wraps them up and puts them into boxes. Two major subroutines are given the responsibility for this task: |hpack| applies to horizontal lists (hlists) and |vpack| applies to vertical lists (vlists). The main duty of |hpack| and |vpack| is to compute the dimensions of the resulting boxes, and to adjust the glue if one of those dimensions is pre-specified. The computed sizes normally enclose all of the material inside the new box; but some items may stick out if negative glue is used, if the box is overfull, or if a \.{\\vbox} includes other boxes that have been shifted left. The subroutine call |hpack(p,w,m)| returns a pointer to an |hlist←node| for a box containing the hlist that starts at |p|. Parameter |w| specifies a width; and parameter |m| is either `|exactly|' or `|additional|'. Thus, |hpack(p,w,exactly)| produces a box whose width is exactly |w|, while |hpack(p,w,additional)| yields a box whose width is the natural width plus |w|. It is convenient to define a macro called `|natural|' to cover the most common case, so that we can say |hpack(p,natural)| to get a box that has the natural width of list |p|. Similarly, |vpack(p,w,m)| returns a pointer to a |vlist←node| for a box containing the vlist that starts at |p|. In this case |w| represents a height instead of a width; the parameter |m| is interpreted as in |hpack|. @d exactly=0 {a box dimension is pre-specified} @d additional=1 {a box dimension is increased from the natural one} @d natural==0,additional {shorthand for parameters to |hpack| and |vpack|} @ The parameters to |hpack| and |vpack| correspond to \TeX's primitives like `\.{\\hbox} \.{to} \.{300pt}', `\.{\\hbox} \.{spread} \.{10pt}'; note that `\.{\\hbox}' with no dimension following it is equivalent to `\.{\\hbox} \.{spread} \.{0pt}'. The |scan←spec| subroutine scans such constructions in the user's input, including the mandatory left brace that follows them, and it puts the specification onto |save←stack| so that the desired box can later be obtained by executing the following code: $$\vbox{\halign{#\hfil\cr |save←ptr:=save←ptr-2;|\cr |hpack(p,saved(1),saved(0)).|\cr}}$$ @p procedure scan←spec; {scans a box specification and left brace} label found; begin if scan←keyword("to") then saved(0):=exactly @.to@> else if scan←keyword("spread") then saved(0):=additional @.spread@> else begin saved(0):=additional; saved(1):=0; goto found; end; scan←normal←dimen; saved(1):=cur←val; found: save←ptr:=save←ptr+2; scan←left←brace; end; @ To figure out the glue setting, |hpack| and |vpack| determine how much stretchability and shrinkability are present, considering all four orders of infinity. The highest order of infinity that has a nonzero coefficient is then used as if no other orders were present. For example, suppose that the given list contains six glue nodes with the respective stretchabilities 3pt, 8fill, 5fil, 6pt, $-3$fil, $-8$fill. Then the total is essentially 2fil; and if a total additional space of 6pt is to be achieved by stretching, the actual amounts of stretch will be 0pt, 0pt, 15pt, 0pt, $-9$pt, and 0pt, since only `fil' glue will be considered. (The `fill' glue is therefore not really stretching infinitely with respect to `fil'; nobody would actually want that to happen.) The arrays |total←stretch| and |total←shrink| are used to determine how much glue of each kind is present. @<Glob...@>= @!total←stretch, @!total←shrink: array[glue←ord] of scaled; {glue found by |hpack| or |vpack|} @ If the global variable |adjust←tail| is non-null, the |hpack| routine also removes all occurrences of |ins←node|, |mark←node|, and |adjust←node| items and appends the resulting material onto the list that ends at location |adjust←tail|. @< Glob...@>= @!adjust←tail:pointer; {tail of adjustment list} @ @<Set init...@>=adjust←tail:=null; @ Here now is |hpack|, which contains few if any surprises. @p function hpack(@!p:pointer;@!w:scaled;@!m:small←number):pointer; label reswitch, common←ending, exit; var r:pointer; {the box node that will be returned} @!q:pointer; {trails behind |p|} @!h,@!d,@!x:scaled; {height, depth, and natural width} @!s:scaled; {shift amount} @!g:pointer; {points to a glue specification} @!o:glue←ord; {order of infinity} @!f:internal←font←number; {the font in a |char←node|} @!i:four←quarters; {font information about a |char←node|} @!hd:eight←bits; {height and depth indices for a character} @!b:integer; {badness of the new box} begin r:=get←node(box←node←size); type(r):=hlist←node; subtype(r):=min←quarterword; shift←amount(r):=0; q:=r+list←offset; link(q):=p;@/ h:=0; @<Clear dimensions to zero@>; while p<>null do @<Examine node |p| in the hlist, taking account of its effect on the dimensions of the new box, or moving it to the adjustment list; then advance |p| to the next node@>; if adjust←tail<>null then link(adjust←tail):=null; height(r):=h; depth(r):=d;@/ @<Determine the value of |width(r)| and the appropriate glue setting; then |return| or |goto common←ending|@>; common←ending: @<Finish issuing a diagnostic message for an overfull or underfull hbox@>; exit: hpack:=r; end; @ @<Clear dimensions to zero@>= d:=0; x:=0; total←stretch[normal]:=0; total←shrink[normal]:=0; total←stretch[fil]:=0; total←shrink[fil]:=0; total←stretch[fill]:=0; total←shrink[fill]:=0; total←stretch[filll]:=0; total←shrink[filll]:=0 @ @<Examine node |p| in the hlist, taking account of its effect...@>= @↑inner loop@> begin reswitch: while is←char←node(p) do @<Incorporate character dimensions into the dimensions of the hbox that will contain~it, then move to the next node@>; if p<>null then begin case type(p) of hlist←node,vlist←node,rule←node,unset←node: @<Incorporate box dimensions into the dimensions of the hbox that will contain~it@>; ins←node,mark←node,adjust←node: if adjust←tail<>null then @<Transfer node |p| to the adjustment list@>; whatsit←node:@<Incorporate a whatsit node into an hbox@>; glue←node:@<Incorporate glue into the horizontal totals@>; kern←node,math←node: x:=x+width(p); ligature←node: @<Make node |p| look like a |char←node| and |goto reswitch|@>; othercases do←nothing endcases;@/ p:=link(p); end; end @ @<Make node |p| look like a |char←node| and |goto reswitch|@>= begin mem[lig←trick]:=mem[lig←char(p)]; link(lig←trick):=link(p); p:=lig←trick; goto reswitch; end @ The code here implicitly uses the fact that running dimensions are indicated by |null←flag|, which will be ignored in the calculations becase it is a highly negative number. @<Incorporate box dimensions into the dimensions of the hbox...@>= begin x:=x+width(p); if type(p)>=rule←node then s:=0 @+else s:=shift←amount(p); if height(p)-s>h then h:=height(p)-s; if depth(p)+s>d then d:=depth(p)+s; end @ The following code is part of \TeX's inner loop; i.e., adding another character of text to the user's input will cause each of these instructions to be exercised one more time. @↑inner loop@> @<Incorporate character dimensions into the dimensions of the hbox...@>= begin f:=font(p); i:=char←info(f)(character(p)); hd:=height←depth(i); x:=x+char←width(f)(i);@/ s:=char←height(f)(hd);@+if s>h then h:=s; s:=char←depth(f)(hd);@+if s>d then d:=s; p:=link(p); end @ Although node |q| is not necessarily the immediate predecessor of node |p|, it always points to some node in the list preceding |p|. Thus, we can delete nodes by moving |q| when necessary. The algorithm takes linear time, and the extra computation does not intrude on the inner loop unless it is necessary to make a deletion. @<Transfer node |p| to the adjustment list@>= begin while link(q)<>p do q:=link(q); if type(p)=adjust←node then begin link(adjust←tail):=adjust←ptr(p); while link(adjust←tail)<>null do adjust←tail:=link(adjust←tail); p:=link(p); free←node(link(q),small←node←size); end else begin link(adjust←tail):=p; adjust←tail:=p; p:=link(p); end; link(q):=p; p:=q; end @ @<Incorporate glue into the horizontal totals@>= begin g:=glue←ptr(p); x:=x+width(g);@/ o:=stretch←order(g); total←stretch[o]:=total←stretch[o]+stretch(g); o:=shrink←order(g); total←shrink[o]:=total←shrink[o]+shrink(g); if subtype(p)>=a←leaders then begin g:=leader←ptr(p); if height(g)>h then h:=height(g); if depth(g)>d then d:=depth(g); end; end @ When we get to the present part of the program, |x| is the natural width of the box being packaged. @<Determine the value of |width(r)| and the appropriate glue setting...@>= if m=additional then w:=x+w; width(r):=w; x:=w-x; {now |x| is the excess to be made up} if x=0 then begin glue←sign(r):=normal; glue←order(r):=normal; set←glue←ratio←zero(glue←set(r)); return; end else if x>0 then @<Determine horizontal glue stretch setting, then |return| or \hbox{|goto common←ending|}@> else @<Determine horizontal glue shrink setting, then |return| or \hbox{|goto common←ending|}@> @ @<Determine horizontal glue stretch setting...@>= begin @<Determine the stretch order@>; glue←order(r):=o; glue←sign(r):=stretching; if total←stretch[o]<>0 then glue←set(r):=unfloat(x/total←stretch[o]) @↑real division@> else begin glue←sign(r):=normal; set←glue←ratio←zero(glue←set(r)); {there's nothing to stretch} end; if (hbadness<inf←bad)and(o=normal)and(list←ptr(r)<>null) then @<Report an underfull hbox and |goto common←ending|, if this box is sufficiently bad@>; return; end @ @<Determine the stretch order@>= if total←stretch[filll]<>0 then o:=filll else if total←stretch[fill]<>0 then o:=fill else if total←stretch[fil]<>0 then o:=fil else o:=normal @ @<Report an underfull hbox and |goto common←ending|, if...@>= begin b:=badness(x,total←stretch[normal]); if b>hbadness then begin print←ln; if b>100 then print←nl("Underfull")@+else print←nl("Loose"); print(" \hbox (badness "); print←int(b); @.Underfull \\hbox...@> @.Loose \\hbox...@> goto common←ending; end; end @ In order to provide a decent indication of where an overfull or underfull box originated, we use a global variable |pack←begin←line| that is set nonzero only when |hpack| is being called by the paragraph builder or the alignment finishing routine. @<Glob...@>= @!pack←begin←line:integer; {source file line where the current paragraph or alignment began; a negative value denotes alignment} @ @<Set init...@>= pack←begin←line:=0; @ @<Finish issuing a diagnostic message for an overfull or underfull hbox@>= if output←active then print(") has occurred while \output is active") else begin if pack←begin←line<>0 then begin if pack←begin←line>0 then print(") in paragraph at lines ") else print(") in alignment at lines "); print←int(abs(pack←begin←line)); print("--"); end else print(") detected at line "); print←int(line); end; print←ln;@/ font←in←short←display:=null←font; short←display(list←ptr(r)); print←ln;@/ begin←diagnostic; show←box(r); end←diagnostic(true) @ @<Determine horizontal glue shrink setting...@>= begin @<Determine the shrink order@>; glue←order(r):=o; glue←sign(r):=shrinking; if total←shrink[o]<>0 then glue←set(r):=unfloat((-x)/total←shrink[o]) @↑real division@> else begin glue←sign(r):=normal; set←glue←ratio←zero(glue←set(r)); {there's nothing to shrink} end; if (total←shrink[o]<-x)and(o=normal)and(list←ptr(r)<>null) then begin set←glue←ratio←one(glue←set(r)); {this is the maximum shrinkage} @<Report an overfull hbox and |goto common←ending|, if this box is sufficiently bad@>; end else if (hbadness<100)and(o=normal)and(list←ptr(r)<>null) then @<Report a tight hbox and |goto common←ending|, if this box is sufficiently bad@>; return; end @ @<Determine the shrink order@>= if total←shrink[filll]<>0 then o:=filll else if total←shrink[fill]<>0 then o:=fill else if total←shrink[fil]<>0 then o:=fil else o:=normal @ @<Report an overfull hbox and |goto common←ending|, if...@>= if (-x-total←shrink[normal]>hfuzz)or(hbadness<100) then begin if (overfull←rule>0)and(-x-total←shrink[normal]>hfuzz) then begin while link(q)<>null do q:=link(q); link(q):=new←rule; width(link(q)):=overfull←rule; end; print←ln; print←nl("Overfull \hbox ("); @.Overfull \\hbox...@> print←scaled(-x-total←shrink[normal]); print("pt too wide"); goto common←ending; end @ @<Report a tight hbox and |goto common←ending|, if...@>= begin b:=badness(-x,total←shrink[normal]); if b>hbadness then begin print←ln; print←nl("Tight \hbox (badness "); print←int(b); @.Tight \\hbox...@> goto common←ending; end; end @ The |vpack| subroutine is actually a special case of a slightly more general routine called |vpackage|, which has four parameters. The fourth parameter, which is |max←dimen| in the case of |vpack|, specifies the maximum depth of the page box that is constructed. The depth is first computed by the normal rules; if it exceeds this limit, the reference point is simply moved down until the limiting depth is attained. @d vpack(#)==vpackage(#,max←dimen) {special case of unconstrained depth} @p function vpackage(@!p:pointer;@!h:scaled;@!m:small←number;@!l:scaled): pointer; label common←ending, exit; var r:pointer; {the box node that will be returned} @!w,@!d,@!x:scaled; {width, depth, and natural height} @!s:scaled; {shift amount} @!g:pointer; {points to a glue specification} @!o:glue←ord; {order of infinity} @!b:integer; {badness of the new box} begin r:=get←node(box←node←size); type(r):=vlist←node; subtype(r):=min←quarterword; shift←amount(r):=0; list←ptr(r):=p;@/ w:=0; @<Clear dimensions to zero@>; while p<>null do @<Examine node |p| in the vlist, taking account of its effect on the dimensions of the new box; then advance |p| to the next node@>; width(r):=w; if d>l then begin x:=x+d-l; depth(r):=l; end else depth(r):=d; @<Determine the value of |height(r)| and the appropriate glue setting; then |return| or |goto common←ending|@>; common←ending: @<Finish issuing a diagnostic message for an overfull or underfull vbox@>; exit: vpackage:=r; end; @ @<Examine node |p| in the vlist, taking account of its effect...@>= begin if is←char←node(p) then confusion("vpack") @:this can't happen vpack}{\quad vpack@> else case type(p) of hlist←node,vlist←node,rule←node,unset←node: @<Incorporate box dimensions into the dimensions of the vbox that will contain~it@>; whatsit←node:@<Incorporate a whatsit node into a vbox@>; glue←node: @<Incorporate glue into the vertical totals@>; kern←node: begin x:=x+d+width(p); d:=0; end; othercases do←nothing endcases; p:=link(p); end @ @<Incorporate box dimensions into the dimensions of the vbox...@>= begin x:=x+d+height(p); d:=depth(p); if type(p)>=rule←node then s:=0 @+else s:=shift←amount(p); if width(p)+s>w then w:=width(p)+s; end @ @<Incorporate glue into the vertical totals@>= begin x:=x+d; d:=0;@/ g:=glue←ptr(p); x:=x+width(g);@/ o:=stretch←order(g); total←stretch[o]:=total←stretch[o]+stretch(g); o:=shrink←order(g); total←shrink[o]:=total←shrink[o]+shrink(g); if subtype(p)>=a←leaders then begin g:=leader←ptr(p); if width(g)>w then w:=width(g); end; end @ When we get to the present part of the program, |x| is the natural height of the box being packaged. @<Determine the value of |height(r)| and the appropriate glue setting...@>= if m=additional then h:=x+h; height(r):=h; x:=h-x; {now |x| is the excess to be made up} if x=0 then begin glue←sign(r):=normal; glue←order(r):=normal; set←glue←ratio←zero(glue←set(r)); return; end else if x>0 then @<Determine vertical glue stretch setting, then |return| or \hbox{|goto common←ending|}@> else @<Determine vertical glue shrink setting, then |return| or \hbox{|goto common←ending|}@> @ @<Determine vertical glue stretch setting...@>= begin @<Determine the stretch order@>; glue←order(r):=o; glue←sign(r):=stretching; if total←stretch[o]<>0 then glue←set(r):=unfloat(x/total←stretch[o]) @↑real division@> else begin glue←sign(r):=normal; set←glue←ratio←zero(glue←set(r)); {there's nothing to stretch} end; if (vbadness<inf←bad)and(o=normal)and(list←ptr(r)<>null) then @<Report an underfull vbox and |goto common←ending|, if this box is sufficiently bad@>; return; end @ @<Report an underfull vbox and |goto common←ending|, if...@>= begin b:=badness(x,total←stretch[normal]); if b>vbadness then begin print←ln; if b>100 then print←nl("Underfull")@+else print←nl("Loose"); print(" \vbox (badness "); print←int(b); @.Underfull \\vbox...@> @.Loose \\vbox...@> goto common←ending; end; end @ @<Finish issuing a diagnostic message for an overfull or underfull vbox@>= if output←active then print(") has occurred while \output is active") else begin if pack←begin←line<>0 then {it's actually negative} begin print(") in alignment at lines "); print←int(abs(pack←begin←line)); print("--"); end else print(") detected at line "); print←int(line); print←ln;@/ end; begin←diagnostic; show←box(r); end←diagnostic(true) @ @<Determine vertical glue shrink setting...@>= begin @<Determine the shrink order@>; glue←order(r):=o; glue←sign(r):=shrinking; if total←shrink[o]<>0 then glue←set(r):=unfloat((-x)/total←shrink[o]) @↑real division@> else begin glue←sign(r):=normal; set←glue←ratio←zero(glue←set(r)); {there's nothing to shrink} end; if (total←shrink[o]<-x)and(o=normal)and(list←ptr(r)<>null) then begin set←glue←ratio←one(glue←set(r)); {this is the maximum shrinkage} @<Report an overfull vbox and |goto common←ending|, if this box is sufficiently bad@>; end else if (vbadness<100)and(o=normal)and(list←ptr(r)<>null) then @<Report a tight vbox and |goto common←ending|, if this box is sufficiently bad@>; return; end @ @<Report an overfull vbox and |goto common←ending|, if...@>= if (-x-total←shrink[normal]>vfuzz)or(vbadness<100) then begin print←ln; print←nl("Overfull \vbox ("); @.Overfull \\vbox...@> print←scaled(-x-total←shrink[normal]); print("pt too high"); goto common←ending; end @ @<Report a tight vbox and |goto common←ending|, if...@>= begin b:=badness(-x,total←shrink[normal]); if b>vbadness then begin print←ln; print←nl("Tight \vbox (badness "); print←int(b); @.Tight \\vbox...@> goto common←ending; end; end @ When a box is being appended to the current vertical list, the baselineskip calculation is handled by the |append←to←vlist| routine. @p procedure append←to←vlist(@!b:pointer); var d:scaled; {deficiency of space between baselines} @!p:pointer; {a new glue specification} begin if prev←depth>ignore←depth then begin d:=width(baseline←skip)-prev←depth-height(b); if d<line←skip←limit then p:=new←param←glue(line←skip←code) else begin p:=new←skip←param(baseline←skip←code); width(temp←ptr):=d; {|temp←ptr=glue←ptr(p)|} end; link(tail):=p; tail:=p; end; link(tail):=b; tail:=b; prev←depth:=depth(b); end; @* \[34] Data structures for math mode. When \TeX\ reads a formula that is enclosed between \.\$'s, it constructs an {\sl mlist}, which is essentially a tree structure representing that formula. An mlist is a linear sequence of items, but we can regard it as a tree structure because mlists can appear within mlists. For example, many of the entries can be subscripted or superscripted, and such ``scripts'' are mlists in their own right. An entire formula is parsed into such a tree before any of the actual typesetting is done, because the current style of type is usually not known until the formula has been fully scanned. For example, when the formula `\.{\$a+b \\over c+d\$}' is being read, there is no way to tell that `\.{a+b}' will be in script size until `\.{\\over}' has appeared. During the scanning process, each element of the mlist being built is classified as a relation, a binary operator, an open parenthesis, etc., or as a construct like `\.{\\sqrt}' that must be built up. This classification appears in the mlist data structure. After a formula has been fully scanned, the mlist is converted to an hlist so that it can be incorporated into the surrounding text. This conversion is controlled by a recursive procedure that decides all of the appropriate styles by a ``top-down'' process starting at the outermost level and working in towards the subformulas. The formula is ultimately pasted together using combinations of horizontal and vertical boxes, with glue and penalty nodes inserted as necessary. An mlist is represented internally as a linked list consisting chiefly of ``noads'' (pronounced ``no-adds''), to distinguish them from the somewhat similar ``nodes'' in hlists and vlists. Certain kinds of ordinary nodes are allowed to appear in mlists together with the noads; \TeX\ tells the difference by means of the |type| field, since a noad's |type| is always greater than that of a node. An mlist does not contain character nodes, hlist nodes, vlist nodes, math nodes, ligature nodes, mark nodes, insert nodes, adjust nodes, or unset nodes; in particular, each mlist item appears in the variable-size part of |mem|, so the |type| field is always present. @ Each noad is four or more words long. The first word contains the |type| and |subtype| and |link| fields that are already so familiar to us; the second, third, and fourth words are called the noad's |nucleus|, |subscr|, and |supscr| fields. Consider, for example, the simple formula `\.{\$x\↑2\$}', which would be parsed into an mlist containing a single element called an |ord←noad|. The |nucleus| of this noad is a representation of `\.x', the |subscr| is empty, and the |supscr| is a representation of `\.2'. The |nucleus|, |subscr|, and |supscr| fields are further broken into subfields. If |p| points to a noad, and if |q| is one of its principal fields (e.g., |q=subscr(p)|), there are several possibilities for the subfields, depending on the |math←type| of |q|. \yskip\hang|math←type(q)=math←char| means that |fam(q)| refers to one of the sixteen font families, and |character(q)| is the number of a character within a font of that family, as in a character node. \yskip\hang|math←type(q)=math←text←char| is similar, but the character is unsubscripted and unsuperscripted and it is followed immediately by another character from the same font. (This |math←type| setting appears only briefly during the processing; it is used to suppress unwanted italic corrections.) \yskip\hang|math←type(q)=empty| indicates a field with no value (the corresponding attribute of noad |p| is not present). \yskip\hang|math←type(q)=sub←box| means that |info(q)| points to a box node (either an |hlist←node| or a |vlist←node|) that should be used as the value of the field. The |shift←amount| in the subsidiary box node is the amount by which that box will be shifted downward. \yskip\hang|math←type(q)=sub←mlist| means that |info(q)| points to an mlist; the mlist must be converted to an hlist in order to obtain the value of this field. \yskip\noindent In the latter case, we might have |info(q)=null|. This is not the same as |math←type(q)=empty|; for example, `\.{\$P\←\{\}\$}' and `\.{\$P\$}' produce different results (the former will not have the ``italic correction'' added to the width of |P|, but the ``script skip'' will be added). The definitions of subfields given here are evidently wasteful of space, since a halfword is being used for the |math←type| although only three bits would be needed. However, there are hardly ever many noads present at once, since they are soon converted to nodes that take up even more space, so we can afford to represent them in whatever way simplifies the programming. @d noad←size=4 {number of words in a normal noad} @d nucleus(#)==#+1 {the |nucleus| field of a noad} @d supscr(#)==#+2 {the |supscr| field of a noad} @d subscr(#)==#+3 {the |subscr| field of a noad} @d math←type==link {a |halfword| in |mem|} @d fam==font {a |quarterword| in |mem|} @d math←char=1 {|math←type| when the attribute is simple} @d sub←box=2 {|math←type| when the attribute is a box} @d sub←mlist=3 {|math←type| when the attribute is a formula} @d math←text←char=4 {|math←type| when italic correction is dubious} @ Each portion of a formula is classified as Ord, Op, Bin, Rel, Ope, Clo, Pun, or Inn, for purposes of spacing and line breaking. An |ord←noad|, |op←noad|, |bin←noad|, |rel←noad|, |open←noad|, |close←noad|, |punct←noad|, or |inner←noad| is used to represent portions of the various types. For example, an `\.=' sign in a formula leads to the creation of a |rel←noad| whose |nucleus| field is a representation of an equals sign (usually |fam=0|, |character=@'75|). A formula preceded by \.{\\mathrel} also results in a |rel←noad|. When a |rel←noad| is followed by an |op←noad|, say, and possibly separated by one or more ordinary nodes (not noads), \TeX\ will insert a penalty node (with the current |rel←penalty|) just after the formula that corresponds to the |rel←noad|, unless there already was a penalty immediately following; and a ``thick space'' will be inserted just before the formula that corresponds to the |op←noad|. A noad of type |ord←noad|, |op←noad|, \dots, |inner←noad| usually has a |subtype=normal|. The only exception is that an |op←noad| might have |subtype=limits| or |no←limits|, if the normal positioning of limits has been overridden for this operator. @d ord←noad=unset←node+3 {|type| of a noad classified Ord} @d op←noad=ord←noad+1 {|type| of a noad classified Op} @d bin←noad=ord←noad+2 {|type| of a noad classified Bin} @d rel←noad=ord←noad+3 {|type| of a noad classified Rel} @d open←noad=ord←noad+4 {|type| of a noad classified Ope} @d close←noad=ord←noad+5 {|type| of a noad classified Clo} @d punct←noad=ord←noad+6 {|type| of a noad classified Pun} @d inner←noad=ord←noad+7 {|type| of a noad classified Inn} @d limits=1 {|subtype| of |op←noad| whose scripts are to be above, below} @d no←limits=2 {|subtype| of |op←noad| whose scripts are to be normal} @ A |radical←noad| is five words long; the fifth word is the |left←delimiter| field, which usually represents a square root sign. A |fraction←noad| is six words long; it has a |right←delimiter| field as well as a |left←delimiter|. Delimiter fields are of type |four←quarters|, and they have four subfields called |small←fam|, |small←char|, |large←fam|, |large←char|. These subfields represent variable-size delimiters by giving the ``small'' and ``large'' starting characters, as explained in Chapter~17 of {\sl The \TeX book}. @:TeXbook}{\sl The \TeX book@> A |fraction←noad| is actually quite different from all other noads. Not only does it have six words, it has |thickness|, |denominator|, and |numerator| fields instead of |nucleus|, |subscr|, and |supscr|. The |thickness| is a scaled value that tells how thick to make a fraction rule; however, the special value |default←code| is used to stand for the |default←rule←thickness| of the current size. The |numerator| and |denominator| point to mlists that define a fraction; we always have $$\hbox{|math←type(numerator)=math←type(denominator)=sub←mlist|}.$$ The |left←delimiter| and |right←delimiter| fields specify delimiters that will be placed at the left and right of the fraction. In this way, a |fraction←noad| is able to represent all of \TeX's operators \.{\\over}, \.{\\atop}, \.{\\above}, \.{\\overwithdelims}, \.{\\atopwithdelims}, and \.{\\abovewithdelims}. @d left←delimiter(#)==#+4 {first delimiter field of a noad} @d right←delimiter(#)==#+5 {second delimiter field of a fraction noad} @d radical←noad=inner←noad+1 {|type| of a noad for square roots} @d radical←noad←size=5 {number of |mem| words in a radical noad} @d fraction←noad=radical←noad+1 {|type| of a noad for generalized fractions} @d fraction←noad←size=6 {number of |mem| words in a fraction noad} @d small←fam(#)==mem[#].qqqq.b0 {|fam| for ``small'' delimiter} @d small←char(#)==mem[#].qqqq.b1 {|character| for ``small'' delimiter} @d large←fam(#)==mem[#].qqqq.b2 {|fam| for ``large'' delimiter} @d large←char(#)==mem[#].qqqq.b3 {|character| for ``large'' delimiter} @d thickness==width {|thickness| field in a fraction noad} @d default←code==@'10000000000 {denotes |default←rule←thickness|} @d numerator==supscr {|numerator| field in a fraction noad} @d denominator==subscr {|denominator| field in a fraction noad} @ The global variable |empty←field| is set up for initialization of empty fields in new noads. Similarly, |null←delimiter| is for the initialization of delimiter fields. @<Glob...@>= @!empty←field:two←halves; @!null←delimiter:four←quarters; @ @<Set init...@>= empty←field.rh:=empty; empty←field.lh:=null;@/ null←delimiter.b0:=0; null←delimiter.b1:=min←quarterword;@/ null←delimiter.b2:=0; null←delimiter.b3:=min←quarterword; @ The |new←noad| function creates an |ord←noad| that is completely null. @p function new←noad:pointer; var p:pointer; begin p:=get←node(noad←size); type(p):=ord←noad; subtype(p):=normal; mem[nucleus(p)].hh:=empty←field; mem[subscr(p)].hh:=empty←field; mem[supscr(p)].hh:=empty←field; new←noad:=p; end; @ A few more kinds of noads will complete the set: An |under←noad| has its nucleus underlined; an |over←noad| has it overlined. An |accent←noad| places an accent over its nucleus; the accent character appears as |fam(accent←chr(p))| and |character(accent←chr(p))|. A |vcenter←noad| centers its nucleus vertically with respect to the axis of the formula; we always have |math←type(nucleus(p))=sub←box| in this case. And finally, we have |left←noad| and |right←noad| types, to implement \TeX's \.{\\left} and \.{\\right}. The |nucleus| of such noads is replaced by a |delimiter| field; thus, for example, `\.{\\left(}' produces a |left←noad| such that |delimiter(p)| holds the family and character codes for all left parentheses. A |left←noad| never appears in an mlist except as the first element, and a |right←noad| never appears in an mlist except as the last element; furthermore, we either have both a |left←noad| and a |right←noad|, or neither one is present. The |subscr| and |supscr| fields are always |empty| in a |left←noad| and a |right←noad|. @d under←noad=fraction←noad+1 {|type| of a noad for underlining} @d over←noad=under←noad+1 {|type| of a noad for overlining} @d accent←noad=over←noad+1 {|type| of a noad for accented subformulas} @d accent←noad←size=5 {number of |mem| words in an accent noad} @d accent←chr(#)==#+4 {the |accent←chr| field of an accent noad} @d vcenter←noad=accent←noad+1 {|type| of a noad for \.{\\vcenter}} @d left←noad=vcenter←noad+1 {|type| of a noad for \.{\\left}} @d right←noad=left←noad+1 {|type| of a noad for \.{\\right}} @d delimiter==nucleus {|delimiter| field in left and right noads} @d scripts←allowed(#)==(type(#)>=ord←noad)and(type(#)<left←noad) @ Math formulas can also contain instructions like \.{\\textstyle} that override \TeX's normal style rules. A |style←node| is inserted into the data structure to record such instructions; it is three words long, so it is considered a node instead of a noad. The |subtype| is either |display←style| or |text←style| or |script←style| or |script←script←style|. The second and third words of a |style←node| are not used, but they are present because a |choice←node| is converted to a |style←node|. \TeX\ uses even numbers 0, 2, 4, 6 to encode the basic styles |display←style|, \dots, |script←script←style|, and adds~1 to get the ``cramped'' versions of these styles. This gives a numerical order that is backwards from the convention of Appendix~G in {\sl The \TeX book\/}; i.e., a smaller style has a larger numerical value. @:TeXbook}{\sl The \TeX book@> @d style←node=unset←node+1 {|type| of a style node} @d style←node←size=3 {number of words in a style node} @d display←style=0 {|subtype| for \.{\\displaystyle}} @d text←style=2 {|subtype| for \.{\\textstyle}} @d script←style=4 {|subtype| for \.{\\scriptstyle}} @d script←script←style=6 {|subtype| for \.{\\scriptscriptstyle}} @d cramped=1 {add this to an uncramped style if you want to cramp it} @p function new←style(@!s:small←number):pointer; {create a style node} var p:pointer; {the new node} begin p:=get←node(style←node←size); type(p):=style←node; subtype(p):=s; width(p):=0; depth(p):=0; {the |width| and |depth| are not used} new←style:=p; end; @ Finally, the \.{\\mathchoice} primitive creates a |choice←node|, which has special subfields |display←mlist|, |text←mlist|, |script←mlist|, and |script←script←mlist| pointing to the mlists for each style. @d choice←node=unset←node+2 {|type| of a choice node} @d display←mlist(#)==info(#+1) {mlist to be used in display style} @d text←mlist(#)==link(#+1) {mlist to be used in text style} @d script←mlist(#)==info(#+2) {mlist to be used in script style} @d script←script←mlist(#)==link(#+2) {mlist to be used in scriptscript style} @p function new←choice:pointer; {create a choice node} var p:pointer; {the new node} begin p:=get←node(style←node←size); type(p):=choice←node; subtype(p):=0; {the |subtype| is not used} display←mlist(p):=null; text←mlist(p):=null; script←mlist(p):=null; script←script←mlist(p):=null; new←choice:=p; end; @ Let's consider now the previously unwritten part of |show←node←list| that displays the things that can only be present in mlists; this program illustrates how to access the data structures just defined. In the context of the following program, |p| points to a node or noad that should be displayed, and the current string contains the ``recursion history'' that leads to this point. The recursion history consists of a dot for each outer level in which |p| is subsidiary to some node, or in which |p| is subsidiary to the |nucleus| field of some noad; the dot is replaced by `\.[' or `\.(' or `\./' or `\.\\' if |p| is descended from the |subscr| or |supscr| or |denominator| or |numerator| fields of noads. For example, the current string would be `\.{.(.[/}' if |p| points to the |ord←noad| for |x| in the (ridiculous) formula `\.{\$\\sqrt\{a\↑\{\{b\←\{c\\over x+y\} \}\}\}\$}'. @<Cases of |show←node←list| that arise...@>= style←node:print←style(subtype(p)); choice←node:@<Display choice node |p|@>; ord←noad,op←noad,bin←noad,rel←noad,open←noad,close←noad,punct←noad,inner←noad, radical←noad,over←noad,under←noad,vcenter←noad,accent←noad, left←noad,right←noad:@<Display normal noad |p|@>; fraction←noad:@<Display fraction noad |p|@>; @ Here are some simple routines used in the display of noads. @<Declare procedures needed for displaying the elements of mlists@>= procedure print←fam←and←char(@!p:pointer); {prints family and character} begin print←esc("fam"); print←int(fam(p)); print←char(" "); print←ASCII(qo(character(p))); end; @# procedure print←delimiter(@!p:pointer); {prints a delimiter as 24-bit hex value} var a:integer; {accumulator} begin a:=small←fam(p)*256+qo(small←char(p)); a:=a*@"1000+large←fam(p)*256+qo(large←char(p)); if a<0 then print←int(a) {this should never happen} else print←hex(a); end; @ The next subroutine will descend to another level of recursion when a subsidiary mlist needs to be displayed. The parameter |c| indicates what character is to become part of the recursion history. An empty mlist is distinguished from a field with |math←type(p)=empty|, because these are not equivalent (as explained above). @↑recursion@> @<Declare procedures needed for displaying...@>= procedure@?show←info; forward;@t\2@>@?{|show←node←list(info(temp←ptr))|} procedure print←subsidiary←data(@!p:pointer;@!c:ASCII←code); {display a noad field} begin if cur←length>=depth←threshold then begin if math←type(p)<>empty then print(" []"); end else begin append←char(c); {include |c| in the recursion history} temp←ptr:=p; {prepare for |show←info| if recursion is needed} case math←type(p) of math←char: begin print←ln; print←current←string; print←fam←and←char(p); end; sub←box: show←info; {recursive call} sub←mlist: if info(p)=null then begin print←ln; print←current←string; print("{}"); end else show←info; {recursive call} othercases do←nothing {|empty|} endcases;@/ flush←char; {remove |c| from the recursion history} end; end; @ The inelegant introduction of |show←info| in the code above seems better than the alternative of using \PASCAL's strange |forward| declaration for a procedure with parameters. The \PASCAL\ convention about dropping parameters from a post-|forward| procedure is, frankly, so intolerable to the author of \TeX\ that he would rather stoop to communication via a global temporary variable. (A similar stoopidity occurred with respect to |hlist←out| and |vlist←out| above, and it will occur with respect to |mlist←to←hlist| below.) @↑Knuth, Donald Ervin@> @:PASCAL}{\PASCAL@> @p procedure show←info; {the reader will kindly forgive this} begin show←node←list(info(temp←ptr)); end; @ @<Declare procedures needed for displaying...@>= procedure print←style(@!c:integer); begin case c div 2 of 0: print←esc("displaystyle"); {|display←style=0|} 1: print←esc("textstyle"); {|text←style=2|} 2: print←esc("scriptstyle"); {|script←style=4|} 3: print←esc("scriptscriptstyle"); {|script←script←style=6|} othercases print("Unknown style!") endcases; end; @ @<Display choice node |p|@>= begin print←esc("mathchoice"); append←char("D"); show←node←list(display←mlist(p)); flush←char; append←char("T"); show←node←list(text←mlist(p)); flush←char; append←char("S"); show←node←list(script←mlist(p)); flush←char; append←char("s"); show←node←list(script←script←mlist(p)); flush←char; end @ @<Display normal noad |p|@>= begin case type(p) of ord←noad: print←esc("mathord"); op←noad: print←esc("mathop"); bin←noad: print←esc("mathbin"); rel←noad: print←esc("mathrel"); open←noad: print←esc("mathopen"); close←noad: print←esc("mathclose"); punct←noad: print←esc("mathpunct"); inner←noad: print←esc("mathinner"); over←noad: print←esc("overline"); under←noad: print←esc("underline"); vcenter←noad: print←esc("vcenter"); radical←noad: begin print←esc("radical"); print←delimiter(left←delimiter(p)); end; accent←noad: begin print←esc("accent"); print←fam←and←char(accent←chr(p)); end; left←noad: begin print←esc("left"); print←delimiter(nucleus(p)); end; right←noad: begin print←esc("right"); print←delimiter(nucleus(p)); end; end; if subtype(p)<>normal then if subtype(p)=limits then print←esc("limits") else print←esc("nolimits"); if type(p)<left←noad then print←subsidiary←data(nucleus(p),"."); print←subsidiary←data(supscr(p),"↑"); print←subsidiary←data(subscr(p),"←"); end @ @<Display fraction noad |p|@>= begin print←esc("fraction, thickness "); if thickness(p)=default←code then print("= default") else print←scaled(thickness(p)); if (small←fam(left←delimiter(p))<>0)or@+ (small←char(left←delimiter(p))<>min←quarterword)or@| (large←fam(left←delimiter(p))<>0)or@| (large←char(left←delimiter(p))<>min←quarterword) then begin print(", left-delimiter "); print←delimiter(left←delimiter(p)); end; if (small←fam(right←delimiter(p))<>0)or@| (small←char(right←delimiter(p))<>min←quarterword)or@| (large←fam(right←delimiter(p))<>0)or@| (large←char(right←delimiter(p))<>min←quarterword) then begin print(", right-delimiter "); print←delimiter(right←delimiter(p)); end; print←subsidiary←data(numerator(p),"\"); print←subsidiary←data(denominator(p),"/"); end @ That which can be displayed can also be destroyed. @<Cases of |flush←node←list| that arise...@>= style←node: begin free←node(p,style←node←size); goto done; end; choice←node:begin flush←node←list(display←mlist(p)); flush←node←list(text←mlist(p)); flush←node←list(script←mlist(p)); flush←node←list(script←script←mlist(p)); free←node(p,style←node←size); goto done; end; ord←noad,op←noad,bin←noad,rel←noad,open←noad,close←noad,punct←noad,inner←noad, radical←noad,over←noad,under←noad,vcenter←noad,accent←noad:@t@>@;@/ begin if math←type(nucleus(p))>=sub←box then flush←node←list(info(nucleus(p))); if math←type(supscr(p))>=sub←box then flush←node←list(info(supscr(p))); if math←type(subscr(p))>=sub←box then flush←node←list(info(subscr(p))); if type(p)=radical←noad then free←node(p,radical←noad←size) else if type(p)=accent←noad then free←node(p,accent←noad←size) else free←node(p,noad←size); goto done; end; left←noad,right←noad: begin free←node(p,noad←size); goto done; end; fraction←noad: begin flush←node←list(info(numerator(p))); flush←node←list(info(denominator(p))); free←node(p,fraction←noad←size); goto done; end; @* \[35] Subroutines for math mode. In order to convert mlists to hlists, i.e., noads to nodes, we need several subroutines that are conveniently dealt with now. Let us first introduce the macros that make it easy to get at the parameters and other font information. A size code, which is a multiple of 16, is added to a family number to get an index into the table of internal font numbers for each combination of family and size. (Be alert: Size codes get larger as the type gets smaller.) @d text←size=0 {size code for the largest size in a family} @d script←size=16 {size code for the medium size in a family} @d script←script←size=32 {size code for the smallest size in a family} @<Basic printing procedures@>= procedure print←size(@!s:integer); begin if s=0 then print←esc("textfont") else if s=script←size then print←esc("scriptfont") else print←esc("scriptscriptfont"); end; @ Before an mlist is converted to an hlist, \TeX\ makes sure that the fonts in family~2 have enough parameters to be math-symbol fonts, and that the fonts in family~3 have enough parameters to be math-extension fonts. The math-symbol parameters are referred to by using the following macros, which take a size code as their parameter; for example, |num1(cur←size)| gives the value of the |num1| parameter for the current size. @↑parameters for symbols@> @↑font parameters@> @d mathsy←end(#)==fam←fnt(2+#)]].sc @d mathsy(#)==font←info[#+param←base[mathsy←end @d math←x←height==mathsy(5) {height of `\.x'} @d math←quad==mathsy(6) {\.{18mu}} @d num1==mathsy(8) {numerator shift-up in display styles} @d num2==mathsy(9) {numerator shift-up in non-display, non-\.{\\atop}} @d num3==mathsy(10) {numerator shift-up in non-display \.{\\atop}} @d denom1==mathsy(11) {denominator shift-down in display styles} @d denom2==mathsy(12) {denominator shift-down in non-display styles} @d sup1==mathsy(13) {superscript shift-up in uncramped display style} @d sup2==mathsy(14) {superscript shift-up in uncramped non-display} @d sup3==mathsy(15) {superscript shift-up in cramped styles} @d sub1==mathsy(16) {subscript shift-down if superscript is absent} @d sub2==mathsy(17) {subscript shift-down if superscript is present} @d sup←drop==mathsy(18) {superscript baseline below top of large box} @d sub←drop==mathsy(19) {subscript baseline below bottom of large box} @d delim1==mathsy(20) {size of \.{\\atopwithdelims} delimiters in display styles} @d delim2==mathsy(21) {size of \.{\\atopwithdelims} delimiters in non-displays} @d axis←height==mathsy(22) {height of fraction lines above the baseline} @d total←mathsy←params=22 @ The math-extension parameters have similar macros, but the size code is omitted (since it is always |cur←size| when we refer to such parameters). @↑parameters for symbols@> @↑font parameters@> @d mathex(#)==font←info[#+param←base[fam←fnt(3+cur←size)]].sc @d default←rule←thickness==mathex(8) {thickness of \.{\\over} bars} @d big←op←spacing1==mathex(9) {minimum clearance above a displayed op} @d big←op←spacing2==mathex(10) {minimum clearance below a displayed op} @d big←op←spacing3==mathex(11) {minimum baselineskip above displayed op} @d big←op←spacing4==mathex(12) {minimum baselineskip below displayed op} @d big←op←spacing5==mathex(13) {padding above and below displayed limits} @d total←mathex←params=13 @ We also need to compute the change in style between mlists and their subsidiaries. The following macros define the subsidiary style for an overlined nucleus (|cramped←style|), for a subscript or a superscript (|sub←style| or |sup←style|), or for a numerator or denominator (|num←style| or |denom←style|). @d cramped←style(#)==2*(# div 2)+cramped {cramp the style} @d sub←style(#)==2*(# div 4)+script←style+cramped {smaller and cramped} @d sup←style(#)==2*(# div 4)+script←style+(# mod 2) {smaller} @d num←style(#)==#+2-2*(# div 6) {smaller unless already script-script} @d denom←style(#)==2*(# div 2)+cramped+2-2*(# div 6) {smaller, cramped} @ When the style changes, the following piece of program computes associated information: @<Set up the values of |cur←size| and |cur←mu|, based on |cur←style|@>= begin if cur←style<script←style then cur←size:=text←size else cur←size:=16*((cur←style-text←style) div 2); cur←mu:=x←over←n(math←quad(cur←size),18); end @ Here is a function that returns a pointer to a rule node having a given thickness |t|. The rule will extend horizontally to the boundary of the vlist that eventually contains it. @p function fraction←rule(@!t:scaled):pointer; {construct the bar for a fraction} var p:pointer; {the new node} begin p:=new←rule; height(p):=t; depth(p):=0; fraction←rule:=p; end; @ The |overbar| function returns a pointer to a vlist box that consists of a given box |b|, above which has been placed a kern of height |k| under a fraction rule of thickness |t| under additional space of height |t|. @p function overbar(@!b:pointer;@!k,@!t:scaled):pointer; var p,@!q:pointer; {nodes being constructed} begin p:=new←kern(k); link(p):=b; q:=fraction←rule(t); link(q):=p; p:=new←kern(t); link(p):=q; overbar:=vpack(p,natural); end; @ The |var←delimiter| function, which finds or constructs a sufficiently large delimiter, is the most interesting of the auxiliary functions that currently concern us. Given a pointer |d| to a delimiter field in some noad, together with a size code |s| and a vertical distance |v|, this function returns a pointer to a box that contains the smallest variant of |d| whose height plus depth is |v| or more. (And if no variant is large enough, it returns the largest available variant.) In particular, this routine will construct arbitrarily large delimiters from extensible components, if |d| leads to such characters. The value returned is a box whose |shift←amount| has been set so that the box is vertically centered with respect to the axis in the given size. If a built-up symbol is returned, the height of the box before shifting will be the height of its topmost component. @p@t\4@>@<Declare subprocedures for |var←delimiter|@> function var←delimiter(@!d:pointer;@!s:small←number;@!v:scaled):pointer; label found,continue; var b:pointer; {the box that will be constructed} @!f,@!g: internal←font←number; {best-so-far and tentative font codes} @!c,@!x,@!y: quarterword; {best-so-far and tentative character codes} @!m,@!n: integer; {the number of extensible pieces} @!u: scaled; {height-plus-depth of a tentative character} @!w: scaled; {largest height-plus-depth so far} @!q: four←quarters; {character info} @!hd: eight←bits; {height-depth byte} @!r: four←quarters; {extensible pieces} @!z: small←number; {runs through font family members} @!large←attempt: boolean; {are we trying the ``large'' variant?} begin f:=null←font; w:=0; large←attempt:=false; z:=small←fam(d); x:=small←char(d); loop@+ begin @<Look at the variants of |(z,x)|; set |f| and |c| whenever a better character is found; |goto found| as soon as a large enough variant is encountered@>; if large←attempt then goto found; {there were none large enough} large←attempt:=true; z:=large←fam(d); x:=large←char(d); end; found: if f<>null←font then @<Make variable |b| point to a box for |(f,c)|@> else begin b:=new←null←box; width(b):=null←delimiter←space; {use this width if no delimiter was found} end; shift←amount(b):=half(height(b)-depth(b)) - axis←height(s); var←delimiter:=b; end; @ The search process is complicated slightly by the facts that some of the characters might not be present in some of the fonts, and they might not be probed in increasing order of height. @<Look at the variants of |(z,x)|; set |f| and |c|...@>= if (z<>0)or(x<>min←quarterword) then begin z:=z+s+16; repeat z:=z-16; g:=fam←fnt(z); if g<>null←font then @<Look at the list of characters starting with |x| in font |g|; set |f| and |c| whenever a better character is found; |goto found| as soon as a large enough variant is encountered@>; until z<16; end @ @<Look at the list of characters starting with |x|...@>= begin y:=x; continue: if (qo(y)>=font←bc[g])and(qo(y)<=font←ec[g]) then begin q:=char←info(g)(y); if char←exists(q) then begin if char←tag(q)=ext←tag then begin f:=g; c:=y; goto found; end; hd:=height←depth(q); u:=char←height(g)(hd)+char←depth(g)(hd); if u>w then begin f:=g; c:=y; w:=u; if u>=v then goto found; end; if char←tag(q)=list←tag then begin y:=rem←byte(q); goto continue; end; end; end; end @ Here is a subroutine that creates a new box, whose contents is a single character, and whose width includes the italic correction for that character. The height or depth of the box will be negative, if the height or depth of the character is negative; thus, this routine may deliver a slightly different result than |hpack| would produce. @<Declare subprocedures for |var←delimiter|@>= function char←box(@!f:internal←font←number;@!c:quarterword):pointer; var q:four←quarters; @!hd:eight←bits; {|height←depth| byte} @!b,@!p:pointer; {the new box and its character node} begin q:=char←info(f)(c); hd:=height←depth(q); b:=new←null←box; width(b):=char←width(f)(q)+char←italic(f)(q); height(b):=char←height(f)(hd); depth(b):=char←depth(f)(hd); p:=get←avail; character(p):=c; font(p):=f; list←ptr(b):=p; char←box:=b; end; @ When the following code is executed, |char←tag(q)| will be equal to |ext←tag| if and only if a built-up symbol is supposed to be returned. @<Make variable |b| point to a box for |(f,c)|@>= if char←tag(q)=ext←tag then @<Construct an extensible character in a new box |b|, using recipe |rem←byte(q)| and font |f|@> else b:=char←box(f,c) @ When we build an extensible character, it's handy to have the following subroutine, which puts a given character on top of the characters already in box |b|: @<Declare subprocedures for |var←delimiter|@>= procedure stack←into←box(@!b:pointer;@!f:internal←font←number; @!c:quarterword); var p:pointer; {new node placed into |b|} begin p:=char←box(f,c); link(p):=list←ptr(b); list←ptr(b):=p; height(b):=height(p); end; @ Another handy subroutine computes the height plus depth of a given character: @<Declare subprocedures for |var←delimiter|@>= function height←plus←depth(@!f:internal←font←number;@!c:quarterword):scaled; var q:four←quarters; @!hd:eight←bits; {|height←depth| byte} begin q:=char←info(f)(c); hd:=height←depth(q); height←plus←depth:=char←height(f)(hd)+char←depth(f)(hd); end; @ @<Construct an extensible...@>= begin b:=new←null←box; type(b):=vlist←node; r:=font←info[exten←base[f]+rem←byte(q)].qqqq;@/ @<Compute the minimum suitable height, |w|, and the corresponding number of extension steps, |n|; also set |width(b)|@>; c:=ext←bot(r); if c<>min←quarterword then stack←into←box(b,f,c); c:=ext←rep(r); for m:=1 to n do stack←into←box(b,f,c); c:=ext←mid(r); if c<>min←quarterword then begin stack←into←box(b,f,c); c:=ext←rep(r); for m:=1 to n do stack←into←box(b,f,c); end; c:=ext←top(r); if c<>min←quarterword then stack←into←box(b,f,c); depth(b):=w-height(b); end @ The width of an extensible character is the width of the repeatable module. If this module does not have positive height plus depth, we don't use any copies of it, otherwise we use as few as possible (in groups of two if there is a middle part). @<Compute the minimum suitable height, |w|, and...@>= c:=ext←rep(r); u:=height←plus←depth(f,c); w:=0; q:=char←info(f)(c); width(b):=char←width(f)(q)+char←italic(f)(q);@/ c:=ext←bot(r);@+if c<>min←quarterword then w:=w+height←plus←depth(f,c); c:=ext←mid(r);@+if c<>min←quarterword then w:=w+height←plus←depth(f,c); c:=ext←top(r);@+if c<>min←quarterword then w:=w+height←plus←depth(f,c); n:=0; if u>0 then while w<v do begin w:=w+u; incr(n); if ext←mid(r)<>min←quarterword then w:=w+u; end @ The next subroutine is much simpler; it is used for numerators and denominators of fractions as well as for displayed operators and their limits above and below. It takes a given hlist box |b| and changes it so that the new box is centered in a box of width |w|. The centering is done by putting \.{\\hss} glue at the left and right of the list inside |b|, then packaging the new box; thus, the actual box might not really be centered, if it already contains infinite glue. The given box might contain a single character whose italic correction has been added to the width of the box; in this case a compensating kern is inserted. @p function rebox(@!b:pointer;@!w:scaled):pointer; var p:pointer; {temporary register for list manipulation} @!f:internal←font←number; {font in a one-character box} @!v:scaled; {width of a character without italic correction} begin if (width(b)<>w)and(list←ptr(b)<>null) then begin if type(b)=vlist←node then b:=hpack(b,natural); p:=list←ptr(b); if (is←char←node(p))and(link(p)=null) then begin f:=font(p); v:=char←width(f)(char←info(f)(character(p))); if v<>width(b) then link(p):=new←kern(width(b)-v); end; free←node(b,box←node←size); b:=new←glue(ss←glue); link(b):=p; while link(p)<>null do p:=link(p); link(p):=new←glue(ss←glue); rebox:=hpack(b,w,exactly); end else begin width(b):=w; rebox:=b; end; end; @ Here is a subroutine that creates a new glue specification from another one that is expressed in `\.{mu}', given the value of the math unit. @d mu←mult(#)==nx←plus←y(n,#,xn←over←d(#,f,@'200000)) @p function math←glue(@!g:pointer;@!m:scaled):pointer; var p:pointer; {the new glue specification} @!n:integer; {integer part of |m|} @!f:scaled; {fraction part of |m|} begin n:=x←over←n(m,@'200000); f:=remainder;@/ p:=get←node(glue←spec←size); width(p):=mu←mult(width(g)); {convert \.{mu} to \.{pt}} stretch←order(p):=stretch←order(g); if stretch←order(p)=normal then stretch(p):=mu←mult(stretch(g)) else stretch(p):=stretch(g); shrink←order(p):=shrink←order(g); if shrink←order(p)=normal then shrink(p):=mu←mult(shrink(g)) else shrink(p):=shrink(g); math←glue:=p; end; @ The |math←kern| subroutine removes |mu←glue| from a kern node, given the value of the math unit. @p procedure math←kern(@!p:pointer;@!m:scaled); var @!n:integer; {integer part of |m|} @!f:scaled; {fraction part of |m|} begin if subtype(p)=mu←glue then begin n:=x←over←n(m,@'200000); f:=remainder;@/ width(p):=mu←mult(width(p)); subtype(p):=normal; end; end; @ Sometimes it is necessary to destroy an mlist. The following subroutine empties the current list, assuming that |abs(mode)=mmode|. @p procedure flush←math; begin flush←node←list(link(head)); flush←node←list(incompleat←noad); link(head):=null; tail:=head; incompleat←noad:=null; end; @* \[36] Typesetting math formulas. \TeX's most important routine for dealing with formulas is called |mlist←to←hlist|. After a formula has been scanned and represented as an mlist, this routine converts it to an hlist that can be placed into a box or incorporated into the text of a paragraph. There are three implicit parameters, passed in global variables: |cur←mlist| points to the first node or noad in the given mlist (and it might be |null|); |cur←style| is a style code; and |mlist←penalties| is |true| if penalty nodes for potential line breaks are to be inserted into the resulting hlist. After |mlist←to←hlist| has acted, |link(temp←head)| points to the translated hlist. Since mlists can be inside mlists, the procedure is recursive. And since this is not part of \TeX's inner loop, the program has been written in a manner that stresses compactness over efficiency. @↑recursion@> @<Glob...@>= @!cur←mlist:pointer; {beginning of mlist to be translated} @!cur←style:small←number; {style code at current place in the list} @!cur←size:small←number; {size code corresponding to |cur←style|} @!cur←mu:scaled; {the math unit width corresponding to |cur←size|} @!mlist←penalties:boolean; {should |mlist←to←hlist| insert penalties?} @ The recursion in |mlist←to←hlist| is due primarily to a subroutine called |clean←box| that puts a given noad field into a box using a given math style; |mlist←to←hlist| can call |clean←box|, which can call |mlist←to←hlist|. The box returned by |clean←box| is ``clean'' in the sense that its |shift←amount| is zero. @p procedure@?mlist←to←hlist; forward;@t\2@>@/ function clean←box(@!p:pointer;@!s:small←number):pointer; label found; var q:pointer; {beginning of a list to be boxed} @!save←style:small←number; {|cur←style| to be restored} @!x:pointer; {box to be returned} @!r:pointer; {temporary pointer} begin case math←type(p) of math←char: begin cur←mlist:=new←noad; mem[nucleus(cur←mlist)]:=mem[p]; end; sub←box: begin q:=info(p); goto found; end; sub←mlist: cur←mlist:=info(p); othercases begin q:=new←null←box; goto found; end endcases;@/ save←style:=cur←style; cur←style:=s; mlist←penalties:=false;@/ mlist←to←hlist; q:=link(temp←head); {recursive call} cur←style:=save←style; {restore the style} @<Set up the values of |cur←size| and |cur←mu|, based on |cur←style|@>; found: if is←char←node(q)or(q=null) then x:=hpack(q,natural) else if (link(q)=null)and(type(q)<=vlist←node)and(shift←amount(q)=0) then x:=q {it's already clean} else x:=hpack(q,natural); @<Simplify a trivial box@>; clean←box:=x; end; @ Here we save memory space in a common case. @<Simplify a trivial box@>= q:=list←ptr(x); if is←char←node(q) then begin r:=link(q); if r<>null then if link(r)=null then if not is←char←node(r) then if type(r)=kern←node then begin free←node(r,small←node←size); link(q):=null; end; end @ It is convenient to have a procedure that converts a |math←char| field to an ``unpacked'' form. The |fetch| routine sets |cur←f|, |cur←c|, and |cur←i| to the font code, character code, and character information bytes of a given noad field. It also takes care of issuing error messages for nonexistent characters; in such cases, |char←exists(cur←i)| will be |false| after |fetch| has acted, and the field will also have been reset to |empty|. @p procedure fetch(@!a:pointer); {unpack the |math←char| field |a|} begin cur←c:=character(a); cur←f:=fam←fnt(fam(a)+cur←size); if cur←f=null←font then @<Complain about an undefined family and set |cur←i| null@> else begin if (qo(cur←c)>=font←bc[cur←f])and(qo(cur←c)<=font←ec[cur←f]) then cur←i:=char←info(cur←f)(cur←c) else cur←i:=null←character; if not(char←exists(cur←i)) then begin char←warning(cur←f,qo(cur←c)); math←type(a):=empty; end; end; end; @ @<Complain about an undefined family...@>= begin print←err(""); print←size(cur←size); print←char(" "); print←int(fam(a)); print(" is undefined (character "); print←ASCII(qo(cur←c)); print←char(")"); help4("Somewhere in the math formula just ended, you used the")@/ ("stated character from an undefined font family. For example,")@/ ("plain TeX doesn't allow \it or \sl in subscripts. Proceed,")@/ ("and I'll try to forget that I needed that character."); error; cur←i:=null←character; math←type(a):=empty; end @ The outputs of |fetch| are placed in global variables. @<Glob...@>= @!cur←f:internal←font←number; {the |font| field of a |math←char|} @!cur←c:quarterword; {the |character| field of a |math←char|} @!cur←i:four←quarters; {the |char←info| of a |math←char|, or a lig/kern instruction} @ We need to do a lot of different things, so |mlist←to←hlist| makes two passes over the given mlist. The first pass does most of the processing: It removes ``mu'' spacing from glue, it recursively evaluates all subsidiary mlists so that only the top-level mlist remains to be handled, it puts fractions and square roots and such things into boxes, it attaches subscripts and superscripts, and it computes the overall height and depth of the top-level mlist so that the size of delimiters for a |left←noad| and a |right←noad| will be known. The hlist resulting from each noad is recorded in that noad's |new←hlist| field, an integer field that replaces the |nucleus| or |thickness|. The second pass eliminates all noads and inserts the correct glue and penalties between nodes. @d new←hlist(#)==mem[nucleus(#)].int {the translation of an mlist} @ Here is the overall plan of |mlist←to←hlist|, and the list of its local variables. @d done←with←noad=80 {go here when a noad has been fully translated} @d done←with←node=81 {go here when a node has been fully converted} @d check←dimensions=82 {go here to update |max←h| and |max←d|} @d delete←q=83 {go here to delete |q| and move to the next node} @p@t\4@>@<Declare math construction procedures@> procedure mlist←to←hlist; label reswitch, check←dimensions, done←with←noad, done←with←node, delete←q, done; var mlist:pointer; {beginning of the given list} @!penalties:boolean; {should penalty nodes be inserted?} @!style:small←number; {the given style} @!save←style:small←number; {holds |cur←style| during recursion} @!q:pointer; {runs through the mlist} @!r:pointer; {the most recent noad preceding |q|} @!r←type:small←number; {the |type| of noad |r|, or |op←noad| if |r=null|} @!t:small←number; {the effective |type| of noad |q| during the second pass} @!p,@!x,@!y,@!z: pointer; {temporary registers for list construction} @!pen:integer; {a penalty to be inserted} @!s:small←number; {the size of a noad to be deleted} @!max←h,@!max←d:scaled; {maximum height and depth of the list translated so far} @!delta:scaled; {offset between subscript and superscript} begin mlist:=cur←mlist; penalties:=mlist←penalties; style:=cur←style; {tuck global parameters away as local variables} q:=mlist; r:=null; r←type:=op←noad; max←h:=0; max←d:=0; @<Set up the values of |cur←size| and |cur←mu|, based on |cur←style|@>; while q<>null do @<Process node-or-noad |q| as much as possible in preparation for the second pass of |mlist←to←hlist|, then move to the next item in the mlist@>; @<Convert \(a)a final |bin←noad| to an |ord←noad|@>; @<Make a second pass over the mlist, removing all noads and inserting the proper spacing and penalties@>; end; @ We use the fact that no character nodes appear in an mlist, hence the field |type(q)| is always present. @<Process node-or-noad...@>= begin @<Do first-pass processing based on |type(q)|; |goto done←with←noad| if a noad has been fully processed, |goto check←dimensions| if it has been translated into |new←hlist(q)|, or |goto done←with←node| if a node has been fully processed@>; check←dimensions: z:=hpack(new←hlist(q),natural); if height(z)>max←h then max←h:=height(z); if depth(z)>max←d then max←d:=depth(z); free←node(z,box←node←size); done←with←noad: r:=q; r←type:=type(r); done←with←node: q:=link(q); end @ One of the things we must do on the first pass is change a |bin←noad| to an |ord←noad| if the |bin←noad| is not in the context of a binary operator. The values of |r| and |r←type| make this fairly easy. @<Do first-pass processing...@>= reswitch: delta:=0; case type(q) of bin←noad: case r←type of bin←noad,op←noad,rel←noad,open←noad,punct←noad,left←noad: begin type(q):=ord←noad; goto reswitch; end; othercases do←nothing endcases; rel←noad,close←noad,punct←noad,right←noad: begin@t@>@;@/ @<Convert \(a)a final |bin←noad| to an |ord←noad|@>; if type(q)=right←noad then goto done←with←noad; end; @t\4@>@<Cases for noads that can follow a |bin←noad|@>@; @t\4@>@<Cases for nodes that can appear in an mlist, after which we |goto done←with←node|@>@; othercases confusion("mlist1") @:this can't happen mlist1}{\quad mlist1@> endcases;@/ @<Convert \(n)|nucleus(q)| to an hlist and attach the sub/superscripts@> @ @<Convert \(a)a final |bin←noad| to an |ord←noad|@>= if r←type=bin←noad then type(r):=ord←noad @ @<Cases for nodes that can appear in an mlist...@>= style←node: begin cur←style:=subtype(q); @<Set up the values of |cur←size| and |cur←mu|, based on |cur←style|@>; goto done←with←node; end; choice←node: @<Change this node to a style node followed by the correct choice, then |goto done←with←node|@>; ins←node,mark←node,adjust←node, whatsit←node,penalty←node,disc←node: goto done←with←node; rule←node: begin if height(q)>max←h then max←h:=height(q); if depth(q)>max←d then max←d:=depth(q); goto done←with←node; end; glue←node: begin @<Convert \(m)math glue to ordinary glue@>; goto done←with←node; end; kern←node: begin math←kern(q,cur←mu); goto done←with←node; end; @ @d choose←mlist(#)==begin p:=#(q); #(q):=null;@+end @<Change this node to a style node...@>= begin case cur←style div 2 of 0: choose←mlist(display←mlist); {|display←style=0|} 1: choose←mlist(text←mlist); {|text←style=2|} 2: choose←mlist(script←mlist); {|script←style=4|} 3: choose←mlist(script←script←mlist); {|script←script←style=6|} end; {there are no other cases} flush←node←list(display←mlist(q)); flush←node←list(text←mlist(q)); flush←node←list(script←mlist(q)); flush←node←list(script←script←mlist(q));@/ type(q):=style←node; subtype(q):=cur←style; width(q):=0; depth(q):=0; if p<>null then begin z:=link(q); link(q):=p; while link(p)<>null do p:=link(p); link(p):=z; end; goto done←with←node; end @ Conditional math glue (`\.{\\nonscript}') results in a |glue←node| pointing to |zero←glue|, with |subtype(q)=cond←math←glue|; in such a case the node following will be eliminated if it is a glue or kern node and if the current size is different from |text←size|. Unconditional math glue (`\.{\\muskip}') is converted to normal glue by multiplying the dimensions by |cur←mu|. @<Convert \(m)math glue to ordinary glue@>= if subtype(q)=mu←glue then begin x:=glue←ptr(q); y:=math←glue(x,cur←mu); delete←glue←ref(x); glue←ptr(q):=y; subtype(q):=normal; end else if (cur←size<>text←size)and(subtype(q)=cond←math←glue) then begin p:=link(q); if p<>null then if (type(p)=glue←node)or(type(p)=kern←node) then begin link(q):=link(p); link(p):=null; flush←node←list(p); end; end @ @<Cases for noads that can follow a |bin←noad|@>= left←noad: goto done←with←noad; fraction←noad: begin make←fraction(q); goto check←dimensions; end; op←noad: begin delta:=make←op(q); if subtype(q)=limits then goto check←dimensions; end; ord←noad: make←ord(q); open←noad,inner←noad: do←nothing; radical←noad: make←radical(q); over←noad: make←over(q); under←noad: make←under(q); accent←noad: make←math←accent(q); vcenter←noad: make←vcenter(q); @ Most of the actual construction work of |mlist←to←hlist| is done by procedures like |make←fraction|, |make←radical|, etc. To illustrate the general setup of such procedures, let's begin with a couple of simple ones. @<Declare math...@>= procedure make←over(@!q:pointer); begin info(nucleus(q)):=@| overbar(clean←box(nucleus(q),cramped←style(cur←style)),@| 3*default←rule←thickness,default←rule←thickness); math←type(nucleus(q)):=sub←box; end; @ @<Declare math...@>= procedure make←under(@!q:pointer); var p,@!x,@!y: pointer; {temporary registers for box construction} @!delta:scaled; {overall height plus depth} begin x:=clean←box(nucleus(q),cur←style); p:=new←kern(3*default←rule←thickness); link(x):=p; link(p):=fraction←rule(default←rule←thickness); y:=vpack(x,natural); delta:=height(y)+depth(y)+default←rule←thickness; height(y):=height(x); depth(y):=delta-height(y); info(nucleus(q)):=y; math←type(nucleus(q)):=sub←box; end; @ @<Declare math...@>= procedure make←vcenter(@!q:pointer); var v:pointer; {the box that should be centered vertically} @!delta:scaled; {its height plus depth} begin v:=info(nucleus(q)); if type(v)<>vlist←node then confusion("vcenter"); @:this can't happen vcenter}{\quad vcenter@> delta:=height(v)+depth(v); height(v):=axis←height(cur←size)+half(delta); depth(v):=delta-height(v); end; @ According to the rules in the \.{DVI} file specifications, we ensure alignment @↑square roots@> between a square root sign and the rule above its nucleus by assuming that the baseline of the square-root symbol is the same as the bottom of the rule. The height of the square-root symbol will be the thickness of the rule, and the depth of the square-root symbol should exceed or equal the height-plus-depth of the nucleus plus a certain minimum clearance~|clr|. The symbol will be placed so that the actual clearance is |clr| plus half the excess. @<Declare math...@>= procedure make←radical(@!q:pointer); var x,@!y:pointer; {temporary registers for box construction} @!delta,@!clr:scaled; {dimensions involved in the calculation} begin x:=clean←box(nucleus(q),cramped←style(cur←style)); if cur←style<text←style then {display style} clr:=default←rule←thickness+(abs(math←x←height(cur←size)) div 4) else begin clr:=default←rule←thickness; clr:=clr + (abs(clr) div 4); end; y:=var←delimiter(left←delimiter(q),cur←size,height(x)+depth(x)+clr+ default←rule←thickness); delta:=depth(y)-(height(x)+depth(x)+clr); if delta>0 then clr:=clr+half(delta); {increase the actual clearance} shift←amount(y):=-(height(x)+clr); link(y):=overbar(x,clr,height(y)); info(nucleus(q)):=hpack(y,natural); math←type(nucleus(q)):=sub←box; end; @ Slants are not considered when placing accents in math mode. The accenter is centered over the accentee, and the accent width is treated as zero with respect to the size of the final box. @<Declare math...@>= procedure make←math←accent(@!q:pointer); label done,done1; var p,@!x,@!y:pointer; {temporary registers for box construction} @!a:integer; {address of lig/kern instruction} @!c:quarterword; {accent character} @!f:internal←font←number; {its font} @!i:four←quarters; {its |char←info|} @!s:scaled; {amount to skew the accent to the right} @!h:scaled; {height of character being accented} @!delta:scaled; {space to remove between accent and accentee} @!w:scaled; {width of the accentee, not including sub/superscripts} begin fetch(accent←chr(q)); if char←exists(cur←i) then begin i:=cur←i; c:=cur←c; f:=cur←f;@/ @<Compute the amount of skew@>; x:=clean←box(nucleus(q),cramped←style(cur←style)); w:=width(x); h:=height(x); @<Switch to a larger accent if available and appropriate@>; if h<x←height(f) then delta:=h@+else delta:=x←height(f); if (math←type(supscr(q))<>empty)or(math←type(subscr(q))<>empty) then if math←type(nucleus(q))=math←char then @<Swap the subscript and superscript into box |x|@>; y:=char←box(f,c); shift←amount(y):=s+half(w-width(y)); width(y):=0; p:=new←kern(-delta); link(p):=x; link(y):=p; y:=vpack(y,natural); width(y):=width(x); if height(y)<h then @<Make the height of box |y| equal to |h|@>; info(nucleus(q)):=y; math←type(nucleus(q)):=sub←box; end; end; @ @<Make the height of box |y|...@>= begin p:=new←kern(h-height(y)); link(p):=list←ptr(y); list←ptr(y):=p; height(y):=h; end @ @<Switch to a larger accent if available and appropriate@>= loop@+ begin if char←tag(i)<>list←tag then goto done; y:=rem←byte(i); i:=char←info(f)(y); if char←width(f)(i)>w then goto done; c:=y; end; done: @ @<Compute the amount of skew@>= s:=0; if math←type(nucleus(q))=math←char then begin fetch(nucleus(q)); if char←tag(cur←i)=lig←tag then begin a:=lig←kern←start(cur←f)(cur←i); repeat cur←i:=font←info[a].qqqq; if qo(next←char(cur←i))=skew←char[cur←f] then begin if op←bit(cur←i)>=kern←flag then s:=char←kern(cur←f)(cur←i); goto done1; end; incr(a); until stop←bit(cur←i)>=stop←flag; end; end; done1: @ @<Swap the subscript and superscript into box |x|@>= begin flush←node←list(x); x:=new←noad; mem[nucleus(x)]:=mem[nucleus(q)]; mem[supscr(x)]:=mem[supscr(q)]; mem[subscr(x)]:=mem[subscr(q)];@/ mem[supscr(q)].hh:=empty←field; mem[subscr(q)].hh:=empty←field;@/ math←type(nucleus(q)):=sub←mlist; info(nucleus(q)):=x; x:=clean←box(nucleus(q),cur←style); delta:=delta+height(x)-h; h:=height(x); end @ The |make←fraction| procedure is a bit different because it sets |new←hlist(q)| directly rather than making a sub-box. @<Declare math...@>= procedure make←fraction(@!q:pointer); var p,@!v,@!x,@!y,@!z:pointer; {temporary registers for box construction} @!delta,@!delta1,@!delta2,@!shift←up,@!shift←down,@!clr:scaled; {dimensions for box calculations} begin if thickness(q)=default←code then thickness(q):=default←rule←thickness; @<Create equal-width boxes |x| and |z| for the numerator and denominator, and compute the default amounts |shift←up| and |shift←down| by which they are displaced from the baseline@>; if thickness(q)=0 then @<Adjust \(s)|shift←up| and |shift←down| for the case of no fraction line@> else @<Adjust \(s)|shift←up| and |shift←down| for the case of a fraction line@>; @<Construct a vlist box for the fraction, according to |shift←up| and |shift←down|@>; @<Put the \(f)fraction into a box with its delimiters, and make |new←hlist(q)| point to it@>; end; @ @<Create equal-width boxes |x| and |z| for the numerator and denom...@>= x:=clean←box(numerator(q),num←style(cur←style)); z:=clean←box(denominator(q),denom←style(cur←style)); if width(x)<width(z) then x:=rebox(x,width(z)) else z:=rebox(z,width(x)); if cur←style<text←style then {text style} begin shift←up:=num1(cur←size); shift←down:=denom1(cur←size); end else begin shift←down:=denom2(cur←size); if thickness(q)<>0 then shift←up:=num2(cur←size) else shift←up:=num3(cur←size); end @ The numerator and denominator must be separated by a certain minimum clearance, called |clr| in the following program. The difference between |clr| and the actual clearance is |2delta|. @<Adjust \(s)|shift←up| and |shift←down| for the case of no fraction line@>= begin if cur←style<text←style then clr:=7*default←rule←thickness else clr:=3*default←rule←thickness; delta:=half(clr-((shift←up-depth(x))-(height(z)-shift←down))); if delta>0 then begin shift←up:=shift←up+delta; shift←down:=shift←down+delta; end; end @ In the case of a fraction line, the minimum clearance depends on the actual thickness of the line. @<Adjust \(s)|shift←up| and |shift←down| for the case of a fraction line@>= begin if cur←style<text←style then clr:=3*thickness(q) else clr:=thickness(q); delta:=half(thickness(q)); delta1:=clr-((shift←up-depth(x))-(axis←height(cur←size)+delta)); delta2:=clr-((axis←height(cur←size)-delta)-(height(z)-shift←down)); if delta1>0 then shift←up:=shift←up+delta1; if delta2>0 then shift←down:=shift←down+delta2; end @ @<Construct a vlist box for the fraction...@>= v:=new←null←box; type(v):=vlist←node; height(v):=shift←up+height(x); depth(v):=depth(z)+shift←down; width(v):=width(x); {this also equals |width(z)|} if thickness(q)=0 then begin p:=new←kern((shift←up-depth(x))-(height(z)-shift←down)); link(p):=z; end else begin y:=fraction←rule(thickness(q));@/ p:=new←kern((axis←height(cur←size)-delta)-@|(height(z)-shift←down));@/ link(y):=p; link(p):=z;@/ p:=new←kern((shift←up-depth(x))-(axis←height(cur←size)+delta)); link(p):=y; end; link(x):=p; list←ptr(v):=x @ @<Put the \(f)fraction into a box with its delimiters...@>= if cur←style<text←style then delta:=delim1(cur←size) else delta:=delim2(cur←size); x:=var←delimiter(left←delimiter(q), cur←size, delta); link(x):=v;@/ z:=var←delimiter(right←delimiter(q), cur←size, delta); link(v):=z;@/ new←hlist(q):=hpack(x,natural) @ If the nucleus of an |op←noad| is a single character, it is to be centered vertically with respect to the axis, after first being enlarged (via a character list in the font) if we are in display style. The normal convention for placing displayed limits is to put them above and below the operator in display style. The italic correction is removed from the character if there is a subscript and the limits are not being displayed. The |make←op| routine returns the value that should be used as an offset between subscript and superscript. After |make←op| has acted, |subtype(q)| will be |limits| if and only if the limits have been set above and below the operator. In that case, |new←hlist(q)| will already contain the desired final box. @<Declare math...@>= function make←op(@!q:pointer):scaled; var delta:scaled; {offset between subscript and superscript} @!p,@!v,@!x,@!y,@!z:pointer; {temporary registers for box construction} @!shift←up,@!shift←down:scaled; {dimensions for box calculation} begin if (subtype(q)=normal)and(cur←style<text←style) then subtype(q):=limits; if math←type(nucleus(q))=math←char then begin fetch(nucleus(q)); if (cur←style<text←style)and(char←tag(cur←i)=list←tag) then {make it larger} begin cur←c:=rem←byte(cur←i); character(nucleus(q)):=cur←c; cur←i:=char←info(cur←f)(cur←c); end; delta:=char←italic(cur←f)(cur←i); x:=clean←box(nucleus(q),cur←style); if (math←type(subscr(q))<>empty)and(subtype(q)<>limits) then width(x):=width(x)-delta; {remove italic correction} shift←amount(x):=half(height(x)-depth(x)) - axis←height(cur←size); {center vertically} math←type(nucleus(q)):=sub←box; info(nucleus(q)):=x; end else delta:=0; if subtype(q)=limits then @<Construct a box with limits above and below it, skewed by |delta|@>; make←op:=delta; end; @ The following program builds a vlist box |v| for displayed limits. The width of the box is not affected by the fact that the limits may be skewed. @<Construct a box with limits above and below it...@>= begin x:=clean←box(supscr(q),sup←style(cur←style)); y:=clean←box(nucleus(q),cur←style); z:=clean←box(subscr(q),sub←style(cur←style)); v:=new←null←box; type(v):=vlist←node; width(v):=width(y); if width(x)>width(v) then width(v):=width(x); if width(z)>width(v) then width(v):=width(z); x:=rebox(x,width(v)); y:=rebox(y,width(v)); z:=rebox(z,width(v));@/ shift←amount(x):=half(delta); shift←amount(z):=-shift←amount(x); height(v):=height(y); depth(v):=depth(y); @<Attach the limits to |y| and adjust |height(v)|, |depth(v)| to account for their presence@>; new←hlist(q):=v; end @ We use |shift←up| and |shift←down| in the following program for the amount of glue between the displayed operator |y| and its limits |x| and |z|. The vlist inside box |v| will consist of |x| followed by |y| followed by |z|, with kern nodes for the spaces between and around them. @<Attach the limits to |y| and adjust |height(v)|, |depth(v)|...@>= if math←type(supscr(q))=empty then begin free←node(x,box←node←size); list←ptr(v):=y; end else begin shift←up:=big←op←spacing3-depth(x); if shift←up<big←op←spacing1 then shift←up:=big←op←spacing1; p:=new←kern(shift←up); link(p):=y; link(x):=p;@/ p:=new←kern(big←op←spacing5); link(p):=x; list←ptr(v):=p; height(v):=height(v)+big←op←spacing5+height(x)+depth(x)+shift←up; end; if math←type(subscr(q))=empty then free←node(z,box←node←size) else begin shift←down:=big←op←spacing4-height(z); if shift←down<big←op←spacing2 then shift←down:=big←op←spacing2; p:=new←kern(shift←down); link(y):=p; link(p):=z;@/ p:=new←kern(big←op←spacing5); link(z):=p; depth(v):=depth(v)+big←op←spacing5+height(z)+depth(z)+shift←down; end @ A ligature found in a math formula does not create a |ligature←node|, because there is no question of hyphenation afterwards; the ligature will simply be stored in an ordinary |char←node|, after residing in an |ord←noad|. The |math←type| is converted to |math←text←char| here if we would not want to apply an italic correction to the current character unless it belongs to a math font (i.e., a font with |space=0|). @<Declare math...@>= procedure make←ord(@!q:pointer); label restart,exit; var a:integer; {address of lig/kern instruction} @!p:pointer; {temporary register for list manipulation} begin restart:@t@>@;@/ if (math←type(subscr(q))=empty)and(math←type(supscr(q))=empty)and@| (math←type(nucleus(q))=math←char) then begin p:=link(q); if p<>null then if (type(p)>=ord←noad)and(type(p)<=punct←noad) then if math←type(nucleus(p))=math←char then if fam(nucleus(p))=fam(nucleus(q)) then begin math←type(nucleus(q)):=math←text←char; fetch(nucleus(q)); if char←tag(cur←i)=lig←tag then begin a:=lig←kern←start(cur←f)(cur←i); cur←c:=character(nucleus(p)); repeat cur←i:=font←info[a].qqqq;@/ @<If instruction |cur←i| is a kern with |cur←c|, attach the kern after |q| and |return|; or if it is a ligature with |cur←c|, combine noads |q| and |p| and |goto restart|@>; incr(a); until stop←bit(cur←i)>=stop←flag; end; end; end; exit:end; @ Note that a ligature between an |ord←noad| and another kind of noad is replaced by an |ord←noad|. Presumably a font designer will define such ligatures only when this convention makes sense. @<If instruction |cur←i| is a kern with |cur←c|, ...@>= if next←char(cur←i)=cur←c then if op←bit(cur←i)>=kern←flag then begin p:=new←kern(char←kern(cur←f)(cur←i)); link(p):=link(q); link(q):=p; return; end else begin link(q):=link(p); math←type(nucleus(q)):=math←char; character(nucleus(q)):=rem←byte(cur←i);@/ mem[subscr(q)]:=mem[subscr(p)]; mem[supscr(q)]:=mem[supscr(p)]; free←node(p,noad←size); goto restart; end @ When we get to the following part of the program, we have ``fallen through'' from cases that did not lead to |check←dimensions| or |done←with←noad| or |done←with←node|. Thus, |q|~points to a noad whose nucleus may need to be converted to an hlist, and whose subscripts and superscripts need to be appended if they are present. If |nucleus(q)| is not a |math←char|, the variable |delta| is the amount by which a superscript should be moved right with respect to a subscript when both are present. @↑subscripts@> @↑superscripts@> @<Convert \(n)|nucleus(q)| to an hlist and attach the sub/superscripts@>= case math←type(nucleus(q)) of math←char, math←text←char: @<Create a character node |p| for |nucleus(q)|, possibly followed by a kern node for the italic correction, and set |delta| to the italic correction if a subscript is present@>; empty: p:=null; sub←box: p:=info(nucleus(q)); sub←mlist: begin cur←mlist:=info(nucleus(q)); save←style:=cur←style; mlist←penalties:=false; mlist←to←hlist; {recursive call} cur←style:=save←style; @<Set up the values...@>; p:=hpack(link(temp←head),natural); end; othercases confusion("mlist2") @:this can't happen mlist2}{\quad mlist2@> endcases;@/ new←hlist(q):=p; if (math←type(subscr(q))=empty)and(math←type(supscr(q))=empty) then goto check←dimensions; make←scripts(q,delta) @ @<Create a character node |p| for |nucleus(q)|...@>= begin fetch(nucleus(q)); if char←exists(cur←i) then begin delta:=char←italic(cur←f)(cur←i); p:=new←character(cur←f,qo(cur←c)); if (math←type(nucleus(q))=math←text←char)and(space(cur←f)<>0) then delta:=0; {no italic correction in mid-word of text font} if (math←type(subscr(q))=empty)and(delta<>0) then begin link(p):=new←kern(delta); delta:=0; end; end else p:=null; end @ The purpose of |make←scripts(q,delta)| is to attach the subscript and/or superscript of noad |q| to the list that starts at |new←hlist(q)|, given that subscript and superscript aren't both empty. The superscript will appear to the right of the subscript by a given distance |delta|. We set |shift←down| and |shift←up| to the minimum amounts to shift the baseline of subscripts and superscripts based on the given nucleus. @<Declare math...@>= procedure make←scripts(@!q:pointer;@!delta:scaled); var p,@!x,@!y,@!z:pointer; {temporary registers for box construction} @!shift←up,@!shift←down,@!clr:scaled; {dimensions in the calculation} @!t:small←number; {subsidiary size code} begin p:=new←hlist(q); if is←char←node(p) then begin shift←up:=0; shift←down:=0; end else begin z:=hpack(p,natural); if cur←style<script←style then t:=script←size@+else t:=script←script←size; shift←up:=height(z)-sup←drop(t); shift←down:=depth(z)+sub←drop(t); free←node(z,box←node←size); end; if math←type(supscr(q))=empty then @<Construct a subscript box |x| when there is no superscript@> else begin @<Construct a superscript box |x|@>; if math←type(subscr(q))=empty then shift←amount(x):=-shift←up else @<Construct a sub/superscript combination box |x|, with the superscript offset by |delta|@>; end; if new←hlist(q)=null then new←hlist(q):=x else begin p:=new←hlist(q); while link(p)<>null do p:=link(p); link(p):=x; end; end; @ When there is a subscript without a superscript, the top of the subscript should not exceed the baseline plus four-fifths of the x-height. @<Construct a subscript box |x| when there is no superscript@>= begin x:=clean←box(subscr(q),sub←style(cur←style)); width(x):=width(x)+script←space; if shift←down<sub1(cur←size) then shift←down:=sub1(cur←size); clr:=height(x)-(abs(math←x←height(cur←size)*4) div 5); if shift←down<clr then shift←down:=clr; shift←amount(x):=shift←down; end @ The bottom of a superscript should never descend below the baseline plus one-fourth of the x-height. @<Construct a superscript box |x|@>= begin x:=clean←box(supscr(q),sup←style(cur←style)); width(x):=width(x)+script←space; if odd(cur←style) then clr:=sup3(cur←size) else if cur←style<text←style then clr:=sup1(cur←size) else clr:=sup2(cur←size); if shift←up<clr then shift←up:=clr; clr:=depth(x)+(abs(math←x←height(cur←size)) div 4); if shift←up<clr then shift←up:=clr; end @ When both subscript and superscript are present, the subscript must be separated from the superscript by at least four times |default←rule←thickness|. If this condition would be violated, the subscript moves down, after which both subscript and superscript move up so that the bottom of the superscript is at least as high as the baseline plus four-fifths of the x-height. @<Construct a sub/superscript combination box |x|...@>= begin y:=clean←box(subscr(q),sub←style(cur←style)); width(y):=width(y)+script←space; if shift←down<sub2(cur←size) then shift←down:=sub2(cur←size); clr:=4*default←rule←thickness- ((shift←up-depth(x))-(height(y)-shift←down)); if clr>0 then begin shift←down:=shift←down+clr; clr:=(abs(math←x←height(cur←size)*4) div 5)-(shift←up-depth(x)); if clr>0 then begin shift←up:=shift←up+clr; shift←down:=shift←down-clr; end; end; shift←amount(x):=delta; {superscript is |delta| to the right of the subscript} p:=new←kern((shift←up-depth(x))-(height(y)-shift←down)); link(x):=p; link(p):=y; x:=vpack(x,natural); shift←amount(x):=shift←down; end @ We have now tied up all the loose ends of the first pass of |mlist←to←hlist|. The second pass simply goes through and hooks everything together with the proper glue and penalties. It also handles the |left←noad| and |right←noad| that might be present, since |max←h| and |max←d| are now known. Variable |p| points to a node at the current end of the final hlist. @<Make a second pass over the mlist, ...@>= p:=temp←head; link(p):=null; q:=mlist; r←type:=0; cur←style:=style; @<Set up the values of |cur←size| and |cur←mu|, based on |cur←style|@>; while q<>null do begin @<If node |q| is a style node, change the style and |goto delete←q|; otherwise if it is not a noad, put it into the hlist, advance |q|, and |goto done|; otherwise set |s| to the size of noad |q|, set |t| to the associated type (|ord←noad.. inner←noad|), and set |pen| to the associated penalty@>; @<Append inter-element spacing based on |r←type| and |t|@>; @<Append any |new←hlist| entries for |q|, and any appropriate penalties@>; r←type:=t; delete←q: r:=q; q:=link(q); free←node(r,s); done: end @ Just before doing the big |case| switch in the second pass, the program sets up default values so that most of the branches are short. @<If node |q| is a style node, change the style...@>= t:=ord←noad; s:=noad←size; pen:=inf←penalty; case type(q) of op←noad,open←noad,close←noad,punct←noad,inner←noad: t:=type(q); bin←noad: begin t:=bin←noad; pen:=bin←op←penalty; end; rel←noad: begin t:=rel←noad; pen:=rel←penalty; end; ord←noad,vcenter←noad,over←noad,under←noad: do←nothing; radical←noad: s:=radical←noad←size; accent←noad: s:=accent←noad←size; fraction←noad: begin t:=inner←noad; s:=fraction←noad←size; end; left←noad,right←noad: t:=make←left←right(q,style,max←d,max←h); style←node: @<Change the current style and |goto delete←q|@>; whatsit←node,penalty←node,rule←node,disc←node,adjust←node,ins←node,mark←node, glue←node,kern←node:@t@>@;@/ begin link(p):=q; p:=q; q:=link(q); link(p):=null; goto done; end; othercases confusion("mlist3") @:this can't happen mlist3}{\quad mlist3@> endcases @ The |make←left←right| function constructs a left or right delimiter of the required size and returns the value |open←noad| or |close←noad|. The |right←noad| and |left←noad| will both be based on the original |style|, so they will have consistent sizes. We use the fact that |right←noad-left←noad=close←noad-open←noad|. @<Declare math...@>= function make←left←right(@!q:pointer;@!style:small←number; @!max←d,@!max←h:scaled):small←number; var delta,@!delta1,@!delta2:scaled; {dimensions used in the calculation} begin if style<script←style then cur←size:=text←size else cur←size:=16*((style-text←style) div 2); delta2:=max←d+axis←height(cur←size); delta1:=max←h+max←d-delta2; if delta2>delta1 then delta1:=delta2; {|delta1| is max distance from axis} delta:=(delta1 div 500)*delimiter←factor; delta2:=delta1+delta1-delimiter←shortfall; if delta<delta2 then delta:=delta2; new←hlist(q):=var←delimiter(delimiter(q),cur←size,delta); make←left←right:=type(q)-(left←noad-open←noad); {|open←noad| or |close←noad|} end; @ @<Change the current style and |goto delete←q|@>= begin cur←style:=subtype(q); s:=style←node←size; @<Set up the values of |cur←size| and |cur←mu|, based on |cur←style|@>; goto delete←q; end @ The inter-element spacing in math formulas depends on a $8\times8$ table that \TeX\ preloads as a 64-digit string. The elements of this string have the following significance: $$\vbox{\halign{#\hfil\cr \.0 means no space;\cr \.1 means a conditional thin space (\.{\\nonscript\\mskip\\thinmuskip});\cr \.2 means a thin space (\.{\\mskip\\thinmuskip});\cr \.3 means a conditional medium space (\.{\\nonscript\\mskip\\medmuskip});\cr \.4 means a conditional thick space (\.{\\nonscript\\mskip\\thickmuskip});\cr \.* means an impossible case.\cr}}$$ This is all pretty cryptic, but {\sl The \TeX book\/} explains what is supposed to happen, and the string makes it happen. @:TeXbook}{\sl The \TeX book@> A global variable |magic←offset| is computed so that if |a| and |b| are in the range |ord←noad..inner←noad|, then |str←pool[a*8+b+magic←offset]| is the digit for spacing between noad types |a| and |b|. If \PASCAL\ had provided a good way to preload constant arrays, this part of the program would not have been so strange. @:PASCAL}{\PASCAL@> @d math←spacing=@;@/ @t\hskip-35pt@> "0234000122*4000133**3**344*0400400*000000234000111*1111112341011" @t$ \hskip-35pt$@> @<Glob...@>= @!magic←offset:integer; {used to find inter-element spacing} @ @<Compute the magic offset@>= magic←offset:=str←start[math←spacing]-9*ord←noad @ @<Append inter-element spacing based on |r←type| and |t|@>= if r←type>0 then {not the first noad} begin case str←pool[r←type*8+t+magic←offset] of "0": x:=0; "1": if cur←style<script←style then x:=thin←mu←skip←code@+else x:=0; "2": x:=thin←mu←skip←code; "3": if cur←style<script←style then x:=med←mu←skip←code@+else x:=0; "4": if cur←style<script←style then x:=thick←mu←skip←code@+else x:=0; othercases confusion("mlist4") @:this can't happen mlist4}{\quad mlist4@> endcases; if x<>0 then begin y:=math←glue(glue←par(x),cur←mu); z:=new←glue(y); glue←ref←count(y):=null; link(p):=z; p:=z;@/ subtype(z):=x+1; {store a symbolic subtype} end; end @ We insert a penalty node after the hlist entries of noad |q| if |pen| is not an ``infinite'' penalty, and if the node immediately following |q| is not a penalty node or a |rel←noad| or absent entirely. @<Append any |new←hlist| entries for |q|, and any appropriate penalties@>= if new←hlist(q)<>null then begin link(p):=new←hlist(q); repeat p:=link(p); until link(p)=null; end; if penalties then if link(q)<>null then if pen<inf←penalty then begin r←type:=type(link(q)); if r←type<>penalty←node then if r←type<>rel←noad then begin z:=new←penalty(pen); link(p):=z; p:=z; end; end @* \[37] Alignment. It's sort of a miracle whenever \.{\\halign} and \.{\\valign} work, because they cut across so many of the control structures of \TeX. Therefore the present page is probably not the best place for a beginner to start reading this program; it is better to master everything else first. Let us focus our thoughts on an example of what the input might be, in order to get some idea about how the alignment miracle happens. The example doesn't do anything useful, but it is sufficiently general to indicate all of the special cases that must be dealt with; please do not be disturbed by its apparent complexity and meaninglessness. $$\vbox{\halign{\.{#}\hfil\cr {}\\tabskip 2pt plus 3pt\cr {}\\halign to 300pt\{u1\#v1\&\cr \hskip 50pt\\tabskip 1pt plus 1fil u2\#v2\&\cr \hskip 50pt u3\#v3\\cr\cr \hskip 25pt a1\&\\omit a2\&\\vrule\\cr\cr \hskip 25pt \\noalign\{\\vskip 3pt\}\cr \hskip 25pt b1\\span b2\\cr\cr \hskip 25pt \\omit\&c2\\span\\omit\\cr\}\cr}}$$ Here's what happens: \yskip (0) When `\.{\\halign to 300pt\{}' is scanned, the |scan←spec| routine places the 300pt dimension onto the |save←stack|, and an |align←group| code is placed above it. This will make it possible to complete the alignment when the matching `\.\}' is found. (1) The preamble is scanned next. Macros in the preamble are not expanded, @↑preamble@> except as part of a tabskip specification. For example, if \.{u2} had been a macro in the preamble above, it would have been expanded, since \TeX\ must look for `\.{minus...}' as part of the tabskip glue. A ``preamble list'' is constructed based on the user's preamble; in our case it contains the following seven items: $$\vbox{\halign{\.{#}\hfil\qquad&(#)\hfil\cr {}\\glue 2pt plus 3pt&the tabskip preceding column 1\cr {}\\alignrecord, width $-\infty$&preamble info for column 1\cr {}\\glue 2pt plus 3pt&the tabskip between columns 1 and 2\cr {}\\alignrecord, width $-\infty$&preamble info for column 2\cr {}\\glue 1pt plus 1fil&the tabskip between columns 2 and 3\cr {}\\alignrecord, width $-\infty$&preamble info for column 3\cr {}\\glue 1pt plus 1fil&the tabskip following column 3\cr}}$$ These ``alignrecord'' entries have the same size as an |unset←node|, since they will later be converted into such nodes. However, at the moment they have no |type| or |subtype| fields; they have |info| fields instead, and these |info| fields are initially set to the value |end←span|, for reasons explained below. Furthermore, the alignrecord nodes have no |height| or |depth| fields; these are renamed |u←part| and |v←part|, and they point to token lists for the templates of the alignment. For example, the |u←part| field in the first alignrecord points to the token list `\.{u1}', i.e., the template preceding the `\.\#' for column~1. (2) \TeX\ now looks at what follows the \.{\\cr} that ended the preamble. It is not `\.{\\noalign}' or `\.{\\omit}', so this input is put back to be read again, and the template `\.{u1}' is fed to the scanner. Just before reading `\.{u1}', \TeX\ goes into restricted horizontal mode. Just after reading `\.{u1}', \TeX\ will see `\.{a1}', and then (when the {\.\&} is sensed) \TeX\ will see `\.{v1}'. Then \TeX\ scans an |endv| token, indicating the end of a column. At this point an |unset←node| is created, containing the contents of the current hlist (i.e., `\.{u1a1v1}'). The natural width of this unset node replaces the |width| field of the alignrecord for column~1; in general, the alignrecords will record the maximum natural width that has occurred so far in a given column. (3) Since `\.{\\omit}' follows the `\.\&', the templates for column~2 are now bypassed. Again \TeX\ goes into restricted horizontal mode and makes an |unset←node| from the resulting hlist; but this time the hlist contains simply `\.{a2}'. The natural width of the new unset box is remembered in the |width| field of the alignrecord for column~2. (4) A third |unset←node| is created for column 3, using essentially the mechanism that worked for column~1; this unset box contains `\.{u3\\vrule v3}'. The vertical rule in this case has running dimensions that will later extend to the height and depth of the whole first row, since each |unset←node| in a row will eventually inherit the height and depth of its enclosing box. (5) The first row has now ended; it is made into a single unset box comprising the following seven items: $$\vbox{\halign{\hbox to 325pt{\qquad\.{#}\hfil}\cr {}\\glue 2pt plus 3pt\cr {}\\unsetbox for 1 column: u1a1v1\cr {}\\glue 2pt plus 3pt\cr {}\\unsetbox for 1 column: a2\cr {}\\glue 1pt plus 1fil\cr {}\\unsetbox for 1 column: u3\vrule v3\cr {}\\glue 1pt plus 1fil\cr}}$$ The width of this unset row is unimportant, but it has the correct height and depth, so the correct baselineskip glue will be computed as the row is inserted into a vertical list. (6) Since `\.{\\noalign}' follows the current \.{\\cr}, \TeX\ appends additional material (in this case \.{\\vskip 3pt}) to the vertical list. While processing this material, \TeX\ will be in internal vertical mode, and |no←align←group| will be on |save←stack|. (7) The next row produces an unset box that looks like this: $$\vbox{\halign{\hbox to 325pt{\qquad\.{#}\hfil}\cr {}\\glue 2pt plus 3pt\cr {}\\unsetbox for 2 columns: u1b1v1u2b2v2\cr {}\\glue 1pt plus 1fil\cr {}\\unsetbox for 1 column: {\rm(empty)}\cr {}\\glue 1pt plus 1fil\cr}}$$ The natural width of the unset box that spans columns 1~and~2 is stored in a ``span node,'' which we will explain later; the |info| field of the alignrecord for column~1 now points to the new span node, and the |info| of the span node points to |end←span|. (8) The final row produces the unset box $$\vbox{\halign{\hbox to 325pt{\qquad\.{#}\hfil}\cr {}\\glue 2pt plus 3pt\cr {}\\unsetbox for 1 column: {\rm(empty)}\cr {}\\glue 2pt plus 3pt\cr {}\\unsetbox for 2 columns: u2c2v2\cr {}\\glue 1pt plus 1fil\cr}}$$ A new span node is attached to the alignrecord for column 2. (9) The last step is to compute the true column widths and to change all the unset boxes to hboxes, appending the whole works to the vertical list that encloses the \.{\\halign}. The rules for deciding on the final widths of each unset column box will be explained below. \yskip\noindent Note that as \.{\\halign} is being processed, we fearlessly give up control to the rest of \TeX. At critical junctures, an alignment routine is called upon to step in and do some little action, but most of the time these routines just lurk in the background. It's something like post-hypnotic suggestion. @ We have mentioned that alignrecords contain no |height| or |depth| fields. Their |glue←sign| and |glue←order| are pre-empted as well, since it is necessary to store information about what to do when a template ends. This information is called the |extra←info| field. @d u←part(#)==mem[#+height←offset].int {pointer to \<u←j> token list} @d v←part(#)==mem[#+depth←offset].int {pointer to \<v←j> token list} @d extra←info(#)==info(#+list←offset) {info to remember during template} @ Alignments can occur within alignments, so a small stack is used to access the alignrecord information. At each level we have a |preamble| pointer, indicating the beginning of the preamble list; a |cur←align| pointer, indicating the current position in the preamble list; a |cur←span| pointer, indicating the value of |cur←align| at the beginning of a sequence of spanned columns; a |cur←loop| pointer, indicating the tabskip glue before an alignrecord that should be copied next if the current list is extended; and the |align←state| variable, which indicates the nesting of braces so that \.{\\cr} and \.{\\span} and tab marks are properly intercepted. There also are pointers |cur←head| and |cur←tail| to the head and tail of a list of adjustments being moved out from horizontal mode to vertical mode. The current values of these seven quantities appear in global variables; when they have to be pushed down, they are stored in 5-word nodes, and |align←ptr| points to the topmost such node. @d preamble==link(align←head) {the current preamble list} @d align←stack←node←size=5 {number of |mem| words to save alignment states} @<Glob...@>= @!cur←align:pointer; {current position in preamble list} @!cur←span:pointer; {start of currently spanned columns in preamble list} @!cur←loop:pointer; {place to copy when extending a periodic preamble} @!align←ptr:pointer; {most recently pushed-down alignment stack node} @!cur←head,@!cur←tail:pointer; {adjustment list pointers} @ The |align←state| and |preamble| variables are initialized elsewhere. @<Set init...@>= align←ptr:=null; cur←align:=null; cur←span:=null; cur←loop:=null; cur←head:=null; cur←tail:=null; @ Alignment stack maintenance is handled by a pair of trivial routines called |push←alignment| and |pop←alignment|. @p procedure push←alignment; var p:pointer; {the new alignment stack node} begin p:=get←node(align←stack←node←size); link(p):=align←ptr; info(p):=cur←align; llink(p):=preamble; rlink(p):=cur←span; mem[p+2].int:=cur←loop; mem[p+3].int:=align←state; info(p+4):=cur←head; link(p+4):=cur←tail; align←ptr:=p; cur←head:=get←avail; end; @# procedure pop←alignment; var p:pointer; {the top alignment stack node} begin free←avail(cur←head); p:=align←ptr; cur←tail:=link(p+4); cur←head:=info(p+4); align←state:=mem[p+3].int; cur←loop:=mem[p+2].int; cur←span:=rlink(p); preamble:=llink(p); cur←align:=info(p); align←ptr:=link(p); free←node(p,align←stack←node←size); end; @ \TeX\ has eight procedures that govern alignments: |init←align| and |fin←align| are used at the very beginning and the very end; |init←row| and |fin←row| are used at the beginning and end of individual rows; |init←span| is used at the beginning of a sequence of spanned columns (possibly involving only one column); |init←col| and |fin←col| are used at the beginning and end of individual columns; and |align←peek| is used after \.{\\cr} to see whether the next item is \.{\\noalign}. We shall consider these routines in the order they are first used during the course of a complete \.{\\halign}, namely |init←align|, |align←peek|, |init←row|, |init←span|, |init←col|, |fin←col|, |fin←row|, |fin←align|. @ When \.{\\halign} or \.{\\valign} has been scanned in an appropriate mode, \TeX\ calls |init←align|, whose task is to get everything off to a good start. This mostly involves scanning the preamble and putting its information into the preamble list. @↑preamble@> @p @t\4@>@<Declare the procedure called |get←preamble←token|@>@t@>@/ procedure@?align←peek; forward;@t\2@>@/ procedure@?normal←paragraph; forward;@t\2@>@/ procedure init←align; label done, done1, done2, continue; var save←cs←ptr:pointer; {|warning←index| value for error messages} @!p:pointer; {for short-term temporary use} begin save←cs←ptr:=cur←cs; {\.{\\halign} or \.{\\valign}, usually} push←alignment; align←state:=-1000000; {enter a new alignment level} @<Check for improper alignment in displayed math@>; push←nest; {enter a new semantic level} @<Change current mode to |-vmode| for \.{\\halign}, |-hmode| for \.{\\valign}@>; scan←spec; new←save←level(align←group);@/ @<Scan the preamble and record it in the |preamble| list@>; new←save←level(align←group); if every←cr<>null then begin←token←list(every←cr,every←cr←text); align←peek; {look for \.{\\noalign} or \.{\\omit}} end; @ In vertical modes, |prev←depth| already has the correct value. But if we are in |mmode| (displayed formula mode), we reach out to the enclosing vertical mode for the |prev←depth| value that produces the correct baseline calculations. @<Change current mode...@>= if mode=mmode then begin mode:=-vmode; prev←depth:=nest[nest←ptr-2].aux←field; end else if mode>0 then negate(mode) @ When \.{\\halign} is used as a displayed formula, there should be no other pieces of mlists present. @<Check for improper alignment in displayed math@>= if (mode=mmode)and((tail<>head)or(incompleat←noad<>null)) then begin print←err("Improper "); print←esc("halign"); print(" inside $$'s"); @.Improper \\halign...@> help3("Displays can use special alignments (like \eqalignno)")@/ ("only if nothing but the alignment itself is between $$'s.")@/ ("So I've deleted the formulas that preceded this alignment."); error; flush←math; end @ @<Scan the preamble and record it in the |preamble| list@>= preamble:=null; cur←align:=align←head; cur←loop:=null; scanner←status:=aligning; warning←index:=save←cs←ptr; align←state:=-1000000; {at this point, |cur←cmd=left←brace|} loop@+ begin @<Append the current tabskip glue to the preamble list@>; if cur←cmd=car←ret then goto done; {\.{\\cr} ends the preamble} @<Scan preamble text until |cur←cmd| is |tab←mark| or |car←ret|, looking for changes in the tabskip glue; append an alignrecord to the preamble list@>; end; done: scanner←status:=normal @ @<Append the current tabskip glue to the preamble list@>= link(cur←align):=new←param←glue(tab←skip←code); cur←align:=link(cur←align) @ @<Scan preamble text until |cur←cmd| is |tab←mark| or |car←ret|...@>= @<Scan the template \<u←j>, putting the resulting token list in |hold←head|@>; link(cur←align):=new←null←box; cur←align:=link(cur←align); {a new alignrecord} info(cur←align):=end←span; width(cur←align):=null←flag; u←part(cur←align):=link(hold←head); @<Scan the template \<v←j>, putting the resulting token list in |hold←head|@>; v←part(cur←align):=link(hold←head) @ We enter `\.{\\span}' into |eqtb| with |tab←mark| as its command code, and with |span←code| as the command modifier. This makes \TeX\ interpret it essentially the same as an alignment delimiter like `\.\&', yet it is recognizably different when we need to distinguish it from a normal delimiter. It also turns out to be useful to give a special |cr←code| to `\.{\\cr}', and an even larger |cr←cr←code| to `\.{\\crcr}'. The end of a template is represented by two ``frozen'' control sequences called \.{\\endtemplate}. The first has the command code |end←template|, which is |>outer←call|, so it will not easily disappear in the presence of errors. The |get←x←token| routine converts the first into the second, which has |endv| as its command code. @d span←code=128 {distinct from any character} @d cr←code=129 {distinct from |span←code| and from any character} @d cr←cr←code=cr←code+1 {this distinguishes \.{\\crcr} from \.{\\cr}} @d end←template←token==cs←token←flag+frozen←end←template @<Put each of \TeX's primitives into the hash table@>= primitive("span",tab←mark,span←code);@/ @!@:span←}{\.{\\span} primitive@> primitive("cr",car←ret,cr←code); @!@:cr←}{\.{\\cr} primitive@> text(frozen←cr):="cr"; eqtb[frozen←cr]:=eqtb[cur←val];@/ primitive("crcr",car←ret,cr←cr←code); @!@:cr←cr←}{\.{\\crcr} primitive@> text(frozen←end←template):="endtemplate"; text(frozen←endv):="endtemplate"; eq←type(frozen←endv):=endv; equiv(frozen←endv):=null←list; eq←level(frozen←endv):=level←one;@/ eqtb[frozen←end←template]:=eqtb[frozen←endv]; eq←type(frozen←end←template):=end←template; @ @<Cases of |print←cmd←chr|...@>= tab←mark: if chr←code=span←code then print←esc("span") else chr←cmd("alignment tab character "); car←ret: if chr←code=cr←code then print←esc("cr") else print←esc("crcr"); @ The preamble is copied directly, except that \.{\\tabskip} causes a change to the tabskip glue, thereby possibly expanding macros that immediately follow it. An appearance of \.{\\span} also causes such an expansion. Note that if the preamble contains `\.{\\global\\tabskip}', the `\.{\\global}' token survives in the preamble and the `\.{\\tabskip}' defines new tabskip glue (locally). @<Declare the procedure called |get←preamble←token|@>= procedure get←preamble←token; label restart; begin restart: get←token; while (cur←chr=span←code)and(cur←cmd=tab←mark) do begin get←token; {this token will be expanded once} if cur←cmd>max←command then begin expand; get←token; end; end; if (cur←cmd=assign←glue)and(cur←chr=glue←base+tab←skip←code) then begin scan←optional←equals; scan←glue(glue←val); if global←defs>0 then geq←define(glue←base+tab←skip←code,glue←ref,cur←val) else eq←define(glue←base+tab←skip←code,glue←ref,cur←val); goto restart; end; end; @ Spaces are eliminated from the beginning of a template. @<Scan the template \<u←j>...@>= p:=hold←head; link(p):=null; loop@+ begin get←preamble←token; if cur←cmd=mac←param then goto done1; if (cur←cmd<=car←ret)and(cur←cmd>=tab←mark)and(align←state=-1000000) then if (p=hold←head)and(cur←loop=null)and(cur←cmd=tab←mark) then cur←loop:=cur←align else begin print←err("Missing # inserted in alignment preamble"); @.Missing \# inserted...@> help3("There should be exactly one # between &'s, when an")@/ ("\halign or \valign is being set up. In this case you had")@/ ("none, so I've put one in; maybe that will work."); back←error; goto done1; end else if (cur←cmd<>spacer)or(p<>hold←head) then begin link(p):=get←avail; p:=link(p); info(p):=cur←tok; end; end; done1: @ @<Scan the template \<v←j>...@>= p:=hold←head; link(p):=null; loop@+ begin continue: get←preamble←token; if (cur←cmd<=car←ret)and(cur←cmd>=tab←mark)and(align←state=-1000000) then goto done2; if cur←cmd=mac←param then begin print←err("Only one # is allowed per tab"); @.Only one \# is allowed...@> help3("There should be exactly one # between &'s, when an")@/ ("\halign or \valign is being set up. In this case you had")@/ ("more than one, so I'm ignoring all but the first."); error; goto continue; end; link(p):=get←avail; p:=link(p); info(p):=cur←tok; end; done2: link(p):=get←avail; p:=link(p); info(p):=end←template←token {put \.{\\endtemplate} at the end} @ The tricky part about alignments is getting the templates into the scanner at the right time, and recovering control when a row or column is finished. We usually begin a row after each \.{\\cr} has been sensed, unless that \.{\\cr} is followed by \.{\\noalign} or by the right brace that terminates the alignment. The |align←peek| routine is used to look ahead and do the right thing; it either gets a new row started, or gets a \.{\\noalign} started, or finishes off the alignment. @<Declare the procedure called |align←peek|@>= procedure align←peek; label restart; begin restart: align←state:=1000000; @<Get the next non-blank non-call token@>; if cur←cmd=no←align then begin scan←left←brace; new←save←level(no←align←group); if mode=-vmode then normal←paragraph; end else if cur←cmd=right←brace then fin←align else if (cur←cmd=car←ret)and(cur←chr=cr←cr←code) then goto restart {ignore \.{\\crcr}} else begin init←row; {start a new row} init←col; {start a new column and replace what we peeked at} end; end; @ To start a row (i.e., a `row' that rhymes with `dough' but not with `bough'), we enter a new semantic level, copy the first tabskip glue, and change from internal vertical mode to restricted horizontal mode or vice versa. The |space←factor| and |prev←depth| are not used on this semantic level, but we clear |aux| to zero just to be tidy. @p @t\4@>@<Declare the procedure called |init←span|@>@t@>@/ procedure init←row; begin push←nest; mode:=(-hmode-vmode)-mode; aux:=0; tail←append(new←glue(glue←ptr(preamble))); subtype(tail):=tab←skip←code+1;@/ cur←align:=link(preamble); cur←tail:=cur←head; init←span(cur←align); end; @ The parameter to |init←span| is a pointer to the alignrecord where the next column or group of columns will begin. A new semantic level is entered, so that the columns will generate a list for subsequent packaging. @<Declare the procedure called |init←span|@>= procedure init←span(@!p:pointer); begin push←nest; if mode=-hmode then space←factor:=1000 else begin prev←depth:=ignore←depth; normal←paragraph; end; cur←span:=p; end; @ When a column begins, we assume that |cur←cmd| is either |omit| or else the current token should be put back into the input until the \<u←j> template has been scanned. (Note that |cur←cmd| might be |tab←mark| or |car←ret|.) We also assume that |align←state| is approximately 1000000 at this time. We remain in the same mode, and start the template if it is called for. @p procedure init←col; begin extra←info(cur←align):=cur←cmd; if cur←cmd=omit then align←state:=0 else begin back←input; begin←token←list(u←part(cur←align),u←template); end; {now |align←state=1000000|} end; @ The scanner sets |align←state| to zero when the \<u←j> template ends. When a subsequent \.{\\cr} or \.{\\span} or tab mark occurs with |align←state=0|, the scanner activates the following code, which fires up the \<v←j> template. We need to remember the |cur←chr|, which is either |cr←cr←code|, |cr←code|, |span←code|, or a character code, depending on how the column text has ended. This part of the program had better not be activated when the preamble to another alignment is being scanned. @<Insert the \(v)\<v←j>...@>= begin if scanner←status=aligning then fatal←error("(interwoven alignment preambles are not allowed)"); @.interwoven alignment preambles...@> cur←cmd:=extra←info(cur←align); extra←info(cur←align):=cur←chr; if cur←cmd=omit then begin←token←list(omit←template,v←template) else begin←token←list(v←part(cur←align),v←template); align←state:=1000000; goto restart; end @ The token list |omit←template| just referred to is a constant token list that contains \.{\\endtemplate} only. @<Initialize the special...@>= info(omit←template):=end←template←token; {|link(omit←template)=null|} @ When the |endv| command at the end of a \<v←j> template comes through the scanner, things really start to happen; and it is the |fin←col| routine that makes them happen. This routine returns |true| if a row as well as a column has been finished. @p function fin←col:boolean; label exit; var p:pointer; {the alignrecord after the current one} @!q,@!r:pointer; {temporary pointers for list manipulation} @!s:pointer; {a new span node} @!u:pointer; {a new unset box} @!w:scaled; {natural width} @!o:glue←ord; {order of infinity} @!n:halfword; {span counter} begin q:=link(cur←align); if (cur←align=null)or(q=null) then confusion("endv"); @:this can't happen endv}{\quad endv@> p:=link(q); @<If the preamble list has been traversed, check that the row has ended@>; if extra←info(cur←align)<>span←code then begin unsave; new←save←level(align←group);@/ @<Package an unset box for the current column and record its width@>; @<Copy the tabskip glue between columns@>; if extra←info(cur←align)>=cr←code then begin fin←col:=true; return; end; init←span(p); end; align←state:=1000000; @<Get the next non-blank non-call token@>; cur←align:=p; init←col; fin←col:=false; exit: end; @ @<If the preamble list has been traversed, check that the row has ended@>= if (p=null)and(extra←info(cur←align)<cr←code) then if cur←loop<>null then @<Lengthen the preamble periodically@> else begin print←err("Extra alignment tab has been changed to "); @.Extra alignment tab...@> print←esc("cr"); help3("You have given more \span or & marks than there were")@/ ("in the preamble to the \halign or \valign now in progress.")@/ ("So I'll assume that you meant to type \cr instead."); extra←info(cur←align):=cr←code; error; end @ @<Lengthen the preamble...@>= begin link(q):=new←null←box; p:=link(q); {a new alignrecord} info(p):=end←span; width(p):=null←flag; cur←loop:=link(cur←loop); @<Copy the templates from node |cur←loop| into node |p|@>; cur←loop:=link(cur←loop); link(p):=new←glue(glue←ptr(cur←loop)); end @ @<Copy the templates from node |cur←loop| into node |p|@>= q:=hold←head; r:=u←part(cur←loop); while r<>null do begin link(q):=get←avail; q:=link(q); info(q):=info(r); r:=link(r); end; link(q):=null; u←part(p):=link(hold←head); q:=hold←head; r:=v←part(cur←loop); while r<>null do begin link(q):=get←avail; q:=link(q); info(q):=info(r); r:=link(r); end; link(q):=null; v←part(p):=link(hold←head) @ @<Copy the tabskip glue...@>= tail←append(new←glue(glue←ptr(link(cur←align)))); subtype(tail):=tab←skip←code+1 @ @<Package an unset...@>= begin if mode=-hmode then begin adjust←tail:=cur←tail; u:=hpack(link(head),natural); w:=width(u); cur←tail:=adjust←tail; adjust←tail:=null; end else begin u:=vpackage(link(head),natural,0); w:=height(u); end; n:=min←quarterword; {this represents a span count of 1} if cur←span<>cur←align then @<Update width entry for spanned columns@> else if w>width(cur←align) then width(cur←align):=w; type(u):=unset←node; span←count(u):=n;@/ @<Determine the stretch order@>; glue←order(u):=o; glue←stretch(u):=total←stretch[o];@/ @<Determine the shrink order@>; glue←sign(u):=o; glue←shrink(u):=total←shrink[o];@/ pop←nest; link(tail):=u; tail:=u; end @ A span node is a 2-word record containing |width|, |info|, and |link| fields. The |link| field is not really a link, it indicates the number of spanned columns; the |info| field points to a span node for the same starting column, having a greater extent of spanning, or to |end←span|, which has the largest possible |link| field; the |width| field holds the largest natural width corresponding to a particular set of spanned columns. A list of the maximum widths so far, for spanned columns starting at a given column, begins with the |info| field of the alignrecord for that column. @d span←node←size=2 {number of |mem| words for a span node} @<Initialize the special list heads...@>= link(end←span):=max←quarterword+1; info(end←span):=null; @ @<Update width entry for spanned columns@>= begin q:=cur←span; repeat incr(n); q:=link(link(q)); until q=cur←align; if n>max←quarterword then confusion("256 spans"); {this can happen, but won't} @:this can't happen 256 spans}{\quad 256 spans@> q:=cur←span; while link(info(q))<n do q:=info(q); if link(info(q))>n then begin s:=get←node(span←node←size); info(s):=info(q); link(s):=n; info(q):=s; width(s):=w; end else if width(info(q))<w then width(info(q)):=w; end @ At the end of a row, we append an unset box to the current vlist (for \.{\\halign}) or the current hlist (for \.{\\valign}). This unset box contains the unset boxes for the columns, separated by the tabskip glue. Everything will be set later. @p procedure fin←row; var p:pointer; {the new unset box} begin if mode=-hmode then begin p:=hpack(link(head),natural); pop←nest; append←to←vlist(p); if cur←head<>cur←tail then begin link(tail):=link(cur←head); tail:=cur←tail; end; end else begin p:=vpack(link(head),natural); pop←nest; link(tail):=p; tail:=p; space←factor:=1000; end; type(p):=unset←node; glue←stretch(p):=0; if every←cr<>null then begin←token←list(every←cr,every←cr←text); align←peek; end; {note that |glue←shrink(p)=0| since |glue←shrink==shift←amount|} @ Finally, we will reach the end of the alignment, and we can breathe a sigh of relief that memory hasn't overflowed. All the unset boxes will now be set so that the columns line up, taking due account of spanned columns. @p procedure@?do←assignments; forward;@t\2@>@/ procedure@?resume←after←display; forward;@t\2@>@/ procedure@?build←page; forward;@t\2@>@/ procedure fin←align; var @!p,@!q,@!r,@!s,@!u,@!v: pointer; {registers for the list operations} @!t,@!w:scaled; {width of column} @!o:scaled; {shift offset for unset boxes} @!n:halfword; {matching span amount} @!rule←save:scaled; {temporary storage for |overfull←rule|} begin if cur←group<>align←group then confusion("align1"); @:this can't happen align}{\quad align@> unsave; {that |align←group| was for individual entries} if cur←group<>align←group then confusion("align0"); unsave; {that |align←group| was for the whole alignment} if nest[nest←ptr-1].mode←field=mmode then o:=display←indent else o:=0; @<Go through the preamble list, determining the column widths and changing the alignrecords to dummy unset boxes@>; @<Package the preamble list, to determine the actual tabskip glue amounts, and let |p| point to this prototype box@>; @<Set the glue in all the unset boxes of the current list@>; flush←node←list(p); pop←alignment; @<Insert the \(c)current list into its environment@>; end;@/ @t\4@>@<Declare the procedure called |align←peek|@> @ It's time now to dismantle the preamble list and to compute the column widths. Let $w←{ij}$ be the maximum of the natural widths of all entries that span columns $i$ through $j$, inclusive. The alignrecord for column~$i$ contains $w←{ii}$ in its |width| field, and there is also a linked list of the nonzero $w←{ij}$ for increasing $j$, accessible via the |info| field; these span nodes contain the value $j-i-1+|min←quarterword|$ in their |link| fields. The values of $w←{ii}$ were initialized to |null←flag|, which we regard as $-\infty$. The final column widths are defined by the formula $$w←j=\max←{1\L i\L j}\biggl( w←{ij}-\sum←{i\L k<j}(t←k+w←k)\biggr),$$ where $t←k$ is the natural width of the tabskip glue between columns $k$ and~$k+1$. However, if $w←{ij}=-\infty$ for all |i| in the range |1<=i<=j| (i.e., if every entry that involved column~|j| also involved column~|j+1|), we let $w←j=0$, and we zero out the tabskip glue after column~|j|. \TeX\ computes these values by using the following scheme: First $w←1=w←{11}$. Then replace $w←{2j}$ by $\max(w←{2j},w←{1j}-t←1-w←1)$, for all $j>1$. Then $w←2=w←{22}$. Then replace $w←{3j}$ by $\max(w←{3j},w←{2j}-t←2-w←2)$ for all $j>2$; and so on. If any $w←j$ turns out to be $-\infty$, its value is changed to zero and so is the next tabskip. @<Go through the preamble list,...@>= q:=link(preamble); repeat flush←list(u←part(q)); flush←list(v←part(q)); p:=link(link(q)); if width(q)=null←flag then @<Nullify |width(q)| and the tabskip glue following this column@>; if info(q)<>end←span then @<Merge the widths in the span nodes of |q| with those of |p|, destroying the span nodes of |q|@>; type(q):=unset←node; span←count(q):=min←quarterword; height(q):=0; depth(q):=0; glue←order(q):=normal; glue←sign(q):=normal; glue←stretch(q):=0; glue←shrink(q):=0; q:=p; until q=null @ @<Nullify |width(q)| and the tabskip glue following this column@>= begin width(q):=0; r:=link(q); s:=glue←ptr(r); if s<>zero←glue then begin add←glue←ref(zero←glue); delete←glue←ref(s); glue←ptr(r):=zero←glue; end; end @ Merging of two span-node lists is a typical exercise in the manipulation of linearly linked data structures. The essential invariant in the following |repeat| loop is that we want to dispense with node |r|, in |q|'s list, and |u| is its successor; all nodes of |p|'s list up to and including |s| have been processed, and the successor of |s| matches |r| or precedes |r| or follows |r|, according as |link(r)=n| or |link(r)>n| or |link(r)<n|. @<Merge the widths...@>= begin t:=width(q)+width(glue←ptr(link(q))); r:=info(q); s:=end←span; info(s):=p; n:=min←quarterword+1; repeat width(r):=width(r)-t; u:=info(r); while link(r)>n do begin s:=info(s); n:=link(info(s))+1; end; if link(r)<n then begin info(r):=info(s); info(s):=r; decr(link(r)); s:=r; end else begin if width(r)>width(info(s)) then width(info(s)):=width(r); free←node(r,span←node←size); end; r:=u; until r=end←span; end @ Now the preamble list has been converted to a list of alternating unset boxes and tabskip glue, where the box widths are equal to the final column sizes. In case of \.{\\valign}, we change the widths to heights, so that a correct error message will be produced if the alignment is overfull or underfull. @<Package the preamble list...@>= save←ptr:=save←ptr-2; pack←begin←line:=-mode←line; if mode=-vmode then begin rule←save:=overfull←rule; overfull←rule:=0; {prevent rule from being packaged} p:=hpack(preamble,saved(1),saved(0)); overfull←rule:=rule←save; end else begin q:=link(preamble); repeat height(q):=width(q); width(q):=0; q:=link(link(q)); until q=null; p:=vpack(preamble,saved(1),saved(0)); q:=link(preamble); repeat width(q):=height(q); height(q):=0; q:=link(link(q)); until q=null; end; pack←begin←line:=0 @ @<Set the glue in all the unset...@>= q:=link(head); while q<>null do begin if type(q)=unset←node then @<Set the unset box |q| and the unset boxes in it@> else if type(q)=rule←node then @<Make the running dimensions in rule |q| extend to the boundaries of the alignment@>; q:=link(q); end @ @<Make the running dimensions in rule |q| extend...@>= begin if is←running(width(q)) then width(q):=width(p); if is←running(height(q)) then height(q):=height(p); if is←running(depth(q)) then depth(q):=depth(p); end @ The unset box |q| represents a row that contains one or more unset boxes, depending on how soon \.{\\cr} occurred in that row. @<Set the unset box |q| and the unset boxes in it@>= begin if mode=-vmode then begin type(q):=hlist←node; width(q):=width(p); end else begin type(q):=vlist←node; height(q):=height(p); end; glue←order(q):=glue←order(p); glue←sign(q):=glue←sign(p); glue←set(q):=glue←set(p); shift←amount(q):=o; r:=link(list←ptr(q)); s:=link(list←ptr(p)); repeat @<Set the glue in node |r| and change it from an unset node@>; r:=link(link(r)); s:=link(link(s)); until r=null; end @ A box made from spanned columns will be followed by tabskip glue nodes and by empty boxes as if there were no spanning. This permits perfect alignment of subsequent entries, and it prevents values that depend on floating point arithmetic from entering into the dimensions of any boxes. @<Set the glue in node |r|...@>= n:=span←count(r); t:=width(s); w:=t; u:=hold←head; while n>min←quarterword do begin decr(n); @<Append tabskip glue and an empty box to list |u|, and update |s| and |t| as the prototype nodes are passed@>; end; if mode=-vmode then @<Make the unset node |r| into an |hlist←node| of width |w|, setting the glue as if the width were |t|@> else @<Make the unset node |r| into a |vlist←node| of height |w|, setting the glue as if the height were |t|@>; shift←amount(r):=0; if u<>hold←head then {append blank boxes to account for spanned nodes} begin link(u):=link(r); link(r):=link(hold←head); r:=u; end @ @<Append tabskip glue and an empty box to list |u|...@>= s:=link(s); v:=glue←ptr(s); link(u):=new←glue(v); u:=link(u); subtype(u):=tab←skip←code+1; t:=t+width(v); if glue←sign(p)=stretching then begin if stretch←order(v)=glue←order(p) then t:=t+round(float(glue←set(p))*stretch(v)); @↑real multiplication@> end else if glue←sign(p)=shrinking then begin if shrink←order(v)=glue←order(p) then t:=t-round(float(glue←set(p))*shrink(v)); end; s:=link(s); link(u):=new←null←box; u:=link(u); t:=t+width(s); if mode=-vmode then width(u):=width(s)@+else begin type(u):=vlist←node; height(u):=width(s); end @ @<Make the unset node |r| into an |hlist←node| of width |w|...@>= begin height(r):=height(q); depth(r):=depth(q); if t=width(r) then begin glue←sign(r):=normal; glue←order(r):=normal; set←glue←ratio←zero(glue←set(r)); end else if t>width(r) then begin glue←sign(r):=stretching; if glue←stretch(r)=0 then set←glue←ratio←zero(glue←set(r)) else glue←set(r):=unfloat((t-width(r))/glue←stretch(r)); @↑real division@> end else begin glue←order(r):=glue←sign(r); glue←sign(r):=shrinking; if glue←shrink(r)=0 then set←glue←ratio←zero(glue←set(r)) else if (glue←order(r)=normal)and(width(r)-t>glue←shrink(r)) then set←glue←ratio←one(glue←set(r)) else glue←set(r):=unfloat((width(r)-t)/glue←shrink(r)); end; width(r):=w; type(r):=hlist←node; end @ @<Make the unset node |r| into a |vlist←node| of height |w|...@>= begin width(r):=width(q); if t=height(r) then begin glue←sign(r):=normal; glue←order(r):=normal; set←glue←ratio←zero(glue←set(r)); end else if t>height(r) then begin glue←sign(r):=stretching; if glue←stretch(r)=0 then set←glue←ratio←zero(glue←set(r)) else glue←set(r):=unfloat((t-height(r))/glue←stretch(r)); @↑real division@> end else begin glue←order(r):=glue←sign(r); glue←sign(r):=shrinking; if glue←shrink(r)=0 then set←glue←ratio←zero(glue←set(r)) else if (glue←order(r)=normal)and(height(r)-t>glue←shrink(r)) then set←glue←ratio←one(glue←set(r)) else glue←set(r):=unfloat((height(r)-t)/glue←shrink(r)); end; height(r):=w; type(r):=vlist←node; end @ We now have a completed alignment, in the list that starts at |head| and ends at |tail|. This list will be merged with the one that encloses it. (In case the enclosing mode is |mmode|, for displayed formulas, we will need to insert glue before and after the display; that part of the program will be deferred until we're more familiar with such operations.) @<Insert the \(c)current list into its environment@>= t:=aux; p:=link(head); q:=tail; pop←nest; if mode=mmode then @<Finish an alignment in a display@> else begin aux:=t; link(tail):=p; if p<>null then tail:=q; if mode=vmode then build←page; end @* \[38] Breaking paragraphs into lines. We come now to what is probably the most interesting algorithm of \TeX: the mechanism for choosing the ``best possible'' breakpoints that yield the individual lines of a paragraph. \TeX's line-breaking algorithm takes a given horizontal list and converts it to a sequence of boxes that are appended to the current vertical list. In the course of doing this, it creates a special data structure containing three kinds of records that are not used elsewhere in \TeX. Such nodes are created while a paragraph is being processed, and they are destroyed afterwards; thus, the other parts of \TeX\ do not need to know anything about how line-breaking is done. The method used here is based on an approach devised by Michael F. Plass and @↑Plass, Michael Frederick@> @↑Knuth, Donald Ervin@> the author in 1977, subsequently generalized and improved by the same two people in 1980. A detailed discussion appears in {\sl SOFTWARE---Practice \AM\ Experience \bf11} (1981), 1119--1184, where it is shown that the line-breaking problem can be regarded as a special case of the problem of computing the shortest path in an acyclic network. The cited paper includes numerous examples and describes the history of line breaking as it has been practiced by printers through the ages. The present implementation adds two new ideas to the algorithm of 1980: memory space requirements are considerably reduced by using smaller records for inactive nodes than for active ones, and arithmetic overflow is avoided by using ``delta distances'' instead of keeping track of the total distance from the beginning of the paragraph to the current point. @ The |line←break| procedure should be invoked only in horizontal mode; it leaves that mode and places its output into the current vlist of the enclosing vertical mode (or internal vertical mode). There is one explicit parameter: |final←widow←penalty| is the amount of additional penalty to be inserted before the final line of the paragraph. There are also a number of implicit parameters: The hlist to be broken starts at |link(head)|, and it is nonempty. The value of |prev←graf| in the enclosing semantic level tells where the paragraph should begin in the sequence of line numbers, in case hanging indentation or \.{\\parshape} are in use; |prev←graf| is zero unless this paragraph is being continued after a displayed formula. Other implicit parameters, such as the |par←shape←ptr| and various penalties to use for hyphenation, etc., appear in |eqtb|. After |line←break| has acted, it will have updated the current vlist and the value of |prev←graf|. Furthermore, the global variable |just←box| will point to the final box created by |line←break|, so that the width of this line can be ascertained when it is necessary to decide whether to use |above←display←skip| or |above←display←short←skip| before a displayed formula. @<Glob...@>= @!just←box:pointer; {the |hlist←node| for the last line of the new paragraph} @ Since |line←break| is a rather lengthy procedure---sort of a small world unto itself---we must build it up little by little, somewhat more cautiously than we have done with the simpler procedures of \TeX. Here is the general outline. @p@t\4@>@<Declare subprocedures for |line←break|@> procedure line←break(@!final←widow←penalty:integer); label done,done1,done2,done3,done4; var @<Local variables for line breaking@>@; begin pack←begin←line:=mode←line; {this is for over/underfull box messages} @<Get ready to start line breaking@>; @<Find optimal breakpoints@>; @<Break the paragraph at the chosen breakpoints, justify the resulting lines to the correct widths, and append them to the current vertical list@>; @<Clean up the memory by removing the break nodes@>; pack←begin←line:=0; end; @ The first task is to move the list from |head| to |temp←head| and go into the enclosing semantic level. We also append the \.{\\parfillskip} glue to the end of the paragraph, removing a space (or other glue node) if it was there, since spaces usually precede blank lines and instances of `\.{\$\$}'. The |par←fill←skip| is preceded by an infinite penalty, so it will never be considered as a potential breakpoint. This code assumes that a |glue←node| and a |penalty←node| occupy the same number of words in |mem|. @↑data structure assumptions@> @<Get ready...@>= link(temp←head):=link(head); if is←char←node(tail) then tail←append(new←penalty(inf←penalty)) else if type(tail)<>glue←node then tail←append(new←penalty(inf←penalty)) else begin type(tail):=penalty←node; delete←glue←ref(glue←ptr(tail)); flush←node←list(leader←ptr(tail)); penalty(tail):=inf←penalty; end; link(tail):=new←param←glue(par←fill←skip←code); pop←nest; @ When looking for optimal line breaks, \TeX\ creates a ``break node'' for each break that is {\sl feasible}, in the sense that there is a way to end a line at the given place without requiring any line to stretch more than a given tolerance. A break node is characterized by three things: the position of the break (which is a pointer to a |glue←node|, |math←node|, |penalty←node|, or |disc←node|); the ordinal number of the line that will follow this breakpoint; and the fitness classification of the line that has just ended, i.e., |tight←fit|, |decent←fit|, |loose←fit|, or |very←loose←fit|. @d tight←fit=3 {fitness classification for lines shrinking 0.5 to 1.0 of their shrinkability} @d loose←fit=1 {fitness classification for lines stretching 0.5 to 1.0 of their stretchability} @d very←loose←fit=0 {fitness classification for lines stretching more than their stretchability} @d decent←fit=2 {fitness classification for all other lines} @ The algorithm essentially determines the best possible way to achieve each feasible combination of position, line, and fitness. Thus, it answers questions like, ``What is the best way to break the opening part of the paragraph so that the fourth line is a tight line ending at such-and-such a place?'' However, the fact that all lines are to be the same length after a certain point makes it possible to regard all sufficiently large line numbers as equivalent, when the looseness parameter is zero, and this makes it possible for the algorithm to save space and time. An ``active node'' and a ``passive node'' are created in |mem| for each feasible breakpoint that needs to be considered. Active nodes are three words long and passive nodes are two words long. We need active nodes only for breakpoints near the place in the paragraph that is currently being examined, so they are recycled within a comparatively short time after they are created. @ An active node for a given breakpoint contains six fields: \yskip\hang|link| points to the next node in the list of active nodes; the last active node has |link=last←active|. \yskip\hang|break←node| points to the passive node associated with this breakpoint. \yskip\hang|line←number| is the number of the line that follows this breakpoint. \yskip\hang|fitness| is the fitness classification of the line ending at this breakpoint. \yskip\hang|type| is either |hyphenated| or |unhyphenated|, depending on whether this breakpoint is a |disc←node|. \yskip\hang|total←demerits| is the minimum possible sum of demerits over all lines leading from the beginning of the paragraph to this breakpoint. \yskip\noindent The value of |link(active)| points to the first active node on a linked list of all currently active nodes. This list is in order by |line←number|, except that nodes with |line←number>easy←line| may be in any order relative to each other. @d active←node←size=3 {number of words in active nodes} @d fitness==subtype {|very←loose←fit..tight←fit| on final line for this break} @d break←node==rlink {pointer to the corresponding passive node} @d line←number==llink {line that begins at this breakpoint} @d total←demerits(#)==mem[#+2].int {the quantity that \TeX\ minimizes} @d unhyphenated=0 {the |type| of a normal active break node} @d hyphenated=1 {the |type| of an active node that breaks at a |disc←node|} @d last←active==active {the active list ends where it begins} @ @<Initialize the special list heads...@>= type(last←active):=hyphenated; line←number(last←active):=max←halfword; subtype(last←active):=0; {the |subtype| is never examined by the algorithm} @ The passive node for a given breakpoint contains only four fields: \yskip\hang|link| points to the passive node created just before this one, if any, otherwise it is |null|. \yskip\hang|cur←break| points to the position of this breakpoint in the horizontal list for the paragraph being broken. \yskip\hang|prev←break| points to the passive node that should precede this one in an optimal path to this breakpoint. \yskip\hang|serial| is equal to |n| if this passive node is the |n|th one created during the current pass. (This field is used only when printing out detailed statistics about the line-breaking calculations.) \yskip\noindent There is a global variable called |passive| that points to the most recently created passive node. Another global variable, |printed←node|, is used to help print out the paragraph when detailed information about the line-breaking computation is being displayed. @d passive←node←size=2 {number of words in passive nodes} @d cur←break==rlink {in passive node, points to position of this breakpoint} @d prev←break==llink {points to passive node that should precede this one} @d serial==info {serial number for symbolic identification} @<Glob...@>= @!passive:pointer; {most recent node on passive list} @!printed←node:pointer; {most recent node that has been printed} @!pass←number:pointer; {the number of passive nodes allocated on this pass} @ The active list also contains ``delta'' nodes that help the algorithm compute the badness of individual lines. Such nodes appear only between two active nodes, and they have |type=delta|. If |p| and |r| are active nodes and if |q| is a delta node between them, so that |link(p)=q| and |link(q)=r|, then |q| tells the space difference between lines in the horizontal list that start after breakpoint |p| and lines that start after breakpoint |r|. In other words, if we know the length of the line that starts after |p| and ends at our current position, then the corresponding length of the line that starts after |r| is obtained by adding the amounts in node~|q|. A delta node contains six scaled numbers, since it must record the net change in glue stretchability with respect to all orders of infinity. The natural width difference appears in |mem[q+1].sc|; the stretch differences in units of pt, fil, fill, and filll appear in |mem[q+2..q+5].sc|; and the shrink difference appears in |mem[q+6].sc|. The |subtype| field of a delta node is not used. @d delta←node←size=7 {number of words in a delta node} @d delta←node=2 {|type| field in a delta node} @ As the algorithm runs, it maintains a set of six delta-like registers for the length of the line following the first active breakpoint to the current position in the given hlist. When it makes a pass through the active list, it also maintains a similar set of six registers for the length following the active breakpoint of current interest. A third set holds the length of an empty line (namely, the sum of \.{\\leftskip} and \.{\\rightskip}); and a fourth set is used to create new delta nodes. When we pass a delta node we want to do operations like `\ignorespaces|for k:=1 to 6 do cur←active←width[k]:=cur←active←width[k]+mem[q+k].sc|', and we want to do this without the overhead of |for| loops. The |do←all←six| macro makes such six-tuples convenient. @d do←all←six(#)==#(1);#(2);#(3);#(4);#(5);#(6) @<Glo...@>= @!active←width:array[1..6] of scaled; {distance from first active node to~|cur←p|} @!cur←active←width:array[1..6] of scaled; {distance from current active node} @!background:array[1..6] of scaled; {length of an ``empty'' line} @!break←width:array[1..6] of scaled; {length being computed after current break} @ Let's state the principles of the delta nodes more precisely and concisely, so that the following programs will be less obscure. For each legal breakpoint~|p| in the paragraph, we define two quantities $\alpha(p)$ and $\beta(p)$ such that the length of material in a line from breakpoint~|p| to breakpoint~|q| is $\gamma+\beta(q)-\alpha(p)$, for some fixed $\gamma$. Intuitively, $\alpha(p)$ and $\beta(q)$ are the total length of material from the beginning of the paragraph to a point ``after'' a break at |p| and to a point ``before'' a break at |q|; and $\gamma$ is the width of an empty line, namely the length contributed by \.{\\leftskip} and \.{\\rightskip}. Suppose, for example, that the paragraph consists entirely of alternating boxes and glue skips; let the boxes have widths $x←1\ldots x←n$ and let the skips have widths $y←1\ldots y←n$, so that the paragraph can be represented by $x←1y←1\ldots x←ny←n$. Let $p←i$ be the legal breakpoint at $y←i$; then $\alpha(p←i)=x←1+y←1+\cdots+x←i+y←i$, and $\beta(p←i)= x←1+y←1+\cdots+x←i$. To check this, note that the length of material from $p←2$ to $p←5$, say, is $\gamma+x←3+y←3+x←4+y←4+x←5=\gamma+\beta(p←5) -\alpha(p←2)$. The quantities $\alpha$, $\beta$, $\gamma$ involve glue stretchability and shrinkability as well as a natural width. If we were to compute $\alpha(p)$ and $\beta(p)$ for each |p|, we would need multiple precision arithmetic, and the multiprecise numbers would have to be kept in the active nodes. \TeX\ avoids this problem by working entirely with relative differences or ``deltas.'' Suppose, for example, that the active list contains $a←1\,\delta←1\,a←2\,\delta←2\,a←3$, where the |a|'s are active breakpoints and the $\delta$'s are delta nodes. Then $\delta←1=\alpha(a←1)-\alpha(a←2)$ and $\delta←2=\alpha(a←2)-\alpha(a←3)$. If the line breaking algorithm is currently positioned at some other breakpoint |p|, the |active←width| array contains the value $\gamma+\beta(p)-\alpha(a←1)$. If we are scanning through the list of active nodes and considering a tentative line that runs from $a←2$ to~|p|, say, the |cur←active←width| array will contain the value $\gamma+\beta(p)-\alpha(a←2)$. Thus, when we move from $a←2$ to $a←3$, we want to add $\alpha(a←2)-\alpha(a←3)$ to |cur←active←width|; and this is just $\delta←2$, which appears in the active list between $a←2$ and $a←3$. The |background| array contains $\gamma$. The |break←width| array will be used to calculate values of new delta nodes when the active list is being updated. @ Glue nodes in a horizontal list that is being paragraphed are not supposed to include ``infinite'' shrinkability; that is why the algorithm maintains four registers for stretching but only one for shrinking. If the user tries to introduce infinite shrinkability, the shrinkability will be reset to finite and an error message will be issued. A boolean variable |no←shrink←error←yet| prevents this error message from appearing more than once per paragraph. @d check←shrinkage(#)==if (shrink←order(#)<>normal)and(shrink(#)<>0) then begin #:=finite←shrink(#); end @<Glob...@>= @!no←shrink←error←yet:boolean; {have we complained about infinite shrinkage?} @ @<Declare subprocedures for |line←break|@>= function finite←shrink(@!p:pointer):pointer; {recovers from infinite shrinkage} var q:pointer; {new glue specification} begin if no←shrink←error←yet then begin no←shrink←error←yet:=false; print←err("Infinite glue shrinkage found in a paragraph"); @.Infinite glue shrinkage...@> help5("The paragraph just ended includes some glue that has")@/ ("infinite shrinkability, e.g., `\hskip 0pt minus 1fil'.")@/ ("Such glue doesn't belong there---it allows a paragraph")@/ ("of any length to fit on one line. But it's safe to proceed,")@/ ("since the offensive shrinkability has been made finite."); error; end; q:=new←spec(p); shrink←order(q):=normal; delete←glue←ref(p); finite←shrink:=q; end; @ @<Get ready...@>= no←shrink←error←yet:=true;@/ check←shrinkage(left←skip); check←shrinkage(right←skip);@/ q:=left←skip; r:=right←skip; background[1]:=width(q)+width(r);@/ background[2]:=0; background[3]:=0; background[4]:=0; background[5]:=0;@/ background[2+stretch←order(q)]:=stretch(q);@/ background[2+stretch←order(r)]:=@|background[2+stretch←order(r)]+stretch(r);@/ background[6]:=shrink(q)+shrink(r); @ A pointer variable |cur←p| runs through the given horizontal list as we look for breakpoints. This variable is global, since it is used both by |line←break| and by its subprocedure |try←break|. Another global variable called |threshold| is used to determine the feasibility of individual lines: breakpoints are feasible if there is a way to reach them without creating lines whose badness exceeds |threshold|. (The badness is compared to |threshold| before penalties are added, so that penalty values do not affect the feasibility of breakpoints, except that no break is allowed when the penalty is 10000 or more.) If |threshold| is 10000 or more, all legal breaks are considered feasible, since the |badness| function specified above never returns a value greater than~10000. Two passes might be made through the paragraph in an attempt to find at least one set of feasible breakpoints. On the first pass, we have |threshold=pretolerance| and |second←pass=false|. If this pass fails to find a feasible solution, |threshold| is set to |tolerance|, |second←pass| is set |true|, and an attempt is made to hyphenate as many words as possible. @<Glob...@>= @!cur←p:pointer; {the current breakpoint under consideration} @!second←pass:boolean; {is this our second attempt to break this paragraph?} @!threshold:integer; {maximum badness on feasible lines} @ The heart of the line-breaking procedure is `|try←break|', a subroutine that tests if the current breakpoint |cur←p| is feasible, by running through the active list to see what lines of text can be made from active nodes to~|cur←p|. If feasible breaks are possible, new break nodes are created. If |cur←p| is too far from an active node, that node is deactivated. The parameter |pi| to |try←break| is the penalty associated with a break at |cur←p|; we have |pi=eject←penalty| if the break is forced, and |pi=inf←penalty| if the break is illegal. The other parameter, |break←type|, is set to |hyphenated| or |unhyphenated|, depending on whether or not the current break is at a |disc←node|. The end of a paragraph is also regarded as `|hyphenated|'; this case is distinguishable by the condition |cur←p=null|. @d copy←to←cur←active(#)==cur←active←width[#]:=active←width[#] @d deactivate=60 {go here when node |r| should be deactivated} @<Declare subprocedures for |line←break|@>= procedure try←break(@!pi:integer;@!break←type:small←number); label exit,done,done1,continue,deactivate; var r:pointer; {runs through the active list} @!prev←r:pointer; {stays a step behind |r|} @!old←l:halfword; {maximum line number in current equivalence class of lines} @!no←break←yet:boolean; {have we found a feasible break at |cur←p|?} @<Other local variables for |try←break|@>@; begin @<Make sure that |pi| is in the proper range@>; no←break←yet:=true; prev←r:=active; old←l:=0; do←all←six(copy←to←cur←active); loop@+ begin continue: r:=link(prev←r); @<If node |r| is of type |delta←node|, update |cur←active←width|, set |prev←r| and |prev←prev←r|, then |goto continue|@>; @<If a line number class has ended, create new active nodes for the best feasible breaks in that class; then |return| if |r=last←active|, otherwise compute the new |line←width|@>; @<Consider the demerits for a line from |r| to |cur←p|; deactivate node |r| if it should no longer be active; then |goto continue| if a line from |r| to |cur←p| is infeasible, otherwise record a new feasible break@>; end; exit: @!stat @<Update |printed←node| for symbolic displays@>@+tats@; end; @ @<Other local variables for |try←break|@>= @!prev←prev←r:pointer; {a step behind |prev←r|, if |type(prev←r)=delta←node|} @!s:pointer; {runs through nodes ahead of |cur←p|} @!q:pointer; {points to a new node being created} @!v:pointer; {points to a glue specification} @!t:quarterword; {replacement count, if |cur←p| is a discretionary node} @!f:internal←font←number; {used in character width calculation} @!l:halfword; {line number of current active node} @!node←r←stays←active:boolean; {should node |r| remain in the active list?} @!line←width:scaled; {the current line will be justified to this width} @!fit←class:very←loose←fit..tight←fit; {possible fitness class of test line} @!b:halfword; {badness of test line} @!d:integer; {demerits of test line} @!artificial←badness:boolean; {has |b| been forced to zero?} @!save←link:pointer; {temporarily holds value of |link(cur←p)|} @!shortfall:scaled; {used in badness calculations} @ @<Make sure that |pi| is in the proper range@>= if abs(pi)>=inf←penalty then if pi>0 then return {this breakpoint is inhibited by infinite penalty} else pi:=eject←penalty {this breakpoint will be forced} @ The following code uses the fact that |type(last←active)<>delta←node|. @d update←width(#)==@| cur←active←width[#]:=cur←active←width[#]+mem[r+#].sc @<If node |r|...@>= @↑inner loop@> if type(r)=delta←node then begin do←all←six(update←width); prev←prev←r:=prev←r; prev←r:=r; goto continue; end @ As we consider various ways to end a line at |cur←p|, in a given line number class, we keep track of the best total demerits known, in an array with one entry for each of the fitness classifications. For example, |minimal←demerits[tight←fit]| contains the fewest total demerits of feasible line breaks ending at |cur←p| with a |tight←fit| line; |best←place[tight←fit]| points to the passive node for the break before~|cur←p| that achieves such an optimum; and |best←pl←line[tight←fit]| is the |line←number| field in the active node corresponding to |best←place[tight←fit]|. When no feasible break sequence is known, the |minimal←demerits| entries will be equal to |awful←bad|, which is $2↑{30}-1$. Another variable, |minimum←demerits|, keeps track of the smallest value in the |minimal←demerits| array. @d awful←bad==@'7777777777 {more than a billion demerits} @<Global...@>= @!minimal←demerits:array[very←loose←fit..tight←fit] of scaled; {best total demerits known for current line class and position, given the fitness} @!minimum←demerits:scaled; {best total demerits known for current line class and position} @!best←place:array[very←loose←fit..tight←fit] of pointer; {how to achieve |minimal←demerits|} @!best←pl←line:array[very←loose←fit..tight←fit] of halfword; {corresponding line number} @ @<Get ready...@>= minimum←demerits:=awful←bad; minimal←demerits[tight←fit]:=awful←bad; minimal←demerits[decent←fit]:=awful←bad; minimal←demerits[loose←fit]:=awful←bad; minimal←demerits[very←loose←fit]:=awful←bad; @ The first part of the following code is part of \TeX's inner loop, so we don't want to waste any time. The current active node, namely node |r|, contains the line number that will be considered next. At the end of the list we have arranged the data structure so that |r=last←active| and |line←number(last←active)>old←l|. @↑inner loop@> @<If a line number class...@>= begin l:=line←number(r); if l>old←l then begin {now we are no longer in the inner loop} if (minimum←demerits<awful←bad)and@| ((old←l<>easy←line)or(r=last←active)) then @<Create new active nodes for the best feasible breaks just found@>; if r=last←active then return; @<Compute the new line width@>; end; end @ It is not necessary to create new active nodes having |minimal←demerits >minimum←demerits+abs(adj←demerits)|, since such active nodes will never be chosen in the final paragraph breaks. This observation allows us to omit a substantial number of feasible breakpoints from further consideration. @<Create new active nodes...@>= begin if no←break←yet then @<Compute the values of |break←width|@>; @<Insert a delta node to prepare for breaks at |cur←p|@>; minimum←demerits:=minimum←demerits+abs(adj←demerits); for fit←class:=very←loose←fit to tight←fit do begin if minimal←demerits[fit←class]<=minimum←demerits then @<Insert a new active node from |best←place[fit←class]| to |cur←p|@>; minimal←demerits[fit←class]:=awful←bad; end; minimum←demerits:=awful←bad; @<Insert a delta node to prepare for the next active node@>; end @ When we insert a new active node for a break at |cur←p|, suppose this new node is to be placed just before active node |a|; then we essentially want to insert `$\delta\,|cur←p|\,\delta↑\prime$' before |a|, where $\delta=\alpha(a)-\alpha(|cur←p|)$ and $\delta↑\prime=\alpha(|cur←p|)-\alpha(a)$ in the notation explained above. The |cur←active←width| array now hold $\gamma+\beta(|cur←p|)-\alpha(a)$; so $\delta$ can be obtained by subtracting |cur←active←width| from the quantity $\gamma+\beta(|cur←p|)- \alpha(|cur←p|)$. The latter quantity can be regarded as the length of a line ``from |cur←p| to |cur←p|''; we call it the |break←width| at |cur←p|. The |break←width| is usually negative, since it consists of the background (which is normally zero) minus the width of nodes following~|cur←p| that are eliminated after a break. If, for example, node |cur←p| is a glue node, the width of this glue is subtracted from the background; and we also look ahead to eliminate all subsequent glue and penalty and kern and math nodes, subtracting their widths as well. @d set←break←width←to←background(#)==break←width[#]:=background[#] @<Compute the values of |break...@>= begin no←break←yet:=false; do←all←six(set←break←width←to←background); if (break←type=unhyphenated)or(cur←p=null) then begin s:=cur←p; while s<>null do begin if is←char←node(s) then goto done; case type(s) of glue←node:@<Subtract glue from |break←width|@>; penalty←node: do←nothing; math←node,kern←node: break←width[1]:=break←width[1]-width(s); othercases goto done endcases;@/ s:=link(s); end; end else @<Compute the discretionary |break←width| values@>; done: end @ @<Subtract glue from |break...@>= begin v:=glue←ptr(s); break←width[1]:=break←width[1]-width(v); break←width[2+stretch←order(v)]:=break←width[2+stretch←order(v)]-stretch(v); break←width[6]:=break←width[6]-shrink(v); end @ When |cur←p| is a discretionary break, the length of a line ``from |cur←p| to |cur←p|'' has to be defined properly so that the other calculations work out. Suppose that the pre-break text at |cur←p| has length $l←0$, the post-break text has length $l←1$, and the replacement text has length |l|. Suppose also that |q| is the node following the replacement text. Then length of a line from |cur←p| to |q| will be computed as $\gamma+\beta(q)-\alpha(|cur←p|)$, where $\beta(q)=\beta(|cur←p|)-l←0+l$. The actual length will be the background plus $l←1$, so the length from |cur←p| to |cur←p| should be $\gamma+l←0+l←1-l$. The value of $l←0$ need not be computed, since |line←break| will put it into the global variable |disc←width| before calling |try←break|. @<Glob...@>= @!disc←width:scaled; {the length of discretionary material preceding a break} @ @<Compute the discretionary |break...@>= begin t:=replace←count(cur←p); s:=cur←p; while t>0 do begin decr(t); s:=link(s); @<Subtract the width of node |s| from |break←width|@>; end; s:=post←break(cur←p); while s<>null do begin @<Add the width of node |s| to |break←width|@>; s:=link(s); end; break←width[1]:=break←width[1]+disc←width; end @ Replacement texts and discretionary texts are supposed to contain only character nodes, kern nodes, and ligature nodes. @<Subtract the width of node |s|...@>= if is←char←node(s) then begin f:=font(s); break←width[1]:=break←width[1]-char←width(f)(char←info(f)(character(s))); end else case type(s) of ligature←node: begin f:=font(lig←char(s));@/ break←width[1]:=@|break←width[1]- char←width(f)(char←info(f)(character(lig←char(s)))); end; hlist←node,vlist←node,rule←node,kern←node: break←width[1]:=break←width[1]-width(s); othercases confusion("disc1") @:this can't happen disc1}{\quad disc1@> endcases @ @<Add the width of node |s| to |b...@>= if is←char←node(s) then begin f:=font(s); break←width[1]:=@|break←width[1]+char←width(f)(char←info(f)(character(s))); end else case type(s) of ligature←node: begin f:=font(lig←char(s)); break←width[1]:=break←width[1]+ char←width(f)(char←info(f)(character(lig←char(s)))); end; hlist←node,vlist←node,rule←node,kern←node: break←width[1]:=break←width[1]+width(s); othercases confusion("disc2") @:this can't happen disc2}{\quad disc2@> endcases @ We use the fact that |type(active)<>delta←node|. @d convert←to←break←width(#)==@| mem[prev←r+#].sc:=@|@t\hskip10pt@>mem[prev←r+#].sc -cur←active←width[#]+break←width[#] @d store←break←width(#)==active←width[#]:=break←width[#] @d new←delta←to←break←width(#)==@| mem[q+#].sc:=break←width[#]-cur←active←width[#] @<Insert a delta node to prepare for breaks at |cur←p|@>= if type(prev←r)=delta←node then {modify an existing delta node} begin do←all←six(convert←to←break←width); end else if prev←r=active then {no delta node needed at the beginning} begin do←all←six(store←break←width); end else begin q:=get←node(delta←node←size); link(q):=r; type(q):=delta←node; subtype(q):=0; {the |subtype| is not used} do←all←six(new←delta←to←break←width); link(prev←r):=q; prev←prev←r:=prev←r; prev←r:=q; end @ When the following code is performed, we will have just inserted at least one active node before |r|, so |type(prev←r)<>delta←node|. @d new←delta←from←break←width(#)==@|mem[q+#].sc:= cur←active←width[#]-break←width[#] @<Insert a delta node to prepare for the next active node@>= if r<>last←active then begin q:=get←node(delta←node←size); link(q):=r; type(q):=delta←node; subtype(q):=0; {the |subtype| is not used} do←all←six(new←delta←from←break←width); link(prev←r):=q; prev←prev←r:=prev←r; prev←r:=q; end @ When we create an active node, we also create the corresponding passive node. @<Insert a new active node from |best←place[fit←class]| to |cur←p|@>= begin q:=get←node(passive←node←size); link(q):=passive; passive:=q; cur←break(q):=cur←p; @!stat incr(pass←number); serial(q):=pass←number;@+tats@;@/ prev←break(q):=best←place[fit←class];@/ q:=get←node(active←node←size); break←node(q):=passive; line←number(q):=best←pl←line[fit←class]+1; fitness(q):=fit←class; type(q):=break←type; total←demerits(q):=minimal←demerits[fit←class]; link(q):=r; link(prev←r):=q; prev←r:=q; @!stat if tracing←paragraphs>0 then @<Print a symbolic description of the new break node@>; tats@;@/ end @ @<Print a symbolic description of the new break node@>= begin print←nl("@@@@"); print←int(serial(passive)); @.\AT!\AT!@> print(": line "); print←int(line←number(q)-1); print←char("."); print←int(fit←class); if break←type=hyphenated then print←char("-"); print(" t="); print←int(total←demerits(q)); print(" -> @@@@"); if prev←break(passive)=null then print←char("0") else print←int(serial(prev←break(passive))); end @ The length of lines depends on whether the user has specified \.{\\parshape} or \.{\\hangindent}. If |par←shape←ptr| is not null, it points to a $(2n+1)$-word record in |mem|, where the |info| in the first word contains the value of |n|, and the other $2n$ words contain the left margins and line lengths for the first |n| lines of the paragraph; the specifications for line |n| apply to all subsequent lines. If |par←shape←ptr=null|, the shape of the paragraph depends on the value of |n=hang←after|; if |n>=0|, hanging indentation takes place on lines |n+1|, |n+2|, \dots, otherwise it takes place on lines 1, \dots, $\vert n\vert$. When hanging indentation is active, the left margin is |hang←indent|, if |hang←indent>=0|, else it is 0; the line length is $|hsize|-\vert|hang←indent|\vert$. The normal setting is |par←shape←ptr=null|, |hang←after=0|, and |hang←indent=1|. Note that if |hang←indent=0|, the value of |hang←after| is irrelevant. @↑length of lines@> @↑hanging indentation@> @<Glob...@>= @!easy←line:halfword; {line numbers |>easy←line| are equivalent in break nodes} @!last←special←line:halfword; {line numbers |>last←special←line| all have the same width} @!first←width:scaled; {the width of all lines |<=last←special←line|, if no \.{\\parshape} has been specified} @!second←width:scaled; {the width of all lines |>last←special←line|} @!first←indent:scaled; {left margin to go with |first←width|} @!second←indent:scaled; {left margin to go with |second←width|} @ We compute the values of |easy←line| and the other local variables relating to line length when the |line←break| procedure is initializing itself. @<Get ready...@>= if par←shape←ptr=null then if hang←indent=0 then begin last←special←line:=0; second←width:=hsize; second←indent:=0; end else @<Set line length parameters in preparation for hanging indentation@> else begin last←special←line:=info(par←shape←ptr)-1; second←width:=mem[par←shape←ptr+2*(last←special←line+1)].sc; second←indent:=mem[par←shape←ptr+2*last←special←line+1].sc; end; if looseness=0 then easy←line:=last←special←line else easy←line:=max←halfword @ @<Set line length parameters in preparation for hanging indentation@>= begin last←special←line:=abs(hang←after); if hang←after<0 then begin first←width:=hsize-abs(hang←indent); if hang←indent>=0 then first←indent:=hang←indent else first←indent:=0; second←width:=hsize; second←indent:=0; end else begin first←width:=hsize; first←indent:=0; second←width:=hsize-abs(hang←indent); if hang←indent>=0 then second←indent:=hang←indent else second←indent:=0; end; end @ When we come to the following code, we have just encountered the first active node~|r| whose |line←number| field contains |l|. Thus we want to compute the length of the $l\,$th line of the current paragraph. Furthermore we want to set |old←l| to the last number in the class of line numbers equivalent to~|l|. @<Compute the new line width@>= if l>easy←line then begin line←width:=second←width; old←l:=max←halfword-1; end else begin old←l:=l; if l>last←special←line then line←width:=second←width else if par←shape←ptr=null then line←width:=first←width else line←width:=mem[par←shape←ptr+2*l@,].sc; end @ The remaining part of |try←break| deals with the calculation of demerits for a break from |r| to |cur←p|. The first thing to do is calculate the badness, |b|. This value will always be between zero and |inf←bad+1|; the latter value occurs only in the case of lines from |r| to |cur←p| that cannot shrink enough to fit the necessary width. In such cases, node |r| will be deactivated. We also deactivate node~|r| when a break at~|cur←p| is forced, since future breaks must go through a forced break. @<Consider the demerits for a line from |r| to |cur←p|...@>= begin @!stat artificial←badness:=false;@+tats@/ @↑inner loop@> shortfall:=line←width-cur←active←width[1]; {we're this much too short} if shortfall>0 then @<Set the value of |b| to the badness for stretching the line, and compute the corresponding |fit←class|@> else @<Set the value of |b| to the badness for shrinking the line, and compute the corresponding |fit←class|@>; if (b>inf←bad)or(pi=eject←penalty) then @<Prepare to deactivate node~|r|, and |goto deactivate| unless there is a reason to consider lines of text from |r| to |cur←p|@> else begin prev←r:=r; if b>threshold then goto continue; node←r←stays←active:=true; end; @<Record a new feasible break@>; if node←r←stays←active then goto continue; {|prev←r| has been set to |r|} deactivate: @<Deactivate node |r|@>; end @ When a line must stretch, the available stretchability appears in the subarray |cur←active←width[2..5]|, in units of points, fil, fill, and filll. The present section is part of \TeX's inner loop, and it is most often performed when the badness is infinite; therefore it is worth while to make a quick test for large width excess and small stretchability, before calling the |badness| subroutine. @↑inner loop@> @<Set the value of |b| to the badness for stretching...@>= if (cur←active←width[3]<>0)or(cur←active←width[4]<>0)or@| (cur←active←width[5]<>0) then begin b:=0; fit←class:=decent←fit; {infinite stretch} end else begin if shortfall>7230584 then if cur←active←width[2]<1663497 then begin b:=inf←bad; fit←class:=very←loose←fit; goto done1; end; b:=badness(shortfall,cur←active←width[2]); if b>12 then if b>99 then fit←class:=very←loose←fit else fit←class:=loose←fit else fit←class:=decent←fit; done1: end @ Shrinkability is never infinite in a paragraph; we can shrink the line from |r| to |cur←p| by at most |cur←active←width[6]|. @<Set the value of |b| to the badness for shrinking...@>= begin if -shortfall>cur←active←width[6] then b:=inf←bad+1 else b:=badness(-shortfall,cur←active←width[6]); if b>12 then fit←class:=tight←fit@+else fit←class:=decent←fit; end @ During the second pass, we dare not lose all active nodes, lest we lose touch with the line breaks already found. The code shown here makes sure that such a catastrophe does not happen, by permitting overfull boxes as a last resort. This particular part of \TeX\ was a source of several subtle bugs before the correct program logic was finally discovered; readers who seek to ``improve'' \TeX\ should therefore think thrice before daring to make any changes here. @↑overfull boxes@> @<Prepare to deactivate node~|r|, and |goto deactivate| unless...@>= begin if second←pass and (minimum←demerits=awful←bad) and@| (link(r)=last←active) and (prev←r=active) then begin b:=0; {set badness zero, this break is forced} @!stat artificial←badness:=true;@+tats end else if b>threshold then goto deactivate; node←r←stays←active:=false; end @ When we get to this part of the code, the line from |r| to |cur←p| is feasible, its badness is~|b|, and its fitness classification is |fit←class|. We don't want to make an active node for this break yet, but we will compute the total demerits and record them in the |minimal←demerits| array, if such a break is the current champion among all ways to get to |cur←p| in a given line-number class and fitness class. @<Record a new feasible break@>= @<Compute the demerits, |d|, from |r| to |cur←p|@>; @!stat if tracing←paragraphs>0 then @<Print a symbolic description of this feasible break@>; tats@;@/ d:=d+total←demerits(r); {this is the minimum total demerits from the beginning to |cur←p| via |r|} if d<=minimal←demerits[fit←class] then begin minimal←demerits[fit←class]:=d; best←place[fit←class]:=break←node(r); best←pl←line[fit←class]:=l; if d<minimum←demerits then minimum←demerits:=d; end @ @<Print a symbolic description of this feasible break@>= begin if printed←node<>cur←p then @<Print the list between |printed←node| and |cur←p|, then set |printed←node:=cur←p|@>; print←nl("@@"); @.\AT!@> if cur←p=null then print←esc("par") else if type(cur←p)<>glue←node then begin if type(cur←p)=penalty←node then print←esc("penalty") else if type(cur←p)=disc←node then print←esc("discretionary") else if type(cur←p)=kern←node then print←esc("kern") else print←esc("math"); end; print(" via @@@@"); if break←node(r)=null then print←char("0") else print←int(serial(break←node(r))); print(" b="); if artificial←badness then print←char("*")@+else print←int(b); print(" p="); print←int(pi); print(" d="); print←int(d); end @ @<Print the list between |printed←node| and |cur←p|...@>= begin print←nl(""); if cur←p=null then short←display(link(printed←node)) else begin save←link:=link(cur←p); link(cur←p):=null; print←nl(""); short←display(link(printed←node)); link(cur←p):=save←link; end; printed←node:=cur←p; end @ When the data for a discretionary break is being displayed, we will have printed the |pre←break| and |post←break| lists; we want to skip over the third list, so that the discretionary data will not appear twice. The following code is performed at the very end of |try←break|. @<Update |printed←node|...@>= if cur←p=printed←node then if cur←p<>null then if type(cur←p)=disc←node then begin t:=replace←count(cur←p); while t>0 do begin decr(t); printed←node:=link(printed←node); end; end @ @<Compute the demerits, |d|, from |r| to |cur←p|@>= d:=line←penalty+b; d:=d*d; if pi<>0 then if pi>0 then d:=d+pi*pi else if pi>eject←penalty then d:=d-pi*pi; if (break←type=hyphenated)and(type(r)=hyphenated) then if cur←p<>null then d:=d+double←hyphen←demerits else d:=d+final←hyphen←demerits; if abs(fit←class-fitness(r))>1 then d:=d+adj←demerits @ When an active node disappears, we must delete an adjacent delta node if the active node was at the beginning or the end of the active list, or if it was surrounded by delta nodes. We also must preserve the property that |cur←active←width| represents the length of material from |link(prev←r)| to~|cur←p|. @d combine←two←deltas(#)==@|mem[prev←r+#].sc:=mem[prev←r+#].sc+mem[r+#].sc @d downdate←width(#)==@|cur←active←width[#]:=cur←active←width[#]- mem[prev←r+#].sc @<Deactivate node |r|@>= link(prev←r):=link(r); free←node(r,active←node←size); if prev←r=active then @<Update the active widths, since the first active node has been deleted@> else if type(prev←r)=delta←node then begin r:=link(prev←r); if r=last←active then begin do←all←six(downdate←width); link(prev←prev←r):=last←active; free←node(prev←r,delta←node←size); prev←r:=prev←prev←r; end else if type(r)=delta←node then begin do←all←six(update←width); do←all←six(combine←two←deltas); link(prev←r):=link(r); free←node(r,delta←node←size); end; end @ The following code uses the fact that |type(last←active)<>delta←node|. If the active list has just become empty, we do not need to update the |active←width| array, since it will be initialized when an active node is next inserted. @d update←active(#)==active←width[#]:=active←width[#]+mem[r+#].sc @<Update the active widths,...@>= begin r:=link(active); if type(r)=delta←node then begin do←all←six(update←active); do←all←six(copy←to←cur←active); link(active):=link(r); free←node(r,delta←node←size); end; end @* \[39] Breaking paragraphs into lines, continued. So far we have gotten a little way into the |line←break| routine, having covered its important |try←break| subroutine. Now let's consider the rest of the process. The main loop of |line←break| traverses the hlist of the given paragraph, starting at |link(temp←head)|, and calls |try←break| at each legal breakpoint. A variable called |auto←breaking| is set to true except within math formulas, since glue nodes are not legal breakpoints when they appear in formulas. The current node of interest in the hlist is pointed to by |cur←p|. Another variable, |prev←p|, is usually one step behind |cur←p|, but the real meaning of |prev←p| is this: If |type(cur←p)=glue←node| then |cur←p| is a legal breakpoint if and only if |auto←breaking| is true and |prev←p| does not point to a glue node, penalty node, kern node, or math node. The following declarations provide for a few other local variables that are used in special calculations. @<Local variables for line breaking@>= @!auto←breaking:boolean; {is node |cur←p| outside a formula?} @!prev←p:pointer; {helps to determine when glue nodes are breakpoints} @!q,@!r,@!s:pointer; {miscellaneous nodes of temporary interest} @!f:internal←font←number; {used when calculating character widths} @ The `\ignorespaces|loop|\unskip' in the following code is performed at most twice per call of |line←break|, since it is actually a pass over the entire paragraph. @<Find optimal breakpoints@>= threshold:=pretolerance; if threshold>=0 then begin @!stat if tracing←paragraphs>0 then begin begin←diagnostic; print←nl("@@firstpass");@+end;@;@+tats@;@/ second←pass:=false; end else begin threshold:=tolerance; second←pass:=true; @!stat if tracing←paragraphs>0 then begin←diagnostic;@+tats@; end; loop@+ begin @<Create an active breakpoint representing the beginning of the paragraph@>; cur←p:=link(temp←head); auto←breaking:=true;@/ prev←p:=cur←p; {glue at beginning is not a legal breakpoint} while (cur←p<>null)and(link(active)<>last←active) do @<Call |try←break| if |cur←p| is a legal breakpoint; on the second pass, also try to hyphenate the next word, if |cur←p| is a glue node; then advance |cur←p| to the next node of the paragraph that could possibly be a legal breakpoint@>; if cur←p=null then @<Try the final line break at the end of the paragraph, and |goto done| if the desired breakpoints have been found@>; @<Clean up the memory by removing the break nodes@>; @!stat if tracing←paragraphs>0 then print←nl("@@secondpass");@;@+tats@/ threshold:=tolerance; second←pass:=true; {if at first you don't succeed, \dots} end; done: @!stat if tracing←paragraphs>0 then end←diagnostic(true);@;@+tats@/ @ The active node that represents the starting point does not need a corresponding passive node. @d store←background(#)==active←width[#]:=background[#] @<Create an active breakpoint representing the beginning of the paragraph@>= q:=get←node(active←node←size); type(q):=unhyphenated; fitness(q):=decent←fit; link(q):=last←active; break←node(q):=null; line←number(q):=prev←graf+1; total←demerits(q):=0; link(active):=q; do←all←six(store←background);@/ passive:=null; printed←node:=temp←head; pass←number:=0; font←in←short←display:=null←font @ @<Clean...@>= q:=link(active); while q<>last←active do begin cur←p:=link(q); if type(q)=delta←node then free←node(q,delta←node←size) else free←node(q,active←node←size); q:=cur←p; end; q:=passive; while q<>null do begin cur←p:=link(q); free←node(q,passive←node←size); q:=cur←p; end @ Here is the main switch in the |line←break| routine, where legal breaks are determined. As we move through the hlist, we need to keep the |active←width| array up to date, so that the badness of individual lines is readily calculated by |try←break|. It is convenient to use the short name |act←width| for the component of active width that represents real width as opposed to glue. @d act←width==active←width[1] {length from first active node to current node} @d kern←break==begin@t@>@;@/ if not is←char←node(link(cur←p)) and auto←breaking then if type(link(cur←p))=glue←node then try←break(0,unhyphenated); act←width:=act←width+width(cur←p); end @<Call |try←break| if |cur←p| is a legal breakpoint...@>= begin if is←char←node(cur←p) then @<Advance \(c)|cur←p| to the node following the present string of characters@>; case type(cur←p) of hlist←node,vlist←node,rule←node: act←width:=act←width+width(cur←p); whatsit←node: @<Advance \(p)past a whatsit node in the |line←break| loop@>; glue←node: begin @<If node |cur←p| is a legal breakpoint, call |try←break|@>; @<Update the active widths by including the glue in |glue←ptr(cur←p)|@>; if second←pass and auto←breaking then @<Try to hyphenate the following word@>; end; kern←node: kern←break; ligature←node: begin f:=font(lig←char(cur←p)); act←width:=act←width+char←width(f)(char←info(f)(character(lig←char(cur←p)))); end; disc←node: @<Try to break after a discretionary fragment@>; math←node: begin auto←breaking:=(subtype(cur←p)=after); kern←break; end; penalty←node: try←break(penalty(cur←p),unhyphenated); mark←node,ins←node,adjust←node: do←nothing; othercases confusion("paragraph") @:this can't happen paragraph}{\quad paragraph@> endcases;@/ prev←p:=cur←p; cur←p:=link(cur←p); end @ The code that passes over the characters of words in a paragraph is part of \TeX's inner loop, so it has been streamlined for speed. We use the fact that `\.{\\parfillskip}' glue appears at the end of each paragraph; it is therefore unnecessary to check if |link(cur←p)=null| when |cur←p| is a character node. @↑inner loop@> @<Advance \(c)|cur←p| to the node following the present string...@>= begin prev←p:=cur←p; repeat f:=font(cur←p); act←width:=act←width+char←width(f)(char←info(f)(character(cur←p))); cur←p:=link(cur←p); until not is←char←node(cur←p); end @ When node |cur←p| is a glue node, we look at |prev←p| to see whether or not a breakpoint is legal at |cur←p|, as explained above. @<If node |cur←p| is a legal breakpoint, call...@>= if auto←breaking then begin if is←char←node(prev←p) then try←break(0,unhyphenated) else if precedes←break(prev←p) then try←break(0,unhyphenated); end @ @<Update the active widths by including the glue in |glue←ptr(cur←p)|@>= begin check←shrinkage(glue←ptr(cur←p)); q:=glue←ptr(cur←p); act←width:=act←width+width(q);@| active←width[2+stretch←order(q)]:=@| active←width[2+stretch←order(q)]+stretch(q);@/ active←width[6]:=active←width[6]+shrink(q); end @ The following code knows that discretionary texts contain only character nodes, kern nodes, box nodes, rule nodes, and ligature nodes. @<Try to break after a discretionary fragment@>= begin s:=pre←break(cur←p); disc←width:=0; if s=null then try←break(ex←hyphen←penalty,hyphenated) else begin repeat @<Add the width of node |s| to |disc←width|@>; s:=link(s); until s=null; act←width:=act←width+disc←width; try←break(hyphen←penalty,hyphenated); act←width:=act←width-disc←width; end; end @ @<Add the width of node |s| to |disc←width|@>= if is←char←node(s) then begin f:=font(s); disc←width:=disc←width+char←width(f)(char←info(f)(character(s))); end else case type(s) of ligature←node: begin f:=font(lig←char(s)); disc←width:=disc←width+ char←width(f)(char←info(f)(character(lig←char(s)))); end; hlist←node,vlist←node,rule←node,kern←node: disc←width:=disc←width+width(s); othercases confusion("disc3") @:this can't happen disc3}{\quad disc3@> endcases @ The forced line break at the paragraph's end will reduce the list of breakpoints so that all active nodes represent breaks at |cur←p=null|. On the first pass, we insist on finding an active node that has the correct ``looseness.'' On the second pass, there will be at least one active node, and we will match the desired looseness as well as we can. The global variable |best←bet| will be set to the active node for the best way to break the paragraph, and a few other variables are used to help determine what is best. @<Glob...@>= @!best←bet:pointer; {use this passive node and its predecessors} @!fewest←demerits:integer; {the demerits associated with |best←bet|} @!best←line:halfword; {line number following the last line of the new paragraph} @!actual←looseness:integer; {the difference between |line←number(best←bet)| and the optimum |best←line|} @!line←diff:integer; {the difference between the current line number and the optimum |best←line|} @ @<Try the final line break at the end of the paragraph...@>= begin try←break(eject←penalty,hyphenated); if link(active)<>last←active then begin @<Find an active node with fewest demerits@>; if looseness=0 then goto done; @<Find the best active node for the desired looseness@>; if (actual←looseness=looseness)or second←pass then goto done; end; end @ @<Find an active node...@>= r:=link(active); fewest←demerits:=awful←bad; repeat if type(r)<>delta←node then if total←demerits(r)<fewest←demerits then begin fewest←demerits:=total←demerits(r); best←bet:=r; end; r:=link(r); until r=last←active; best←line:=line←number(best←bet) @ The adjustment for a desired looseness is a slightly more complicated version of the loop just considered. Note that if a paragraph is broken into segments by displayed equations, each segment will be subject to the looseness calculation, independently of the other segments. @<Find the best active node...@>= begin r:=link(active); actual←looseness:=0; repeat if type(r)<>delta←node then begin line←diff:=line←number(r)-best←line; if ((line←diff<actual←looseness)and(looseness<=line←diff))or@| ((line←diff>actual←looseness)and(looseness>=line←diff)) then begin best←bet:=r; actual←looseness:=line←diff; fewest←demerits:=total←demerits(r); end else if (line←diff=actual←looseness)and@| (total←demerits(r)<fewest←demerits) then begin best←bet:=r; fewest←demerits:=total←demerits(r); end; end; r:=link(r); until r=last←active; best←line:=line←number(best←bet); end @ Once the best sequence of breakpoints has been found, we call on the procedure |post←line←break| to finish the remainder of the work. (By introducing this subprocedure, we are able to keep |line←break| from getting extremely long.) @<Break the paragraph at the chosen...@>= post←line←break(final←widow←penalty) @ The total number of lines that will be set by |post←line←break| is |best←line-prev←graf-1|. The last breakpoint is specified by |break←node(best←bet)|, and this passive node points to the other breakpoints via the |prev←break| links. The finishing-up phase starts by linking the relevant passive nodes in forward order, changing |prev←break| to |next←break|. (The |next←break| fields actually reside in the same memory space as the |prev←break| fields did, but we give them a new name because of their new significance.) Then the lines are justified, one by one. @d next←break==prev←break {new name for |prev←break| after links are reversed} @<Declare subprocedures for |line←break|@>= procedure post←line←break(@!final←widow←penalty:integer); label done,done1; var q,@!r,@!s:pointer; {temporary registers for list manipulation} @!disc←break:boolean; {was the current break at a discretionary node?} @!cur←width:scaled; {width of line number |cur←line|} @!cur←indent:scaled; {left margin of line number |cur←line|} @!t:quarterword; {used for replacement counts in discretionary nodes} @!pen:integer; {use when calculating penalties between lines} @!cur←line: halfword; {the current line number being justified} begin @<Reverse the links of the relevant passive nodes, setting |cur←p| to the first breakpoint@>; cur←line:=prev←graf+1; repeat @<Justify the line ending at breakpoint |cur←p|, and append it to the current vertical list, together with associated penalties and other insertions@>; incr(cur←line); cur←p:=next←break(cur←p); if cur←p<>null then @<Prune unwanted nodes at the beginning of the next line@>; until cur←p=null; if (cur←line<>best←line)or(link(temp←head)<>null) then confusion("line breaking"); @:this can't happen line breaking}{\quad line breaking@> prev←graf:=best←line-1; end; @ The job of reversing links in a list is conveniently regarded as the job of taking items off one stack and putting them on another. In this case we take them off a stack pointed to by |q| and having |prev←break| fields; we put them on a stack pointed to by |cur←p| and having |next←break| fields. Node |r| is the passive node being moved from stack to stack. @<Reverse the links of the relevant passive nodes...@>= q:=break←node(best←bet); cur←p:=null; repeat r:=q; q:=prev←break(q); next←break(r):=cur←p; cur←p:=r; until q=null @ Glue and penalty and kern and math nodes are deleted at the beginning of a line, except in the unusual case that the node to be deleted is actually one of the chosen breakpoints. The pruning done here is designed to match the lookahead computation in |try←break|, where the |break←width| values are computed for non-discretionary breakpoints. @<Prune unwanted nodes at the beginning of the next line@>= begin r:=temp←head; loop@+ begin q:=link(r); if q=cur←break(cur←p) then goto done1; {|cur←break(cur←p)| is the next breakpoint} {now |q| cannot be |null|} if is←char←node(q) then goto done1; if non←discardable(q) then goto done1; r:=q; {now |type(q)=glue←node|, |kern←node|, |math←node| or |penalty←node|} end; done1: if r<>temp←head then begin link(r):=null; flush←node←list(link(temp←head)); link(temp←head):=q; end; end @ The current line to be justified appears in a horizontal list starting at |link(temp←head)| and ending at |cur←break(cur←p)|. If |cur←break(cur←p)| is a glue node, we reset the glue to equal the |right←skip| glue; otherwise we append the |right←skip| glue at the right. If |cur←break(cur←p)| is a discretionary node, we modify the list so that the discretionary break is compulsory, and we set |disc←break| to |true|. We also append the |left←skip| glue at the left of the line, unless it is zero. @<Justify the line ending at breakpoint |cur←p|, and append it...@>= @<Modify the end of the line to reflect the nature of the break and to include \.{\\rightskip}; also set the proper value of |disc←break|@>; @<Put the \(l)\.{\\leftskip} glue at the left and detach this line@>; @<Call the packaging subroutine, setting |just←box| to the justified box@>; @<Append the new box to the current vertical list, followed by the list of special nodes taken out of the box by the packager@>; @<Append a penalty node, if a nonzero penalty is appropriate@> @ At the end of the following code, |q| will point to the final node on the list about to be justified. @<Modify the end of the line...@>= q:=cur←break(cur←p); disc←break:=false; if q<>null then {|q| cannot be a |char←node|} if type(q)=glue←node then begin delete←glue←ref(glue←ptr(q)); glue←ptr(q):=right←skip; subtype(q):=right←skip←code+1; add←glue←ref(right←skip); goto done; end else begin if type(q)=disc←node then @<Change discretionary to compulsory and set |disc←break:=true|@>; if not is←char←node(q) then if (type(q)=math←node)or(type(q)=kern←node) then width(q):=0; end else begin q:=temp←head; while link(q)<>null do q:=link(q); end; @<Put the \(r)\.{\\rightskip} glue after node |q|@>; done: @ @<Change discretionary to compulsory...@>= begin t:=replace←count(q); @<Destroy the |t| nodes following |q|, but save the last one if it is a necessary kern; make |r| point to the following node@>; if post←break(q)<>null then @<Transplant the post-break list@>; if pre←break(q)<>null then @<Transplant the pre-break list@>; link(q):=r; disc←break:=true; end @ A subtle bug that would perhaps never have been detected is avoided here by preserving a kern node that just might equal |cur←break(next←break(cur←p))|. @<Destroy the |t| nodes following |q|...@>= if t=0 then r:=link(q) else begin r:=q; while t>1 do begin r:=link(r); decr(t); end; s:=link(r); if not is←char←node(s) then if next←break(cur←p)<>null then if cur←break(next←break(cur←p))=s then s:=r; r:=link(s); link(s):=null; flush←node←list(link(q)); replace←count(q):=0; end @ We move the post-break list from inside node |q| to the main list by re\-attaching it just before the present node |r|, then resetting |r|. @<Transplant the post-break list@>= begin s:=post←break(q); while link(s)<>null do s:=link(s); link(s):=r; r:=post←break(q); post←break(q):=null; end @ We move the pre-break list from inside node |q| to the main list by re\-attaching it just after the present node |q|, then resetting |q|. @<Transplant the pre-break list@>= begin s:=pre←break(q); link(q):=s; while link(s)<>null do s:=link(s); pre←break(q):=null; q:=s; end @ @<Put the \(r)\.{\\rightskip} glue after node |q|@>= r:=new←param←glue(right←skip←code); link(r):=link(q); link(q):=r; q:=r @ The following code begins with |q| at the end of the list to be justified. It ends with |q| at the beginning of that list, and with |link(temp←head)| pointing to the remainder of the paragraph, if any. @<Put the \(l)\.{\\leftskip} glue at the left...@>= r:=link(q); link(q):=null; q:=link(temp←head); link(temp←head):=r; if left←skip<>zero←glue then begin r:=new←param←glue(left←skip←code); link(r):=q; q:=r; end @ @<Append the new box to the current vertical list...@>= append←to←vlist(just←box); if adjust←head<>adjust←tail then begin link(tail):=link(adjust←head); tail:=adjust←tail; end; adjust←tail:=null @ Now |q| points to the hlist that represents the current line of the paragraph. We need to compute the appropriate line width, pack the line into a box of this size, and shift the box by the appropriate amount of indentation. @<Call the packaging subroutine...@>= if cur←line>last←special←line then begin cur←width:=second←width; cur←indent:=second←indent; end else if par←shape←ptr=null then begin cur←width:=first←width; cur←indent:=first←indent; end else begin cur←width:=mem[par←shape←ptr+2*cur←line].sc; cur←indent:=mem[par←shape←ptr+2*cur←line-1].sc; end; adjust←tail:=adjust←head; just←box:=hpack(q,cur←width,exactly); shift←amount(just←box):=cur←indent @ Penalties between the lines of a paragraph come from club and widow lines, from the |inter←line←penalty| parameter, and from lines that end at discretionary breaks. Breaking between lines of a two-line paragraph gets both club-line and widow-line penalties. The local variable |pen| will be set to the sum of all relevant penalties for the current line, except that the final line is never penalized. @<Append a penalty node, if a nonzero penalty is appropriate@>= if cur←line+1<>best←line then begin pen:=inter←line←penalty; if cur←line=prev←graf+1 then pen:=pen+club←penalty; if cur←line+2=best←line then pen:=pen+final←widow←penalty; if disc←break then pen:=pen+broken←penalty; if pen<>0 then begin r:=new←penalty(pen); link(tail):=r; tail:=r; end; end @* \[40] Pre-hyphenation. When the line-breaking routine is unable to find a feasible sequence of breakpoints, it makes a second pass over the paragraph, attempting to hyphenate the hyphenatable words. The goal of hyphenation is to insert discretionary material into the paragraph so that there are more potential places to break. The general rules for hyphenation are somewhat complex and technical, because we want to be able to hyphenate words that are preceded or followed by punctuation marks, and because we want the rules to work for languages other than English. We also must contend with the fact that hyphens might radically alter the ligature and kerning structure of a word. A sequence of characters will be considered for hyphenation only if it belongs to a ``potentially hyphenatable part'' of the current paragraph. This is a sequence of nodes $p←0p←1\ldots p←m$ where $p←0$ is a glue node, $p←1\ldots p←{m-1}$ are either character or ligature or whatsit or implicit kern nodes, and $p←m$ is a glue or penalty or insertion or adjust or mark or whatsit or explicit kern node. (Therefore hyphenation is disabled by boxes, math formulas, and discretionary nodes already inserted by the user.) The ligature nodes among $p←1\ldots p←{m-1}$ are effectively expanded into the original non-ligature characters; the kern nodes and whatsits are ignored. Each character |c| is now classified as either a nonletter (if |c>=128| or |lc←code(c)=0|), a lowercase letter (if |lc←code(c)=c|), or an uppercase letter (otherwise); an uppercase letter is treated as if it were |lc←code(c)| for purposes of hyphenation. The characters generated by $p←1\ldots p←{m-1}$ may begin with nonletters; let $c←1$ be the first letter that is not in the middle of a ligature. Whatsit nodes preceding $c←1$ are ignored; a whatsit found after $c←1$ will be the terminating node $p←m$. All characters that do not have the same font as $c←1$ will be treated as nonletters. The |hyphen←char| for that font must be between 0 and 255, otherwise hyphenation will not be attempted. \TeX\ looks ahead for as many consecutive letters $c←1\ldots c←n$ as possible; however, |n| must be less than 64, so a character that would otherwise be $c←{64}$ is effectively not a letter. Furthermore $c←n$ must not be in the middle of a ligature. In this way we obtain a string of letters $c←1\ldots c←n$ that are generated by nodes $p←a\ldots p←b$, where |1<=a<=b+1<=m|. If |n>=5|, this string qualifies for hyphenation; however, |uc←hyph| must be positive, if $c←1$ is uppercase. The hyphenation process takes place in three stages. First, the candidate sequence $c←1\ldots c←n$ is found; then potential positions for hyphens are determined by referring to hyphenation tables; and finally, the nodes $p←a\ldots p←b$ are replaced by a new sequence of nodes that includes the discretionary breaks found. Fortunately, we do not have to do all this calculation very often, because of the way it has been taken out of \TeX's inner loop. For example, when the second edition of the author's 700-page book {\sl Seminumerical Algorithms} was typeset by \TeX, only about 1.2 hyphenations needed to be @↑Knuth, Donald Ervin@> tried per paragraph, since the line breaking algorithm needed to use two passes on only about 5 per cent of the paragraphs. @ The letters $c←1\ldots c←n$ that are candidates for hyphenation are placed into an array called |hc|; the number |n| is placed into |hn|; pointers to nodes $p←a$ and~$p←b$ in the description above are placed into variables |ha| and |hb|; and the font number is placed into |hf|. @<Glob...@>= @!hc:array[0..65] of halfword; {word to be hyphenated} @!hn:small←number; {the number of positions occupied in |hc|} @!ha,@!hb:pointer; {nodes |ha..hb| should be replaced by the hyphenated result} @!hf:internal←font←number; {font number of the letters in |hc|} @!hu:array[1..63] of ASCII←code; {like |hc|, before conversion to lowercase} @ Hyphenation routines need a few more local variables. @<Local variables for line...@>= @!j:small←number; {an index into |hc| or |hu|} @!c:0..255; {character being considered for hyphenation} @ When the following code is activated, the |line←break| procedure is in its second pass, and |cur←p| points to a glue node. @<Try to hyphenate...@>= begin s:=link(cur←p); if s<>null then begin @<Skip to node |ha|, or |goto done1| if no hyphenation should be attempted@>; @<Skip to node |hb|, putting letters into |hu| and |hc|@>; @<Check that the nodes following |hb| permit hyphenation and that at least five letters have been found, otherwise |goto done1|@>; hyphenate; end; done1: end @ @<Declare subprocedures for |line←break|@>= @t\4@>@<Declare the function called |reconstitute|@> procedure hyphenate; label done,found,not←found,found1,exit; var @<Local variables for hyphenation@>@; begin @<Find hyphen locations for the word in |hc|@>; @<If no hyphens were found, |return|@>; @<Replace nodes |ha..hb| by a sequence of nodes that includes the discretionary hyphens@>; exit:end; @ The first thing we need to do is find the node |ha| that contains the first letter. @<Skip to node |ha|, or |goto done1|...@>= loop@+ begin if is←char←node(s) then begin c:=qo(character(s)); hf:=font(s); end else if type(s)=ligature←node then begin q:=lig←ptr(s); c:=qo(character(q)); hf:=font(q); end else if (type(s)=kern←node)and(subtype(s)=normal) then c:=128 else if type(s)=whatsit←node then c:=128 else goto done1; if c<128 then if lc←code(c)<>0 then if (lc←code(c)=c)or(uc←hyph>0) then goto done2 else goto done1; s:=link(s); end; done2: if hyphen←char[hf]<0 then goto done1; if hyphen←char[hf]>255 then goto done1; ha:=s @ The word to be hyphenated is now moved to the |hu| and |hc| arrays. @<Skip to node |hb|, putting letters...@>= hn:=0; loop@+ begin if is←char←node(s) then begin if font(s)<>hf then goto done3; c:=qo(character(s)); if c>=128 then goto done3; if (lc←code(c)=0)or(hn=63) then goto done3; hb:=s; incr(hn); hu[hn]:=c; hc[hn]:=lc←code(c)-1; end else if type(s)=ligature←node then @<Move the characters of a ligature node to |hu| and |hc|; but |goto done3| if they are not all letters@> else if (type(s)<>kern←node)or(subtype(s)<>normal) then goto done3; s:=link(s); end; done3: @ We let |j| be the index of the character being stored when a ligature node is being expanded, since we do not want to advance |hn| until we are sure that the entire ligature consists of letters. Note that it is possible to get to |done3| with |hn=0| and |hb| not set to any value. @<Move the characters of a ligature node to |hu| and |hc|...@>= begin j:=hn; q:=lig←ptr(s); if font(q)<>hf then goto done3; repeat c:=qo(character(q)); if c>=128 then goto done3; if (lc←code(c)=0)or(j=63) then goto done3; incr(j); hu[j]:=c; hc[j]:=lc←code(c)-1;@/ q:=link(q); until q=null; hb:=s; hn:=j; end @ @<Check that the nodes following |hb| permit hyphenation...@>= if hn<5 then goto done1; loop@+ begin if not(is←char←node(s)) then case type(s) of ligature←node: do←nothing; kern←node: if subtype(s)<>normal then goto done4; whatsit←node,glue←node,penalty←node,ins←node,adjust←node,mark←node: goto done4; othercases goto done1 endcases; s:=link(s); end; done4: @* \[41] Post-hyphenation. If a hyphen may be inserted between |hc[j]| and |hc[j+1]|, the hyphenation procedure will set |hyf[j]| to some small odd number. But before we look at \TeX's hyphenation procedure, which is independent of the rest of the line-breaking algorithm, let us consider what we will do with the hyphens it finds, since it is better to work on this part of the program before forgetting what |ha| and |hb|, etc., are all about. @<Glob...@>= @!hyf:array [0..64] of 0..9; {odd values indicate discretionary hyphens} @ @<Local variables for hyphenation@>= @!i,@!j,@!l:0..65; {indices into |hc| or |hu|} @!q,@!r,@!s:pointer; {temporary registers for list manipulation} @ \TeX\ will never insert a hyphen that has fewer than two letters before it or fewer than three after it; hence, a five-letter word has comparatively little chance of being hyphenated. If no hyphens have been found, we can save time by not having to make any changes to the paragraph. @<If no hyphens were found, |return|@>= for j:=2 to hn-3 do if odd(hyf[j]) then goto found1; return; found1: @ If hyphens are in fact going to be inserted, \TeX\ first deletes the subsequence of nodes |ha..hb|. The variable |s| will point to the node preceding |ha|, and |q| will point to the node following |hb|, so that things can be hooked up after we reconstitute the hyphenated word. @<Replace nodes |ha..hb| by a sequence of nodes...@>= q:=link(hb); link(hb):=null; s:=cur←p; while link(s)<>ha do s:=link(s); link(s):=null; flush←node←list(ha); @<Reconstitute nodes for the hyphenated word, inserting discretionary hyphens@> @ We must now face the fact that the battle is not over, even though the {\def\!{\kern-1pt} hyphens have been found: The process of reconstituting a word can be nontrivial because ligatures might change when a hyphen is present. {\sl The \TeX book\/} discusses the difficulties of the word ``difficult'', but since fonts can include highly general ligatures, the discretionary material surrounding a hyphen can be even more complex than that. For example, suppose that \.{abcdef} is a word in a font for which the only ligatures are \.{b\!c}, \.{c\!d}, \.{d\!e}, and \.{e\!f}. If this word is to permit hyphenation between \.b and \.c, the two patterns with and without hyphenation are $\.a\,\.{b\!c}\,\.{d\!e}\,\.f$ and $\.a\,\.b\,\.-\,\.{c\!d}\,\.{e\!f}$. Thus the insertion of a hyphen might cause effects to ripple arbitrarily far into the rest of the word. A further complication arises if additional hyphens appear together with such rippling, e.g., if the word in the example just given could also be hyphenated between \.c and \.d; \TeX\ avoids this by simply ignoring the additional hyphens in such weird cases.} @ The processing is facilitated by a subroutine called |reconstitute|. Given an index~|j|, this function creates a node for the next character or ligature found in the |hu| array starting at |hu[j]|, using font |hf|. For example, if |hu[j..j+2]| contains the three letters `f', `i', and `x', and if font |hf| contains an `fi' ligature but no `fix' ligature, then |reconstitute| will create a ligature node for `fi'. The index of the last character consumed, in this case |j+1|, will be returned. Furthermore, a kern node is created and appended, if kerning is called for between the consumed character or ligature and the next (unconsumed) character. A second parameter, |n|, gives the limit beyond which this procedure does not advance. In other words, |hu[n]| might be consumed, but |hu[n+1]| is never accessed. The global variable |hyphen←passed| is set to~|k| if this procedure consumes two characters |hu[k]| and |hu[k+1]| such that |hyf[k]| is odd, i.e., if the ligature might have to be broken by a hyphen, or if a kern is inserted between |hu[k]| and |hu[k+1]|. If this condition holds for more than one value of |k|, the smallest value is used; and if the condition holds for no values of |k|, |hyphen←passed| is set to zero. After |reconstitute| has acted, |link(hold←head)| points to the character or ligature node that was created, and |link(link(hold←head))| will either be |null| or a pointer to the kern node that was appended. @<Glob...@>= @!hyphen←passed:small←number; {first hyphen in a ligature, if any} @ @<Declare the function called |reconstitute|@>= function reconstitute(@!j,@!n:small←number): small←number; label continue,done; var p:pointer; {a node being created} @!s:pointer; {a node being appended to} @!q:four←quarters; {character information or a lig/kern instruction} @!c:quarterword; {current character} @!d:quarterword; {current character or ligature} @!w:scaled; {amount of kerning} @!r:0..font←mem←size; {position of current lig/kern instruction} begin @<Build a list of characters in a maximal ligature, and set |w| to the amount of kerning that should follow@>; @<If the list has more than one element, create a ligature node@>; @<Attach kerning, if |w<>0|@>; reconstitute:=j; end; @ @<Build a list of characters in a maximal ligature...@>= hyphen←passed:=0; s:=hold←head; w:=0; d:=qi(hu[j]); c:=d; loop@+ begin continue: p:=get←avail; font(p):=hf; character(p):=c; link(s):=p;@/ @<Look for a ligature or kern between |d| and the following character; update the data structure and |goto continue| if a ligature is found, otherwise update~|w| and |goto done|@>; end; done: @ @<Look for a ligature or kern between |d| and...@>= if j=n then goto done; q:=char←info(hf)(d); if char←tag(q)<>lig←tag then goto done; r:=lig←kern←start(hf)(q); c:=qi(hu[j+1]); loop@+ begin q:=font←info[r].qqqq; if next←char(q)=c then begin if odd(hyf[j])and(hyphen←passed=0) then hyphen←passed:=j; if op←bit(q)<kern←flag then @<Append to the ligature and |goto continue|@> else begin w:=char←kern(hf)(q); goto done; end; end else if stop←bit(q)<stop←flag then incr(r) else goto done; end @ @<Append to the ligature...@>= begin d:=rem←byte(q); incr(j); s:=p; goto continue; end @ After the list has been built, |link(s)| points to the final list element. @<If the list has more than one element, create a ligature node@>= if s<>hold←head then begin p:=new←ligature(hf,d,link(hold←head)); link(hold←head):=p; end @ @<Attach kerning, if |w<>0|@>= if w<>0 then link(link(hold←head)):=new←kern(w) @ Okay, we're ready to insert the potential hyphenations that were found. When the following program is executed, we want to append the word |hu[1..hn]| after node |s|, and node |q| should be appended to the result. During this process, the variable |i| will be a temporary counter or an index into |hu|; the variable |j| will be an index to our current position in |hu|; the variable |l| will be the counterpart of |j|, in a discretionary branch; the variable |r| will point to new nodes being created; and we need two new local variables: @<Local variables for hyph...@>= @!major←tail,@!minor←tail:pointer; {the end of lists in the main and discretionary branches being reconstructed} @!c:ASCII←code; {character temporarily replaced by a hyphen} @ When the following code is performed, |hyf[j]| will be zero for |j=1| and for |j>=hn-2|. @<Reconstitute nodes for the hyphenated word...@>= j:=0; repeat l:=j; j:=reconstitute(j+1,hn); if hyphen←passed<>0 then @<Create and append a discretionary node as an alternative to the ligature, and continue to develop both branches until they become equivalent@> else begin link(s):=link(hold←head); s:=link(s); if link(s)<>null then s:=link(s); end; if odd(hyf[j]) then @<Insert a discretionary hyphen after |s|@>; until j=hn; link(s):=q @ @<Create and append a discretionary node as an alternative...@>= begin r:=get←node(small←node←size); link(s):=r; link(r):=link(hold←head); type(r):=disc←node; major←tail:=link(hold←head); if link(major←tail)<>null then major←tail:=link(major←tail); i:=hyphen←passed; @<Put the \(c)characters |hu[l+1..i]| and a hyphen into |pre←break(r)|@>; @<Put the \(c)characters |hu[i+1..@,]| into |post←break(r)|, appending to this list and to |major←tail| until synchronization has been achieved@>; @<Move pointer |s| to the end of the current list, and set |replace←count(r)| appropriately@>; end @ The new hyphen might combine with the previous character via ligature or kern. At this point we have |i<=j|. @<Put the \(c)characters |hu[l+1..i]| and a hyphen into |pre←break(r)|@>= minor←tail:=null; c:=hu[i+1]; hu[i+1]:=hyphen←char[hf]; repeat l:=reconstitute(l+1,i+1); if minor←tail=null then pre←break(r):=link(hold←head) else link(minor←tail):=link(hold←head); minor←tail:=link(hold←head); if link(minor←tail)<>null then minor←tail:=link(minor←tail); until l>i; hu[i+1]:=c; {restore the character in the hyphen position} decr(l); hyf[l]:=0 @ The synchronization algorithm begins with |l<=j|. @<Put the \(c)characters |hu[i+1..@,]| into |post←break(r)|...@>= minor←tail:=null; post←break(r):=null; while l<j do begin repeat l:=reconstitute(l+1,hn); if minor←tail=null then post←break(r):=link(hold←head) else link(minor←tail):=link(hold←head); minor←tail:=link(hold←head); if link(minor←tail)<>null then begin hyf[l]:=0; minor←tail:=link(minor←tail); {kern present} end; until l>=j; while l>j do begin j:=reconstitute(j+1,hn); link(major←tail):=link(hold←head); major←tail:=link(hold←head); if link(major←tail)<>null then begin hyf[j]:=0; major←tail:=link(major←tail); {kern present} end; end; end @ @<Move pointer |s| to the end of the current list...@>= i:=0; s:=r; while link(s)<>null do begin incr(i); s:=link(s); end; replace←count(r):=i @ At this point |link(s)| is |null|. @<Insert a discretionary hyphen after |s|@>= begin r:=new←disc; pre←break(r):=new←character(hf,hyphen←char[hf]); link(s):=r; s:=r; end @* \[42] Hyphenation. When a word |hc[1..hn]| has been set up to contain a candidate for hyphenation, \TeX\ first looks to see if it is in the user's exception dictionary. If not, hyphens are inserted based on patterns that appear within the given word, using an algorithm due to Frank M. Liang. @↑Liang, Franklin Mark@> Let's consider Liang's method first, since it is much more interesting than the exception-lookup routine. The algorithm begins by setting |hyf[j]| to zero for all |j|, and invalid characters are inserted into |hc[0]| and |hc[hn+1]| to serve as delimiters. Then a reasonably fast method is used to see which of a given set of patterns occurs in the word |hc[0..(hn+1)]|. Each pattern $p←1\ldots p←k$ of length |k| has an associated sequence of |k+1| numbers $n←0\ldots n←k$; and if the pattern occurs in |hc[(j+1)..(j+k)]|, \TeX\ will set |hyf[j+i]:=@tmax@>(hyf[j+i],@t$n←i$@>)| for |0<=i<=k|. After this has been done for each pattern that occurs, a discretionary hyphen will be inserted between |hc[j]| and |hc[j+1]| when |hyf[j]| is odd, as we have already seen. The set of patterns $p←1\ldots p←k$ and associated numbers $n←0\ldots n←k$ depends, of course, on the language whose words are being hyphenated, and on the degree of hyphenation that is desired. A method for finding appropriate |p|'s and |n|'s, from a given dictionary of words and acceptable hyphenations, is discussed in Liang's Ph.D. thesis (Stanford University, 1983); \TeX\ simply starts with the patterns and works from there. @ The patterns are stored in a compact table that is also efficient for retrieval, using a variant of ``trie memory'' [cf.\ {\sl The Art of Computer Programming \bf3} (1973), 481--505]. We can find each pattern $p←1\ldots p←k$ by setting |@t$z←1$@>:=@t$p←1$@>| and then, for |1<i<=k|, setting |@t$z←i$@>:=trie←link@t$(z←{i-1})+p←i$@>|; the pattern will be identified by the number $z←k$. Since all the pattern information is packed together into a single |trie←link| array, it is necessary to prevent confusion between the data from inequivalent patterns, so another table is provided such that |trie←char@t$(z←i)=p←i$@>| for all |i|. There is also a table |trie←op|$(z←k)$ to identify the numbers $n←0\ldots n←k$ associated with $p←1\ldots p←k$. Comparatively few different number sequences $n←0\ldots n←k$ actually occur, since most of the |n|'s are generally zero. Therefore the number sequences are encoded in such a way that |trie←op|$(z←k)$ is only one byte long. If |trie←op(@t$z←k$@>)<>min←quarterword|, when $p←1\ldots p←k$ has matched the letters in |hc[(l-k+1)..l@,]|, we perform all of the required operations for this pattern by carrying out the following little program: Set |v:=trie←op(@t$z←k$@>)|. Then set |hyf[l-hyf←distance[v]]:=@tmax@>( hyf[l-hyf←distance[v]], hyf←num[v])|, and |v:=hyf←next[v]|; repeat, if necessary, until |v=min←quarterword|. @<Types...@>= @!trie←pointer=0..trie←size; {an index into |trie|} @ @d trie←link(#)==trie[#].rh {``downward'' link in a trie} @d trie←char(#)==trie[#].b1 {character matched at this trie location} @d trie←op(#)==trie[#].b0 {program for hyphenation at this trie location} @<Glob...@>= @!trie:array[trie←pointer] of two←halves; {|trie←link|, |trie←char|, |trie←op|} @!hyf←distance:array[quarterword] of small←number; {position |k-j| of $n←j$} @!hyf←num:array[quarterword] of small←number; {value of $n←j$} @!hyf←next:array[quarterword] of quarterword; {continuation of this |trie←op|} @ @<Local variables for hyph...@>= @!z:trie←pointer; {an index into |trie|} @!v:quarterword; {an index into |hyf←distance|, etc.} @ Assuming that these auxiliary tables have been set up properly, the hyphenation algorithm is quite short. In the following code we set |hc[hn+2]| to the impossible value 256, in order to guarantee that |hc[hn+3]| will never be fetched. @<Find hyphen locations for the word in |hc|@>= for j:=0 to hn do hyf[j]:=0; @<Look for the word |hc[1..hn]| in the exception table, and |goto found| (with |hyf| containing the hyphens) if an entry is found@>; hc[0]:=127; hc[hn+1]:=127; hc[hn+2]:=256; {insert delimiters} for j:=0 to hn-2 do begin z:=hc[j]; l:=j; while hc[l]=trie←char(z) do begin if trie←op(z)<>min←quarterword then @<Store \(m)maximum values in the |hyf| table@>; incr(l); z:=trie←link(z)+hc[l]; end; end; found: hyf[1]:=0; hyf[hn-2]:=0; hyf[hn-1]:=0; hyf[hn]:=0 @ @<Store \(m)maximum values in the |hyf| table@>= begin v:=trie←op(z); repeat i:=l-hyf←distance[v]; if hyf←num[v]>hyf[i] then hyf[i]:=hyf←num[v]; v:=hyf←next[v]; until v=min←quarterword; end @ The exception table that is built by \TeX's \.{\\hyphenation} primitive is organized as an ordered hash table [cf.\ Amble and Knuth, {\sl The Computer @↑Amble, Ole@> @↑Knuth, Donald Ervin@> Journal\/ \bf17} (1974), 135--142] using linear probing. If $\alpha$ and $\beta$ are words, we will say that $\alpha<\beta$ if $\vert\alpha\vert< \vert\beta\vert$ or if $\vert\alpha\vert=\vert\beta\vert$ and $\alpha$ is lexicographically smaller than $\beta$. (The notation $\vert \alpha\vert$ stands for the length of $\alpha$.) The idea of ordered hashing is to arrange the table so that a given word $\alpha$ can be sought by computing a hash address $h=h(\alpha)$ and then looking in table positions |h|, |h-1|, \dots, until encountering the first word $\L\alpha$. If this word is different from $\alpha$, we can conclude that $\alpha$ is not in the table. The words in the table point to lists in |mem| that specify hyphen positions in their |info| fields. The list for $c←1\ldots c←n$ contains |k| if the word $c←1\ldots c←n$ has a discretionary hyphen between $c←k$ and $c←{k+1}$. @<Types...@>= @!hyph←pointer=0..hyph←size; {an index into the ordered hash table} @ @<Glob...@>= @!hyph←word:array[hyph←pointer] of str←number; {exception words} @!hyph←list:array[hyph←pointer] of pointer; {list of hyphen positions} @!hyph←count:hyph←pointer; {the number of words in the exception dictionary} @ @<Local variables for init...@>= @!z:hyph←pointer; {runs through the exception dictionary} @ @<Set init...@>= for z:=0 to hyph←size do begin hyph←word[z]:=0; hyph←list[z]:=null; end; hyph←count:=0; @ The algorithm for exception lookup is quite simple, as soon as we have a few more local variables to work with. @<Local variables for hyph...@>= @!h:hyph←pointer; {an index into |hyph←word| and |hyph←list|} @!k:str←number; {an index into |str←start|} @!u:pool←pointer; {an index into |str←pool|} @ First we compute the hash code |h|, then we search until we either find the word or we don't. @<Look for the word |hc[1...@>= h:=hc[1]; for j:=2 to hn do h:=(h+h+hc[j]) mod hyph←size; loop@+ begin @<If the string |hyph←word[h]| is less than \(hc)|hc[1..hn]|, |goto not←found|; but if the two strings are equal, set |hyf| to the hyphen positions and |goto found|@>; if h>0 then decr(h)@+else h:=hyph←size; end; not←found: @ @<If the string |hyph←word[h]| is less than \(hc)...@>= k:=hyph←word[h]; if k=0 then goto not←found; if length(k)<hn then goto not←found; if length(k)=hn then begin j:=1; u:=str←start[k]; repeat if str←pool[u]<hc[j] then goto not←found; if str←pool[u]>hc[j] then goto done; incr(j); incr(u); until j>hn; @<Insert hyphens as specified in |hyph←list[h]|@>; goto found; end; done: @ @<Insert hyphens as specified...@>= s:=hyph←list[h]; while s<>null do begin hyf[info(s)]:=1; s:=link(s); end @ @<Search |hyph←list| for pointers to |p|@>= for q:=0 to hyph←size do begin if hyph←list[q]=p then begin print←nl("HYPH("); print←int(q); print←char(")"); end; end @ We have now completed the hyphenation routine, so the |line←break| procedure is finished at last. Since the hyphenation exception table is fresh in our minds, it's a good time to deal with the routine that adds new entries to it. When \TeX\ has scanned `\.{\\hyphenation}', it calls on a procedure named |new←hyph←exceptions| to do the right thing. @p procedure new←hyph←exceptions; {enters new exceptions} label reswitch, exit, found, not←found, done; var n:small←number; {length of current word} @!j:small←number; {an index into |hc|} @!h:hyph←pointer; {an index into |hyph←word| and |hyph←list|} @!k:str←number; {an index into |str←start|} @!p:pointer; {head of a list of hyphen positions} @!q:pointer; {used when creating a new node for list |p|} @!s,@!t:str←number; {strings being compared or stored} @!u,@!v:pool←pointer; {indices into |str←pool|} begin scan←left←brace; {a left brace must follow \.{\\hyphenation}} @<Enter as many hyphenation exceptions as are listed, until coming to a right brace; then skip an optional space and |return|@>; exit:end; @ @<Enter as many...@>= n:=0; p:=null; loop@+ begin get←x←token; reswitch: case cur←cmd of letter,other←char,char←given:@<Append a new letter or hyphen@>; char←num: begin scan←char←num; cur←chr:=cur←val; cur←cmd:=char←given; goto reswitch; end; spacer,right←brace: begin if n>4 then @<Enter a hyphenation exception@>; if cur←cmd=right←brace then return; n:=0; p:=null; end; othercases @<Give improper \.{\\hyphenation} error@> endcases; end @ @<Give improper \.{\\hyph...@>= begin print←err("Improper "); print←esc("hyphenation"); @.Improper \\hyphenation...@> print(" will be flushed"); help2("Hyphenation exceptions must contain only letters")@/ ("and hyphens. But continue; I'll forgive and forget."); error; end @ @<Append a new letter or hyphen@>= if cur←chr="-" then @<Append the value |n| to list |p|@> else begin if (cur←chr>127)or(lc←code(cur←chr)=0) then begin print←err("Not a letter"); @.Not a letter@> help2("Letters in \hyphenation words must have \lccode>0.")@/ ("Proceed; I'll ignore the character I just read."); error; end else if n<63 then begin incr(n); hc[n]:=lc←code(cur←chr)-1; end; end @ @<Append the value |n| to list |p|@>= begin if n>1 then begin q:=get←avail; link(q):=p; info(q):=n; p:=q; end; end @ @<Enter a hyphenation exception@>= begin str←room(n); h:=0; for j:=1 to n do begin h:=(h+h+hc[j]) mod hyph←size; append←char(hc[j]); end; s:=make←string; loop@+ begin if p=null then goto done; if info(p)<n-2 then goto done; q:=link(p); free←avail(p); p:=q; {eliminate hyphens that \TeX\ doesn't like} end; done: @<Insert the \(p)pair |(s,p)| into the exception table@>; end @ @<Insert the \(p)pair |(s,p)|...@>= if hyph←count=hyph←size then overflow("exception dictionary",hyph←size); @:TeX capacity exceeded exception dictionary}{\quad exception dictionary@> incr(hyph←count); while hyph←word[h]<>0 do begin @<If the string |hyph←word[h]| is less than \(or)or equal to |s|, interchange |(hyph←word[h],hyph←list[h])| with |(s,p)|@>; if h>0 then decr(h)@+else h:=hyph←size; end; hyph←word[h]:=s; hyph←list[h]:=p @ @<If the string |hyph←word[h]| is less than \(or)...@>= k:=hyph←word[h]; if length(k)<length(s) then goto found; if length(k)>length(s) then goto not←found; u:=str←start[k]; v:=str←start[s]; repeat if str←pool[u]<str←pool[v] then goto found; if str←pool[u]>str←pool[v] then goto not←found; incr(u); incr(v); until u=str←start[k+1]; found:q:=hyph←list[h]; hyph←list[h]:=p; p:=q;@/ t:=hyph←word[h]; hyph←word[h]:=s; s:=t; not←found: @* \[43] Initializing the hyphenation tables. The trie for \TeX's hyphenation algorithm is built from a sequence of patterns following a \.{\\patterns} specification. Such a specification is allowed only in \.{INITEX}, since the extra memory for auxiliary tables and for the initialization program itself would only clutter up the production version of \TeX\ with a lot of deadwood. The initialization first builds a trie that is linked instead of packed into sequential storage, so that insertions are readily made. Then it compresses the linked trie by identifying common subtries, and finally the trie is packed into the efficient sequential form that the hyphenation algorithm actually uses. @p @!init @<Declare procedures for preprocessing hyphenation patterns@>@; tini @ Before we discuss trie building in detail, let's consider the simpler problem of creating the |hyf←distance|, |hyf←num|, and |hyf←next| arrays. Suppose, for example, that \TeX\ reads the pattern `\.{ab2cde1}'. This is a pattern of length 5, with $n←0\ldots n←5=0\,0\,2\,0\,0\,1$ in the notation above. We want the corresponding |trie←op| code |v| to have |hyf←distance[v]=3|, |hyf←num[v]=2|, and |hyf←next[v]=@t$v↑\prime$@>|, where the auxiliary |trie←op| code $v↑\prime$ has |hyf←distance[@t$v↑\prime$@>]=0|, |hyf←num[@t$v↑\prime$@>]=1|, and |hyf←next[@t$v↑\prime$@>]=min←quarterword|. \TeX\ computes an appropriate value |v| with the |new←trie←op| subroutine below, by setting $$\hbox{|@t$v↑\prime$@>:=new←trie←op(0,1,min←quarterword)|,\qquad |v:=new←trie←op(3,2,@t$v↑\prime$@>)|.}$$ This subroutine looks up its three parameters in a special hash table, assigning a new value only if these three have not appeared before. The hash table is called |trie←op←hash|, and the number of entries it contains is |trie←op←ptr|. If the table overflows, the excess ops are ignored. @d quarterword←diff=max←quarterword-min←quarterword @d trie←op←hash←size=quarterword←diff+quarterword←diff {double} @<Glob...@>= @!init@! trie←op←hash:array[0..trie←op←hash←size] of quarterword; {trie op codes for triples} tini@;@/ @t\hskip1em@>@!trie←op←ptr:quarterword; {highest |trie←op| assigned} @ The hash function used by |new←trie←op| is based on the observation that 313/510 is an approximation to the golden ratio [cf.\ {\sl The Art of Computer Programming \bf3} (1973), 510--512]; |trie←op←hash←size| is usually a multiple of 510. But the choice is comparatively unimportant in this particular application. @<Declare procedures for preprocessing hyph...@>= function new←trie←op(@!d,@!n:small←number;@!v:quarterword):quarterword; label exit; var h:0..trie←op←hash←size; {trial hash location} @!u:quarterword; {trial op code} begin h:=abs(n+313*d+361*v) mod trie←op←hash←size; loop@+ begin u:=trie←op←hash[h]; if u=min←quarterword then {empty position found} begin if trie←op←ptr=max←quarterword then {overflow} begin new←trie←op:=min←quarterword; return; end; incr(trie←op←ptr); hyf←distance[trie←op←ptr]:=d; hyf←num[trie←op←ptr]:=n; hyf←next[trie←op←ptr]:=v; trie←op←hash[h]:=trie←op←ptr; new←trie←op:=trie←op←ptr; return; end; if (hyf←distance[u]=d)and(hyf←num[u]=n)and(hyf←next[u]=v) then begin new←trie←op:=u; return; end; if h>0 then decr(h)@+else h:=trie←op←hash←size; end; exit:end; @ The linked trie that is used to preprocess hyphenation patterns appears in several global arrays. Each node represents an instruction of the form ``if you see character |c|, then perform operation |o|, move to the next character, and go to node |l|; otherwise go to node |r|.'' The four quantities |c|, |o|, |l|, and |r| are stored in four arrays |trie←c|, |trie←o|, |trie←l|, and |trie←r|. The root of the trie is |trie←l[0]|, and the number of nodes is |trie←ptr|. Null trie pointers are represented by zero. To initialize the trie, we simply set |trie←l[0]| and |trie←ptr| to zero. We also set |trie←c[0]| to some arbitrary value, since the algorithm may access it. The algorithms maintain the condition |trie←c[trie←r[z]]>trie←c[z]| whenever |z<>0| and |trie←r[z]<>0|; in other words, sibling nodes are ordered by their |c| fields. @d trie←root==trie←l[0] {root of the linked trie} @<Glob...@>= @!init @!trie←c:packed array[trie←pointer] of ASCII←code; {characters to match} @t\hskip1em@>@!trie←o:packed array[trie←pointer] of quarterword; {operations to perform} @t\hskip1em@>@!trie←l:packed array[trie←pointer] of trie←pointer; {left subtrie links} @t\hskip1em@>@!trie←r:packed array[trie←pointer] of trie←pointer; {right subtrie links} @t\hskip1em@>@!trie←ptr:trie←pointer; {the number of nodes in the trie} tini @ Let us suppose that a linked trie has already been constructed. Experience shows that we can often reduce its size by recognizing common subtries; therefore another hash table is introduced for this purpose, somewhat similar to |trie←op←hash|. The new hash table will be initialized to zero. @<Glob...@>= @!init @!trie←hash:packed array[trie←pointer] of trie←pointer; tini {to identify equivalent subtries} @ The function |trie←node(p)| returns |p| if |p| is distinct from other nodes that it has seen, otherwise it returns the number of the first equivalent node that it has seen. @<Declare procedures for preprocessing hyph...@>= function trie←node(@!p:trie←pointer):trie←pointer; {converts to a canonical form} label exit; var h:trie←pointer; {trial hash location} @!q:trie←pointer; {trial trie node} begin h:=abs(trie←c[p]+1009*trie←o[p]+@| 2718*trie←l[p]+3142*trie←r[p]) mod trie←size; loop@+ begin q:=trie←hash[h]; if q=0 then begin trie←hash[h]:=p; trie←node:=p; return; end; if (trie←c[q]=trie←c[p])and(trie←o[q]=trie←o[p])and@| (trie←l[q]=trie←l[p])and(trie←r[q]=trie←r[p]) then begin trie←node:=q; return; end; if h>0 then decr(h)@+else h:=trie←size; end; exit:end; @ A neat recursive procedure is now able to compress a trie by traversing it and applying |trie←node| to its nodes in ``bottom up'' fashion. We will compress the entire trie by clearing |trie←hash| to zero and then saying `|trie←root:=compress←trie(trie←root)|'. @↑recursion@> @<Declare procedures for preprocessing hyph...@>= function compress←trie(@!p:trie←pointer):trie←pointer; begin if p=0 then compress←trie:=0 else begin trie←l[p]:=compress←trie(trie←l[p]); trie←r[p]:=compress←trie(trie←r[p]); compress←trie:=trie←node(p); end; end; @ Before we forget how to initialize the data structures that have been mentioned so far, let's write a procedure that does the initialization. @<Declare procedures for preprocessing hyph...@>= procedure init←pattern←memory; {gets ready to build a linked trie} var h:0..trie←op←hash←size; {an index into |trie←op←hash|} @!p:trie←pointer; {an index into |trie←hash|} begin for h:=0 to trie←op←hash←size do trie←op←hash[h]:=min←quarterword; trie←op←ptr:=min←quarterword; trie←root:=0; trie←c[0]:=0; trie←ptr:=0; for p:=0 to trie←size do trie←hash[p]:=0; end; @ The compressed trie will be packed into the |trie| array using a ``top-down first-fit'' procedure. This is a little tricky, so the reader should pay close attention: The |trie←hash| array is cleared to zero again and renamed |trie←ref| for this phase of the operation; later on, |trie←ref[p]| will be nonzero if the linked trie node |p| is the oldest sibling in a family and if the characters |c| of that family have been allocated to locations |trie←ref[p]+c| in the |trie| array. Locations of |trie| that are in use will have |trie←link=0|, while the unused holes in |trie| will be doubly linked with |trie←link| pointing to the next larger vacant location and |trie←back| pointing to the next smaller one. This double linking will have been carried out only as far as |trie←max|, where |trie←max| is the largest index of |trie| that will be needed. Another array |trie←taken| tells whether or not a given location is equal to |trie←ref[p]| for some |p|; this array is used to ensure that distinct nodes in the compressed trie will have distinct |trie←ref| entries. @d trie←ref==trie←hash {where linked trie families go into |trie|} @d trie←back(#)==trie[#].lh {backward links in |trie| holes} @<Glob...@>= @!init@!trie←taken:packed array[trie←pointer] of boolean; {does a family start here?} @t\hskip1em@>@!trie←min:trie←pointer; {all locations |<=trie←min| are vacant in |trie|} tini@;@/ @t\hskip1em@>@!trie←max:trie←pointer; {largest location used in |trie|} @ Here is how these data structures are initialized. @<Declare procedures for preprocessing hyph...@>= procedure init←trie←memory; {gets ready to pack into |trie|} var p:trie←pointer; {index into |trie←ref|, |trie|, |trie←taken|} begin for p:=0 to trie←ptr do trie←ref[p]:=0; trie←max:=128; trie←min:=128; trie←link(0):=1; trie←taken[0]:=false; for p:=1 to 128 do begin trie←back(p):=p-1; trie←link(p):=p+1; trie←taken[p]:=false; end; end; @ Each time \.{\\patterns} appears, it overrides any patterns that were entered earlier, so the arrays are not initialized until \TeX\ sees \.{\\patterns}. However, some of the global variables must be initialized when \.{INITEX} is loaded, in case the user never mentions any \.{\\patterns}. @<Initialize table entries...@>= trie←op←ptr:=min←quarterword;@/ trie←link(0):=0; trie←char(0):=0; trie←op(0):=0; for k:=1 to 127 do trie[k]:=trie[0]; trie←max:=127; @ The |first←fit| procedure finds the smallest hole |z| in |trie| such that a trie family starting at a given node |p| will fit into vacant positions starting at |z|. If |c=trie←c[p]|, this means that location |z-c| must not already be taken by some other family, and that |z-c+@t$c↑\prime$@>| must be vacant for all characters $c↑\prime$ in the family. The procedure sets |trie←ref[p]| to |z-c| when the first fit has been found. @<Declare procedures for preprocessing hyph...@>= procedure first←fit(@!p:trie←pointer); {packs a family into |trie|} label not←found,found; var h:trie←pointer; {candidate for |trie←ref[p]|} @!z:trie←pointer; {runs through holes} @!q:trie←pointer; {runs through the family starting at |p|} @!c:ASCII←code; {smallest character in the family} begin c:=trie←c[p]; {we have |c>0|} if c<trie←min then trie←min:=c; z:=trie←link(trie←min-1); {get the first conceivably good hole} loop@+ begin if z<c then goto not←found; h:=z-c;@/ @<Ensure that |trie←max>=h+128|@>; if trie←taken[h] then goto not←found; @<If all characters of the family fit relative to |h|, then |goto found|,\30\ otherwise |goto not←found|@>; not←found: z:=trie←link(z); {move to the next hole} end; found: @<Pack the family into |trie| relative to |h|@>; end; @ By making sure that |trie←max| is at least |h+128|, we can be sure that |trie←max>z|, since |h=z+c|. It follows that location |trie←max| will never be occupied in |trie|, and we will have |trie←max>=trie←link(z)|. @<Ensure that |trie←max>=h+128|@>= if trie←max<h+128 then begin if trie←size<=h+128 then overflow("pattern memory",trie←size); @:TeX capacity exceeded pattern memory}{\quad pattern memory@> repeat incr(trie←max); trie←taken[trie←max]:=false; trie←link(trie←max):=trie←max+1; trie←back(trie←max):=trie←max-1; until trie←max=h+128; end @ @<If all characters of the family fit relative to |h|...@>= q:=trie←r[p]; while q>0 do begin if trie←link(h+trie←c[q])=0 then goto not←found; q:=trie←r[q]; end; goto found @ @<Pack the family into |trie| relative to |h|@>= trie←taken[h]:=true; trie←ref[p]:=h; q:=p; repeat z:=h+trie←c[q]; trie←back(trie←link(z)):=trie←back(z); trie←link(trie←back(z)):=trie←link(z); trie←link(z):=0; q:=trie←r[q]; until q=0 @ To pack the entire linked trie, we use the following recursive procedure. @↑recursion@> @<Declare procedures for preprocessing hyph...@>= procedure trie←pack(@!p:trie←pointer); {pack subtries of a family} var q:trie←pointer; {a local variable that need not be saved on recursive calls} begin repeat q:=trie←l[p]; if (q>0)and(trie←ref[q]=0) then begin first←fit(q); trie←pack(q); end; p:=trie←r[p]; until p=0; end; @ When the whole trie has been allocated into the sequential table, we must go through it once again so that |trie| contains the correct information. Null pointers in the linked trie will be replaced by the first untaken position |r| in |trie|, since this properly implements an ``empty'' family. The value of |r| is stored in |trie←ref[0]| just before the fixup process starts. Note that |trie←max| will always be at least as large as |r+127|, since it is always at least 128 more than each location that is taken. @<Move the data into |trie|@>= r:=0; while trie←taken[r] do incr(r); trie←ref[0]:=r; {|r| will be used for null pointers} trie←fix(trie←root) {this fixes the non-holes in |trie|} @ The fixing-up procedure is, of course, recursive. Since the linked trie usually has overlapping subtries, the same data may be moved several times; but that causes no harm, and at most as much work is done as it took to build the uncompressed trie. @↑recursion@> @<Declare procedures for preprocessing hyph...@>= procedure trie←fix(@!p:trie←pointer); {moves |p| and its siblings into |trie|} var q:trie←pointer; {a local variable that need not be saved on recursive calls} @!c:ASCII←code; {another one that need not be saved} @!z:trie←pointer; {|trie| reference; this local variable must be saved} begin z:=trie←ref[p]; while p<>0 do begin q:=trie←l[p]; c:=trie←c[p]; trie←link(z+c):=trie←ref[q]; trie←char(z+c):=c; trie←op(z+c):=trie←o[p]; if q>0 then trie←fix(q); p:=trie←r[p]; end; end; @ Now let's put all these routines together. When \.{INITEX} has scanned the `\.{\\patterns}' control sequence, it calls on |new←patterns| to do the right thing. After |new←patterns| has acted, the compacted pattern data will appear in the array |trie[1..trie←max]|, and the associated numeric hyphenation data will appear in locations |[(min←quarterword+1)..trie←op←ptr]| of the arrays |hyf←distance|, |hyf←num|, |hyf←next|. @<Declare procedures for preprocessing hyph...@>= procedure new←patterns; {initializes the hyphenation pattern data} label done, done1; var k,@!l:small←number; {indices into |hc| and |hyf|} @!digit←sensed:boolean; {should the next digit be treated as a letter?} @!v:quarterword; {trie op code} @!p,@!q:trie←pointer; {nodes of trie traversed during insertion} @!first←child:boolean; {is |p=trie←l[q]|?} @!c:ASCII←code; {character being inserted} @!r,@!s:trie←pointer; {used to clean up the packed |trie|} @!h:two←halves; {template used to zero out |trie|'s holes} begin scan←left←brace; {a left brace must follow \.{\\patterns}} init←pattern←memory;@/ @<Enter all of the patterns into a linked trie, until coming to a right brace; then skip an optional space@>; trie←root:=compress←trie(trie←root); {compress the trie} @<Pack the trie@>; end; @ Novices are not supposed to be using \.{\\patterns}, so the error messages are terse. (Note that all error messages appear in \TeX's string pool, even if they are used only by \.{INITEX}.) @<Enter all of the patterns into a linked trie...@>= k:=0; hyf[0]:=0; digit←sensed:=false; loop@+ begin get←x←token; case cur←cmd of letter,other←char:@<Append a new letter or a hyphen level@>; spacer,right←brace: begin if k>0 then @<Insert a new pattern into the linked trie@>; if cur←cmd=right←brace then goto done; k:=0; hyf[0]:=0; digit←sensed:=false; end; othercases begin print←err("Bad "); print←esc("patterns"); @.Bad \\patterns@> help1("(See Appendix H.)"); error; end endcases; end; done: @ @<Append a new letter or a hyphen level@>= if digit←sensed or(cur←chr<"0")or(cur←chr>"9") then begin if cur←chr="." then cur←chr:=128 {edge-of-word delimiter} else begin cur←chr:=lc←code(cur←chr); if cur←chr=0 then begin print←err("Nonletter"); @.Nonletter@> help1("(See Appendix H.)"); error; cur←chr:=128; end; end; if k<63 then begin incr(k); hc[k]:=cur←chr-1; hyf[k]:=0; digit←sensed:=false; end; end else begin hyf[k]:=cur←chr-"0"; if k<63 then digit←sensed:=true; end @ When the following code comes into play, the pattern $p←1\ldots p←k$ appears in |hc[1..k]|, and the corresponding sequence of numbers $n←0\ldots n←k$ appears in |hyf[0..k]|. @<Insert a new pattern into the linked trie@>= begin @<Compute the trie op code, |v|, and set |l:=0|@>; q:=0; while l<k do begin incr(l); c:=hc[l]; p:=trie←l[q]; first←child:=true; while (p>0)and(c>trie←c[p]) do begin q:=p; p:=trie←r[q]; first←child:=false; end; if (p=0)or(c<trie←c[p]) then @<Insert a new trie node between |q| and |p|, and make |p| point to it@>; q:=p; {now node |q| represents $p←1\ldots p←l$} end; if trie←o[q]<>min←quarterword then begin print←err("Duplicate pattern"); @.Duplicate pattern@> help1("(See Appendix H.)"); error; end; trie←o[q]:=v; end @ @<Insert a new trie node between |q| and |p|...@>= begin if trie←ptr=trie←size then overflow("pattern memory",trie←size); @:TeX capacity exceeded pattern memory}{\quad pattern memory@> incr(trie←ptr); trie←r[trie←ptr]:=p; p:=trie←ptr; trie←l[p]:=0; if first←child then trie←l[q]:=p@+else trie←r[q]:=p; trie←c[p]:=c; trie←o[p]:=min←quarterword; end @ @<Compute the trie op code, |v|...@>= if hc[1]=127 then hyf[0]:=0; if hc[k]=127 then hyf[k]:=0; l:=k; v:=min←quarterword; loop@+ begin if hyf[l]<>0 then v:=new←trie←op(k-l,hyf[l],v); if l>0 then decr(l)@+else goto done1; end; done1: @ The following packing routine is rigged so that the root of the linked tree gets mapped into location 0 of |trie|, as required by the hyphenation algorithm. This happens because the first call of |first←fit| will ``take'' location~0. @<Pack the trie@>= init←trie←memory; if trie←root<>0 then begin first←fit(trie←root); trie←pack(trie←root); end; @<Move the data into |trie|@>; r:=0; {finally, we will zero out the holes} h.rh:=0; h.b0:=min←quarterword; h.b1:=0; {|trie←link:=0|, |trie←op:=min←quarterword|, |trie←char:=0|} repeat s:=trie←link(r); trie[r]:=h; r:=s; until r>trie←max @* \[44] Breaking vertical lists into pages. The |vsplit| procedure, which implements \TeX's \.{\\vsplit} operation, is considerably simpler than |line←break| because it doesn't have to worry about hyphenation, and because its mission is to discover a single break instead of an optimum sequence of breakpoints. But before we get into the details of |vsplit|, we need to consider a few more basic things. @ A subroutine called |prune←page←top| takes a pointer to a vlist and returns a pointer to a modified vlist in which all glue, kern, and penalty nodes have been deleted before the first box or rule node. However, the first box or rule is actually preceded by a newly created glue node designed so that the topmost baseline will be at distance |split←top←skip| from the top, whenever this is possible without backspacing. In this routine and those that follow, we make use of the fact that a vertical list contains no character nodes, hence the |type| field exists for each node in the list. @↑data structure assumptions@> @p function prune←page←top(@!p:pointer):pointer; {adjust top after page break} var prev←p:pointer; {lags one step behind |p|} @!q:pointer; {temporary variable for list manipulation} begin prev←p:=temp←head; link(temp←head):=p; while p<>null do case type(p) of hlist←node,vlist←node,rule←node:@<Insert glue for |split←top←skip| and set~|p:=null|@>; whatsit←node,mark←node,ins←node: begin prev←p:=p; p:=link(prev←p); end; glue←node,kern←node,penalty←node: begin q:=p; p:=link(q); link(q):=null; link(prev←p):=p; flush←node←list(q); end; othercases confusion("pruning") @:this can't happen pruning}{\quad pruning@> endcases; prune←page←top:=link(temp←head); end; @ @<Insert glue for |split←top←skip|...@>= begin q:=new←skip←param(split←top←skip←code); link(prev←p):=q; link(q):=p; {now |temp←ptr=glue←ptr(q)|} if width(temp←ptr)>height(p) then width(temp←ptr):=width(temp←ptr)-height(p) else width(temp←ptr):=0; p:=null; end @ The next subroutine finds the best place to break a given vertical list so as to obtain a box of height~|h|, with maximum depth~|d|. A pointer to the beginning of the vertical list is given, and a pointer to the optimum breakpoint is returned. The list is effectively followed by a forced break, i.e., a penalty node with the |eject←penalty|; if the best break occurs at this artificial node, the value |null| is returned. An array of six |scaled| distances is used to keep track of the height from the beginning of the list to the current place, just as in |line←break|. In fact, we use one of the same arrays, only changing its name to reflect its new significance. @d active←height==active←width {new name for the six distance variables} @d cur←height==active←height[1] {the natural height} @d set←height←zero(#)==active←height[#]:=0 {initialize the height to zero} @# @d update←heights=90 {go here to record glue in the |active←height| table} @p function vert←break(@!p:pointer; @!h,@!d:scaled):pointer; {finds optimum page break} label done,not←found,update←heights; var prev←p:pointer; {if |p| is a glue node, |type(prev←p)| determines whether |p| is a legal breakpoint} @!q,@!r:pointer; {glue specifications} @!pi:integer; {penalty value} @!b:integer; {badness at a trial breakpoint} @!least←cost:integer; {the smallest badness plus penalties found so far} @!best←place:pointer; {the most recent break that leads to |least←cost|} @!prev←dp:scaled; {depth of previous box in the list} @!t:small←number; {|type| of the node following a kern} begin prev←p:=p; {an initial glue node is not a legal breakpoint} least←cost:=awful←bad; do←all←six(set←height←zero); prev←dp:=0; loop@+ begin @<If node |p| is a legal breakpoint, check if this break is the best known, and |goto done| if |p| is null or if the page-so-far is already too full to accept more stuff@>; prev←p:=p; p:=link(prev←p); end; done: vert←break:=best←place; end; @ A global variable |best←height←plus←depth| will be set to the natural size of the box that corresponds to the optimum breakpoint found by |vert←break|. (This value is used by the insertion-splitting algorithm of the page builder.) @<Glob...@>= @!best←height←plus←depth:scaled; {height of the best box, without stretching or shrinking} @ A subtle point to be noted here is that the maximum depth~|d| might be negative, so |cur←height| and |prev←dp| might need to be corrected even after a glue or kern node. @<If node |p| is a legal breakpoint, check...@>= if p=null then pi:=eject←penalty else @<Use node |p| to update the current height and depth measurements; if this node is not a legal breakpoint, |goto not←found| or |update←heights|, otherwise set |pi| to the associated penalty at the break@>; @<Check if node |p| is a new champion breakpoint; then \(go)|goto done| if |p| is a forced break or if the page-so-far is already too full@>; if (type(p)<glue←node)or(type(p)>kern←node) then goto not←found; update←heights: @<Update the current height and depth measurements with respect to a glue or kern node~|p|@>; not←found: if prev←dp>d then begin cur←height:=cur←height+prev←dp-d; prev←dp:=d; end; @ @<Use node |p| to update the current height and depth measurements...@>= case type(p) of hlist←node,vlist←node,rule←node: begin@t@>@;@/ cur←height:=cur←height+prev←dp+height(p); prev←dp:=depth(p); goto not←found; end; whatsit←node:@<Process whatsit |p| in |vert←break| loop, |goto not←found|@>; glue←node: if precedes←break(prev←p) then pi:=0 else goto update←heights; kern←node: begin if link(p)=null then t:=penalty←node else t:=type(link(p)); if t=glue←node then pi:=0@+else goto update←heights; end; penalty←node: pi:=penalty(p); mark←node,ins←node: goto not←found; othercases confusion("vertbreak") @:this can't happen vertbreak}{\quad vertbreak@> endcases @ @d deplorable==100000 {more than |inf←bad|, but less than |awful←bad|} @<Check if node |p| is a new champion breakpoint; then \(go)...@>= if pi<inf←penalty then begin @<Compute the badness, |b|, using |awful←bad| if the box is too full@>; if b<awful←bad then if pi<=eject←penalty then b:=pi else if b<inf←bad then b:=b+pi else b:=deplorable; if b<=least←cost then begin best←place:=p; least←cost:=b; best←height←plus←depth:=cur←height+prev←dp; end; if (b=awful←bad)or(pi<=eject←penalty) then goto done; end @ @<Compute the badness, |b|, using |awful←bad| if the box is too full@>= if cur←height<h then if (active←height[3]<>0) or (active←height[4]<>0) or (active←height[5]<>0) then b:=0 else b:=badness(h-cur←height,active←height[2]) else if cur←height-h>active←height[6] then b:=awful←bad else b:=badness(cur←height-h,active←height[6]) @ Vertical lists that are subject to the |vert←break| procedure should not contain infinite shrinkability, since that would permit any amount of information to ``fit'' on one page. @<Update the current height and depth measurements with...@>= if type(p)=kern←node then q:=p else begin q:=glue←ptr(p); active←height[2+stretch←order(q)]:=@| active←height[2+stretch←order(q)]+stretch(q);@/ active←height[6]:=active←height[6]+shrink(q); if (shrink←order(q)<>normal)and(shrink(q)<>0) then begin@t@>@;@/ print←err("Infinite glue shrinkage found in box being split");@/ @.Infinite glue shrinkage...@> help4("The box you are \vsplitting contains some infinitely")@/ ("shrinkable glue, e.g., `\vss' or `\vskip 0pt minus 1fil'.")@/ ("Such glue doesn't belong there; but you can safely proceed,")@/ ("since the offensive shrinkability has been made finite."); error; r:=new←spec(q); shrink←order(r):=normal; delete←glue←ref(q); glue←ptr(p):=r; end; end; cur←height:=cur←height+prev←dp+width(q); prev←dp:=0 @ Now we are ready to consider |vsplit| itself. Most of its work is accomplished by the two subroutines that we have just considered. Given the number of a vlist box |n|, and given a desired page height |h|, the |vsplit| function finds the best initial segment of the vlist and returns a box for a page of height~|h|. The remainder of the vlist, if any, replaces the original box, after removing glue and penalties and adjusting for |split←top←skip|. Mark nodes in the split-off box are used to set the values of |split←first←mark| and |split←bot←mark|; we use the fact that |split←first←mark=null| if and only if |split←bot←mark=null|. The original box becomes ``void'' if and only if it has been entirely extracted. The extracted box is ``void'' if and only if the original box was void (or if it was, erroneously, an hlist box). @p function vsplit(@!n:eight←bits; @!h:scaled):pointer; {extracts a page of height |h| from box |n|} label exit,done; var v:pointer; {the box to be split} p:pointer; {runs through the vlist} q:pointer; {points to where the break occurs} begin v:=box(n); if split←first←mark<>null then begin delete←token←ref(split←first←mark); split←first←mark:=null; delete←token←ref(split←bot←mark); split←bot←mark:=null; end; @<Dispense with trivial cases of void or bad boxes@>; q:=vert←break(list←ptr(v),h,split←max←depth); @<Look at all the marks in nodes before the break, and set the final link to |null| at the break@>; q:=prune←page←top(q); p:=list←ptr(v); free←node(v,box←node←size); if q=null then box(n):=null {the |eq←level| of the box stays the same} else box(n):=vpack(q,natural); vsplit:=vpackage(p,h,exactly,split←max←depth); exit: end; @ @<Dispense with trivial cases of void or bad boxes@>= if v=null then begin vsplit:=null; return; end; if type(v)<>vlist←node then begin print←err(""); print←esc("vsplit"); print(" needs a "); print←esc("vbox"); @:vsplit←}{\.{\\vsplit needs a \\vbox}@> help2("The box you are trying to split is an \hbox.")@/ ("I can't split such a box, so I'll leave it alone."); error; vsplit:=null; return; end @ It's possible that the box begins with a penalty node that is the ``best'' break, so we must be careful to handle this special case correctly. @<Look at all the marks...@>= p:=list←ptr(v); if p=q then list←ptr(v):=null else loop@+begin if type(p)=mark←node then if split←first←mark=null then begin split←first←mark:=mark←ptr(p); split←bot←mark:=split←first←mark; token←ref←count(split←first←mark):=@| token←ref←count(split←first←mark)+2; end else begin delete←token←ref(split←bot←mark); split←bot←mark:=mark←ptr(p); add←token←ref(split←bot←mark); end; if link(p)=q then begin link(p):=null; goto done; end; p:=link(p); end; done: @* \[45] The page builder. When \TeX\ appends new material to its main vlist in vertical mode, it uses a method something like |vsplit| to decide where a page ends, except that the calculations are done ``on line'' as new items are placed on the list. The main complication in this process is that insertions have to be put into their boxes and removed from the vlist, in a more-or-less optimum manner. We shall use the term ``current page'' for that part of the main vlist that is being considered as a candidate for being broken off and sent to the user's output routine. The current page starts at |link(page←head)|, and it ends at |page←tail|. We have |page←head=page←tail| if this list is empty. @↑current page@> Utter chaos would reign if the user kept changing page specifications while a page is being constructed, so the page builder keeps the pertinent specifications frozen as soon as the page receives its first box or insertion. The global variable |page←contents| is |empty| when the current page contains only mark nodes and content-less whatsit nodes; it is |inserts←only| if the page contains only insertion nodes in addition to marks and whatsits. Glue nodes, kern nodes, and penalty nodes are discarded until a box or rule node appears, at which time |page←contents| changes to |box←there|. As soon as |page←contents| becomes non-|empty|, the current |vsize| and |max←depth| are squirreled away into |page←goal| and |page←max←depth|; the latter values will be used until the page has been forwarded to the user's output routine. The \.{\\topskip} adjustment is made when |page←contents| changes to |box←there|. Although |page←goal| starts out equal to |vsize|, it is decreased by the scaled natural height-plus-depth of the insertions considered so far, and by the \.{\\skip} corrections for those insertions. Therefore it represents the size into which the non-inserted material should fit, assuming that all insertions in the current page have been made. The variables |best←page←break| and |least←page←cost| correspond respectively to the variables |best←place| and |least←cost| in the |vert←break| routine that we have already studied; i.e., they record the location and value of the best place currently known for breaking the current page. The value of |page←goal| at the time of the best break is stored in |best←size|. @d inserts←only=1 {|page←contents| when an insert node has been contributed, but no boxes} @d box←there=2 {|page←contents| when a box or rule has been contributed} @<Glob...@>= @!page←tail:pointer; {the final node on the current page} @!page←contents:empty..box←there; {what is on the current page so far?} @!page←max←depth:scaled; {maximum box depth on page being built} @!best←page←break:pointer; {break here to get the best page known so far} @!least←page←cost:integer; {the score for this currently best page} @!best←size:scaled; {its |page←goal|} @ The page builder has another data structure to keep track of insertions. This is a list of four-word nodes, starting and ending at |page←ins←head|. That is, the first element of the list is node |r@t$←1$@>=link(page←ins←head)|; node $r←j$ is followed by |r@t$←{j+1}$@>=link(r@t$←j$@>)|; and if there are |n| items we have |r@t$←{n+1}$@>=page←ins←head|. The |subtype| field of each node in this list refers to an insertion number; for example, `\.{\\insert 250}' would correspond to a node whose |subtype| is |qi(250)| (the same as the |subtype| field of the relevant |ins←node|). These |subtype| fields are in increasing order, and |subtype(page←ins←head)= qi(255)|, so |page←ins←head| serves as a convenient sentinel at the end of the list. A record is present for each insertion number that appears in the current page. The |type| field in these nodes distinguishes two possibilities that might occur as we look ahead before deciding on the optimum page break. If |type(r)=inserting|, then |height(r)| contains the total of the height-plus-depth dimensions of the box and all its inserts seen so far. If |type(r)=split←up|, then no more insertions will be made into this box, because at least one previous insertion was too big to fit on the current page; |broken←ptr(r)| points to the node where that insertion will be split, if \TeX\ decides to split it, |broken←ins(r)| points to the insertion node that was tentatively split, and |height(r)| includes also the natural height plus depth of the part that would be split off. In both cases, |last←ins←ptr(r)| points to the last |ins←node| encountered for box |qo(subtype(r))| that would be at least partially inserted on the next page; and |best←ins←ptr(r)| points to the last such |ins←node| that should actually be inserted, to get the page with minimum badness among all page breaks considered so far. We have |best←ins←ptr(r)=null| if and only if no insertion for this box should be made to produce this optimum page. The data structure definitions here use the fact that the |@!height| field appears in the fourth word of a box node. @↑data structure assumptions@> @d page←ins←node←size=4 {number of words for a page insertion node} @d inserting=0 {an insertion class that has not yet overflowed} @d split←up=1 {an overflowed insertion class} @d broken←ptr(#)==link(#+1) {an insertion for this class will break here if anywhere} @d broken←ins(#)==info(#+1) {this insertion might break at |broken←ptr|} @d last←ins←ptr(#)==link(#+2) {the most recent insertion for this |subtype|} @d best←ins←ptr(#)==info(#+2) {the optimum most recent insertion} @<Initialize the special list heads...@>= subtype(page←ins←head):=qi(255); type(page←ins←head):=split←up; link(page←ins←head):=page←ins←head; @ An array |page←so←far| records the heights and depths of everything on the current page. This array contains six |scaled| numbers, like the similar arrays already considered in |line←break| and |vert←break|; and it also contains |page←goal| and |page←depth|, since these values are all accessible to the user via |set←page←dimen| commands. The value of |page←so←far[1]| is also called |page←total|. The stretch and shrink components of the \.{\\skip} corrections for each insertion are included in |page←so←far|, but the natural space components of these corrections are not, since they have been subtracted from |page←goal|. The variable |page←depth| records the depth of the current page; it has been adjusted so that it is at most |page←max←depth|. The variable |last←glue| points to the glue specification of the most recent node contributed from the contribution list, if this was a glue node; otherwise |last←glue=max←halfword|. (If the contribution list is nonempty, however, the value of |last←glue| is not necessarily accurate.) The variables |last←penalty| and |last←kern| are similar. And finally, |insert←penalties| holds the sum of the penalties associated with all split and floating insertions. @d page←goal==page←so←far[0] {desired height of information on page being built} @d page←total==page←so←far[1] {height of the current page} @d page←shrink==page←so←far[6] {shrinkability of the current page} @d page←depth==page←so←far[7] {depth of the current page} @<Glob...@>= @!page←so←far:array [0..7] of scaled; {height and glue of the current page} @!last←glue:pointer; {used to implement \.{\\lastskip}} @!last←penalty:integer; {used to implement \.{\\lastpenalty}} @!last←kern:scaled; {used to implement \.{\\lastkern}} @!insert←penalties:integer; {sum of the penalties for held-over insertions} @ @<Put each...@>= primitive("pagegoal",set←page←dimen,0); @!@:page←goal←}{\.{\\pagegoal} primitive@> primitive("pagetotal",set←page←dimen,1); @!@:page←total←}{\.{\\pagetotal} primitive@> primitive("pagestretch",set←page←dimen,2); @!@:page←stretch←}{\.{\\pagestretch} primitive@> primitive("pagefilstretch",set←page←dimen,3); @!@:page←fil←stretch←}{\.{\\pagefilstretch} primitive@> primitive("pagefillstretch",set←page←dimen,4); @!@:page←fill←stretch←}{\.{\\pagefillstretch} primitive@> primitive("pagefilllstretch",set←page←dimen,5); @!@:page←filll←stretch←}{\.{\\pagefilllstretch} primitive@> primitive("pageshrink",set←page←dimen,6); @!@:page←shrink←}{\.{\\pageshrink} primitive@> primitive("pagedepth",set←page←dimen,7); @!@:page←depth←}{\.{\\pagedepth} primitive@> @ @<Cases of |print←cmd←chr|...@>= set←page←dimen: case chr←code of 0: print←esc("pagegoal"); 1: print←esc("pagetotal"); 2: print←esc("pagestretch"); 3: print←esc("pagefilstretch"); 4: print←esc("pagefillstretch"); 5: print←esc("pagefilllstretch"); 6: print←esc("pageshrink"); othercases print←esc("pagedepth") endcases; @ @d print←plus←end(#)==print(#);@+end @d print←plus(#)==if page←so←far[#]<>0 then begin print(" plus "); print←scaled(page←so←far[#]); print←plus←end @p procedure print←totals; begin print←scaled(page←total); print←plus(2)(""); print←plus(3)("fil"); print←plus(4)("fill"); print←plus(5)("filll"); if page←shrink<>0 then begin print(" minus "); print←scaled(page←shrink); end; end; @ @<Show the status of the current page@>= if page←head<>page←tail then begin print←nl("### current page:"); if output←active then print(" (held over for next output)"); @.held over for next output@> show←box(link(page←head)); if page←contents>empty then begin print←nl("total height "); print←totals; @:total←height}{\.{total height}@> print←nl(" goal height "); print←scaled(page←goal); @.goal height@> r:=link(page←ins←head); while r<>page←ins←head do begin print←ln; print←esc("insert"); t:=qo(subtype(r)); print←int(t); print(" adds "); t:=x←over←n(height(r),1000)*count(t); print←scaled(t); if type(r)=split←up then begin q:=page←head; t:=0; repeat q:=link(q); if (type(q)=ins←node)and(subtype(q)=subtype(r)) then incr(t); until q=broken←ins(r); print(", #"); print←int(t); print(" might split"); end; r:=link(r); end; end; end @ Here is a procedure that is called when the |page←contents| is changing from |empty| to |inserts←only| or |box←there|. @d set←page←so←far←zero(#)==page←so←far[#]:=0 @p procedure freeze←page←specs(@!s:small←number); begin page←contents:=s; page←goal:=vsize; page←max←depth:=max←depth; page←depth:=0; do←all←six(set←page←so←far←zero); least←page←cost:=awful←bad; @!stat if tracing←pages>0 then begin begin←diagnostic; print←nl("%% goal height="); print←scaled(page←goal); @.goal height@> print(", max depth="); print←scaled(page←max←depth); end←diagnostic(false); end;@;@+tats@;@/ end; @ Pages are built by appending nodes to the current list in \TeX's vertical mode, which is at the outermost level of the semantic nest. This vlist is split into two parts; the ``current page'' that we have been talking so much about already, and the ``contribution list'' that receives new nodes as they are created. The current page contains everything that the page builder has accounted for in its data structures, as described above, while the contribution list contains other things that have been generated by other parts of \TeX\ but not yet seen by the page builder. The contribution list starts at |link(contrib←head)|, and it ends at the current node in \TeX's vertical mode. When \TeX\ has appended new material in vertical mode, it calls the procedure |build←page|, which tries to catch up by moving nodes from the contribution list to the current page. This procedure will succeed in its goal of emptying the contribution list, unless a page break is discovered, i.e., unless the current page has grown to the point where the optimum next page break has been determined. In the latter case, the nodes after the optimum break will go back onto the contribution list, and control will effectively pass to the user's output routine. We make |type(page←head)=glue←node|, so that an initial glue node on the current page will not be considered a valid breakpoint. @<Initialize the special list...@>= type(page←head):=glue←node; subtype(page←head):=normal; @ The global variable |output←active| is true during the time the user's output routine is driving \TeX. @<Glob...@>= @!output←active:boolean; {are we in the midst of an output routine?} @ @<Set init...@>= output←active:=false; insert←penalties:=0; @ The page builder is ready to start a fresh page if we initialize the following state variables. (However, the page insertion list is initialized elsewhere.) @<Start a new current page@>= page←contents:=empty; page←tail:=page←head; link(page←head):=null;@/ last←glue:=max←halfword; last←penalty:=0; last←kern:=0; page←depth:=0; page←max←depth:=0 @ At certain times box 255 is supposed to be void (i.e., |null|), or an insertion box is supposed to be ready to accept a vertical list. If not, an error message is printed, and the following subroutine flushes the unwanted contents, reporting them to the user. @p procedure box←error(@!n:eight←bits); begin error; begin←diagnostic; print←nl("The following box has been deleted:"); @.The following...deleted@> show←box(box(n)); end←diagnostic(true); flush←node←list(box(n)); box(n):=null; end; @ The following procedure guarantees that a given box register does not contain an \.{\\hbox}. @p procedure ensure←vbox(@!n:eight←bits); var p:pointer; {the box register contents} begin p:=box(n); if p<>null then if type(p)=hlist←node then begin print←err("Insertions can only be added to a vbox"); @.Insertions can only...@> help3("Tut tut: You're trying to \insert into a")@/ ("\box register that now contains an \hbox.")@/ ("Proceed, and I'll discard its present contents."); box←error(n); end; end; @ \TeX\ is not always in vertical mode at the time |build←page| is called; the current mode reflects what \TeX\ should return to, after the contribution list has been emptied. A call on |build←page| should be immediately followed by `|goto big←switch|', which is \TeX's central control point. @d contribute=80 {go here to link a node into the current page} @p @<Declare the procedure called |fire←up|@>@;@/ procedure build←page; {append contributions to the current page} label exit,done,done1,continue,contribute,update←heights; var p:pointer; {the node being appended} @!q,@!r:pointer; {nodes being examined} @!b,c:integer; {badness and cost of current page} @!pi:integer; {penalty to be added to the badness} @!n:min←quarterword..255; {insertion box number} @!delta,@!h,@!w:scaled; {sizes used for insertion calculations} begin if (link(contrib←head)=null)or output←active then return; repeat continue: p:=link(contrib←head);@/ @<Update the values of |last←glue|, |last←penalty|, and |last←kern|@>; @<Move node |p| to the current page; if it is time for a page break, put the nodes following the break back onto the contribution list, and |return| to the user's output routine if there is one@>; until link(contrib←head)=null; @<Make the contribution list empty by setting its tail to |contrib←head|@>; exit:end; @ @d contrib←tail==nest[0].tail←field {tail of the contribution list} @<Make the contribution list empty...@>= if nest←ptr=0 then tail:=contrib←head {vertical mode} else contrib←tail:=contrib←head {other modes} @ @<Update the values of |last←glue|...@>= if last←glue<>max←halfword then delete←glue←ref(last←glue); last←penalty:=0; last←kern:=0; if type(p)=glue←node then begin last←glue:=glue←ptr(p); add←glue←ref(last←glue); end else begin last←glue:=max←halfword; if type(p)=penalty←node then last←penalty:=penalty(p) else if type(p)=kern←node then last←kern:=width(p); end @ The code here is an example of a many-way switch into routines that merge together in different places. Some people call this unstructured programming, but the author doesn't see much wrong with it, as long as @↑Knuth, Donald Ervin@> the various labels have a well-understood meaning. @<Move node |p| to the current page; ...@>= @<If the current page is empty and node |p| is to be deleted, |goto done1|; otherwise use node |p| to update the state of the current page; if this node is an insertion, |goto contribute|; otherwise if this node is not a legal breakpoint, |goto contribute| or |update←heights|; otherwise set |pi| to the penalty associated with this breakpoint@>; @<Check if node |p| is a new champion breakpoint; then \(if)if it is time for a page break, prepare for output, and either fire up the user's output routine and |return| or ship out the page and |goto done|@>; if (type(p)<glue←node)or(type(p)>kern←node) then goto contribute; update←heights:@<Update the current page measurements with respect to the glue or kern specified by node~|p|@>; contribute: @<Make sure that |page←max←depth| is not exceeded@>; @<Link node |p| into the current page and |goto done|@>; done1:@<Recycle node |p|@>; done: @ @<Link node |p| into the current page and |goto done|@>= link(page←tail):=p; page←tail:=p; link(contrib←head):=link(p); link(p):=null; goto done @ @<Recycle node |p|@>= link(contrib←head):=link(p); link(p):=null; flush←node←list(p) @ The title of this section is already so long, it seems best to avoid making it more accurate but still longer, by mentioning the fact that a kern node at the end of the contribution list will not be contributed until we know its successor. @<If the current page is empty...@>= case type(p) of hlist←node,vlist←node,rule←node: if page←contents<box←there then @<Initialize the current page, insert the \.{\\topskip} glue ahead of |p|, and |goto continue|@> else @<Prepare to move a box or rule node to the current page, then |goto contribute|@>; whatsit←node: @<Prepare to move whatsit |p| to the current page, then |goto contribute|@>; glue←node: if page←contents<box←there then goto done1 else if precedes←break(page←tail) then pi:=0 else goto update←heights; kern←node: if page←contents<box←there then goto done1 else if link(p)=null then return else if type(link(p))=glue←node then pi:=0 else goto update←heights; penalty←node: if page←contents<box←there then goto done1@+else pi:=penalty(p); mark←node: goto contribute; ins←node: @<Append an insertion to the current page and |goto contribute|@>; othercases confusion("page") @:this can't happen page}{\quad page@> endcases @ @<Initialize the current page, insert the \.{\\topskip} glue...@>= begin if page←contents=empty then freeze←page←specs(box←there) else page←contents:=box←there; q:=new←skip←param(top←skip←code); link(q):=p; {now |temp←ptr=glue←ptr(q)|} if width(temp←ptr)>height(p) then width(temp←ptr):=width(temp←ptr)-height(p) else width(temp←ptr):=0; link(q):=p; link(contrib←head):=q; goto continue; end @ @<Prepare to move a box or rule node to the current page...@>= begin page←total:=page←total+page←depth+height(p); page←depth:=depth(p); goto contribute; end @ @<Make sure that |page←max←depth| is not exceeded@>= if page←depth>page←max←depth then begin page←total:=@| page←total+page←depth-page←max←depth;@/ page←depth:=page←max←depth; end; @ @<Update the current page measurements with respect to the glue...@>= if type(p)=kern←node then q:=p else begin q:=glue←ptr(p); page←so←far[2+stretch←order(q)]:=@| page←so←far[2+stretch←order(q)]+stretch(q);@/ page←shrink:=page←shrink+shrink(q); if (shrink←order(q)<>normal)and(shrink(q)<>0) then begin@t@>@;@/ print←err("Infinite glue shrinkage found on current page");@/ @.Infinite glue shrinkage...@> help4("The page about to be output contains some infinitely")@/ ("shrinkable glue, e.g., `\vss' or `\vskip 0pt minus 1fil'.")@/ ("Such glue doesn't belong there; but you can safely proceed,")@/ ("since the offensive shrinkability has been made finite."); error; r:=new←spec(q); shrink←order(r):=normal; delete←glue←ref(q); glue←ptr(p):=r; end; end; page←total:=page←total+page←depth+width(q); page←depth:=0 @ @<Check if node |p| is a new champion breakpoint; then \(if)...@>= if pi<inf←penalty then begin @<Compute the badness, |b|, of the current page, using |awful←bad| if the box is too full@>; if b<awful←bad then if pi<=eject←penalty then c:=pi else if b<inf←bad then c:=b+pi+insert←penalties else c:=deplorable else c:=b; if insert←penalties>=10000 then c:=awful←bad; @!stat if tracing←pages>0 then @<Display page break cost@>;@+tats@;@/ if c<=least←page←cost then begin best←page←break:=p; best←size:=page←goal; least←page←cost:=c; r:=link(page←ins←head); while r<>page←ins←head do begin best←ins←ptr(r):=last←ins←ptr(r); r:=link(r); end; end; if (c=awful←bad)or(pi<=eject←penalty) then begin fire←up(p); {output the current page at the best place} if output←active then return; {user's output routine will act} goto done; {the page has been shipped out by default output routine} end; end @ @<Display page break cost@>= begin begin←diagnostic; print←nl("%"); print(" t="); print←totals;@/ print(" g="); print←scaled(page←goal);@/ print(" b="); if b=awful←bad then print←char("*")@+else print←int(b); print(" p="); print←int(pi); print(" c="); if c=awful←bad then print←char("*")@+else print←int(c); if c<=least←page←cost then print←char("#"); end←diagnostic(false); end @ @<Compute the badness, |b|, of the current page...@>= if page←total<page←goal then if (page←so←far[3]<>0) or (page←so←far[4]<>0) or@| (page←so←far[5]<>0) then b:=0 else b:=badness(page←goal-page←total,page←so←far[2]) else if page←total-page←goal>page←shrink then b:=awful←bad else b:=badness(page←total-page←goal,page←shrink) @ @<Append an insertion to the current page and |goto contribute|@>= begin if page←contents=empty then freeze←page←specs(inserts←only); n:=subtype(p); r:=page←ins←head; while n>=subtype(link(r)) do r:=link(r); n:=qo(n); if subtype(r)<>qi(n) then @<Create a page insertion node with |subtype(r)=qi(n)|, and include the glue correction for box |n| in the current page state@>; if type(r)=split←up then insert←penalties:=insert←penalties+float←cost(p) else begin last←ins←ptr(r):=p; delta:=page←goal-page←total-page←depth+page←shrink; {this much room is left if we shrink the maximum} if count(n)=1000 then h:=height(p) else h:=x←over←n(height(p),1000)*count(n); {this much room is needed} if ((h<=0)or(h<=delta))and(height(p)+height(r)<=dimen(n)) then begin page←goal:=page←goal-h; height(r):=height(r)+height(p); end else @<Find the best way to split the insertion, and change |type(r)| to |split←up|@>; end; goto contribute; end @ We take note of the value of \.{\\skip} |n| and the height plus depth of \.{\\box}~|n| only when the first \.{\\insert}~|n| node is encountered for a new page. A user who changes the contents of \.{\\box}~|n| after that first \.{\\insert}~|n| had better be either extremely careful or extremely lucky, or both. @<Create a page insertion node...@>= begin q:=get←node(page←ins←node←size); link(q):=link(r); link(r):=q; r:=q; subtype(r):=qi(n); type(r):=inserting; ensure←vbox(n); if box(n)=null then height(r):=0 else height(r):=height(box(n))+depth(box(n)); best←ins←ptr(r):=null;@/ q:=skip(n); if count(n)=1000 then h:=height(r) else h:=x←over←n(height(r),1000)*count(n); page←goal:=page←goal-h-width(q);@/ page←so←far[2+stretch←order(q)]:=@|page←so←far[2+stretch←order(q)]+stretch(q);@/ page←shrink:=page←shrink+shrink(q); if (shrink←order(q)<>normal)and(shrink(q)<>0) then begin print←err("Infinite glue shrinkage inserted from "); print←esc("skip"); @.Infinite glue shrinkage...@> print←int(n); help3("The correction glue for page breaking with insertions")@/ ("must have finite shrinkability. But you may proceed,")@/ ("since the offensive shrinkability has been made finite."); error; end; end @ Here is the code that will split a long footnote between pages, in an emergency. The current situation deserves to be recapitulated: Node |p| is an insertion into box |n|; the insertion will not fit, in its entirety, either because it would make the total contents of box |n| greater than \.{\\dimen} |n|, or because it would make the incremental amount of growth |h| greater than the available space |delta|, or both. (This amount |h| has been weighted by the insertion scaling factor, i.e., by \.{\\count} |n| over 1000.) Now we will choose the best way to break the vlist of the insertion, using the same criteria as in the \.{\\vsplit} operation. @<Find the best way to split the insertion...@>= begin if count(n)<=0 then w:=max←dimen else begin w:=page←goal-page←total-page←depth; if count(n)<>1000 then w:=x←over←n(w,count(n))*1000; end; if w>dimen(n)-height(r) then w:=dimen(n)-height(r); q:=vert←break(ins←ptr(p),w,depth(p)); height(r):=height(r)+best←height←plus←depth; @!stat if tracing←pages>0 then @<Display insertion split cost@>;@+tats@;@/ if count(n)<>1000 then best←height←plus←depth:=x←over←n(best←height←plus←depth,1000)*count(n); page←goal:=page←goal-best←height←plus←depth; type(r):=split←up; broken←ptr(r):=q; broken←ins(r):=p; if q=null then insert←penalties:=insert←penalties+eject←penalty else if type(q)=penalty←node then insert←penalties:=insert←penalties+penalty(q); end @ @<Display insertion split cost@>= begin begin←diagnostic; print←nl("% split"); print←int(n); @.split@> print(" to "); print←scaled(w); print←char(","); print←scaled(best←height←plus←depth);@/ print(" p="); if q=null then print←int(eject←penalty) else if type(q)=penalty←node then print←int(penalty(q)) else print←char("0"); end←diagnostic(false); end @ When the page builder has looked at as much material as could appear before the next page break, it makes its decision. The break that gave minimum badness will be used to put a completed ``page'' into box 255, with insertions appended to their other boxes. We also set the values of |top←mark|, |first←mark|, and |bot←mark|. The program uses the fact that |bot←mark<>null| implies |first←mark<>null|; |bot←mark=null| implies |top←mark=first←mark=null|. The |fire←up| subroutine prepares to output the current page at the best place; then it fires up the user's output routine, if there is one, or it simply ships out the page. There is one parameter, |c|, which represents the node that was being contributed to the page when the decision to force an output was made. @<Declare the procedure called |fire←up|@>= procedure fire←up(@!c:pointer); label exit; var p,@!q,@!r,@!s:pointer; {nodes being examined and/or changed} @!prev←p:pointer; {precedessor of |p|} @!n:min←quarterword..255; {insertion box number} @!wait:boolean; {should the present insertion be held over?} @!save←vbadness:integer; {saved value of |vbadness|} @!save←vfuzz: scaled; {saved value of |vfuzz|} @!save←split←top←skip: pointer; {saved value of |split←top←skip|} begin @<Set the value of |output←penalty|@>; if bot←mark<>null then begin if top←mark<>null then delete←token←ref(top←mark); top←mark:=bot←mark; add←token←ref(top←mark); delete←token←ref(first←mark); first←mark:=null; end; @<Put the \(o)optimal current page into box 255, update |first←mark| and |bot←mark|, append insertions to their boxes, and put the remaining nodes back on the contribution list@>; if (top←mark<>null)and(first←mark=null) then begin first←mark:=top←mark; add←token←ref(top←mark); end; if output←routine<>null then if dead←cycles>=max←dead←cycles then @<Explain that too many dead cycles have occurred in a row@> else @<Fire up the user's output routine and |return|@>; @<Perform the default output routine@>; exit:end; @ @<Set the value of |output←penalty|@>= if type(best←page←break)=penalty←node then begin geq←word←define(int←base+output←penalty←code,penalty(best←page←break)); penalty(best←page←break):=inf←penalty; end else geq←word←define(int←base+output←penalty←code,inf←penalty) @ As the page is finally being prepared for output, |p| runs through the vlist, with |prev←p| trailing behind, and |q| is the tail of a list of insertions that are being held over for a subsequent page. @<Put the \(o)optimal current page into box 255...@>= if c=best←page←break then best←page←break:=null; {|c| not yet linked in} @<Ensure that box 255 is empty before output@>; insert←penalties:=0; {this will count the number of insertions held over} save←split←top←skip:=split←top←skip; @<Prepare all the boxes involved in insertions to act as queues@>; q:=hold←head; link(q):=null; prev←p:=page←head; p:=link(prev←p); while p<>best←page←break do begin if type(p)=ins←node then @<Either insert the material specified by node |p| into the appropriate box, or hold it for the next page; also delete node |p| from the current page@> else if type(p)=mark←node then @<Update the values of |first←mark| and |bot←mark|@>; prev←p:=p; p:=link(prev←p); end; split←top←skip:=save←split←top←skip; @<Break the current page at node |p|, put it in box~255, and put the remaining nodes on the contribution list@>; @<Delete the page-insertion nodes@> @ @<Ensure that box 255 is empty before output@>= if box(255)<>null then begin print←err(""); print←esc("box"); print("255 is not void"); @:box255}{\.{\\box255 is not void}@> help2("You shouldn't use \box255 except in \output routines.")@/ ("Proceed, and I'll discard its present contents."); box←error(255); end @ @<Update the values of |first←mark| and |bot←mark|@>= begin if first←mark=null then begin first←mark:=mark←ptr(p); add←token←ref(first←mark); end; if bot←mark<>null then delete←token←ref(bot←mark); bot←mark:=mark←ptr(p); add←token←ref(bot←mark); end @ When the following code is executed, the current page runs from node |link(page←head)| to node |prev←p|, and the nodes from |p| to |page←tail| are to be placed back at the front of the contribution list. Furthermore the heldover insertions appear in a list from |link(hold←head)| to |q|; we will put them into the current page list for safekeeping while the user's output routine is active. We might have |q=hold←head|; and |p=null| if and only if |prev←p=page←tail|. Error messages are suppressed within |vpackage|, since the box might appear to be overfull or underfull simply because the stretch and shrink from the \.{\\skip} registers for inserts are not actually present in the box. @<Break the current page at node |p|, put it...@>= if p<>null then begin if link(contrib←head)=null then if nest←ptr=0 then tail:=page←tail else contrib←tail:=page←tail; link(page←tail):=link(contrib←head); link(contrib←head):=p; link(prev←p):=null; end; save←vbadness:=vbadness; vbadness:=inf←bad; save←vfuzz:=vfuzz; vfuzz:=max←dimen; {inhibit error messages} box(255):=vpackage(link(page←head),best←size,exactly,page←max←depth); vbadness:=save←vbadness; vfuzz:=save←vfuzz; if last←glue<>max←halfword then delete←glue←ref(last←glue); @<Start a new current page@>; {this sets |last←glue:=max←halfword|} if q<>hold←head then begin link(page←head):=link(hold←head); page←tail:=q; end @ If many insertions are supposed to go into the same box, we want to know the position of the last node in that box, so that we don't need to waste time when linking further information into it. The |last←ins←ptr| fields of the page insertion nodes are therefore used for this purpose during the packaging phase. @<Prepare all the boxes involved in insertions to act as queues@>= r:=link(page←ins←head); while r<>page←ins←head do begin if best←ins←ptr(r)<>null then begin n:=qo(subtype(r)); ensure←vbox(n); if box(n)=null then box(n):=new←null←box; p:=box(n)+list←offset; while link(p)<>null do p:=link(p); last←ins←ptr(r):=p; end; r:=link(r); end @ @<Delete the page-insertion nodes@>= r:=link(page←ins←head); while r<>page←ins←head do begin q:=link(r); free←node(r,page←ins←node←size); r:=q; end; link(page←ins←head):=page←ins←head @ We will set |best←ins←ptr:=null| and package the box corresponding to insertion node~|r|, just after making the final insertion into that box. If this final insertion is `|split←up|', the remainder after splitting and pruning (if any) will be carried over to the next page. @<Either insert the material specified by node |p| into...@>= begin r:=link(page←ins←head); while subtype(r)<>subtype(p) do r:=link(r); if best←ins←ptr(r)=null then wait:=true else begin wait:=false; s:=ins←ptr(p); link(last←ins←ptr(r)):=s; s:=last←ins←ptr(r); if best←ins←ptr(r)=p then @<Wrap up the box specified by node |r|, splitting node |p| if called for; set |wait:=true| if node |p| holds a remainder after splitting@> else begin while link(s)<>null do s:=link(s); last←ins←ptr(r):=s; end; end; @<Either append the insertion node |p| after node |q|, and remove it from the current page, or delete |node(p)|@>; end @ @<Wrap up the box specified by node |r|, splitting node |p| if...@>= begin if type(r)=split←up then if (broken←ins(r)=p)and(broken←ptr(r)<>null) then begin while link(s)<>broken←ptr(r) do s:=link(s); split←top←skip:=split←top←ptr(p); ins←ptr(p):=prune←page←top(broken←ptr(r)); if ins←ptr(p)<>null then begin temp←ptr:=vpack(ins←ptr(p),natural); height(p):=height(temp←ptr)+depth(temp←ptr); free←node(temp←ptr,box←node←size); wait:=true; end; link(s):=null; end; best←ins←ptr(r):=null; n:=qo(subtype(r)); temp←ptr:=list←ptr(box(n)); free←node(box(n),box←node←size); box(n):=vpack(temp←ptr,natural); end @ @<Either append the insertion node |p|...@>= link(prev←p):=link(p); link(p):=null; if wait then begin link(q):=p; q:=p; incr(insert←penalties); end else begin delete←glue←ref(split←top←ptr(p)); free←node(p,ins←node←size); end; p:=prev←p @ The list of heldover insertions, running from |link(page←head)| to |page←tail|, must be moved to the contribution list when the user has specified no output routine. @<Perform the default output routine@>= begin if link(page←head)<>null then begin if link(contrib←head)=null then if nest←ptr=0 then tail:=page←tail@+else contrib←tail:=page←tail else link(page←tail):=link(contrib←head); link(contrib←head):=link(page←head); link(page←head):=null; page←tail:=page←head; end; ship←out(box(255)); box(255):=null; end @ @<Explain that too many dead cycles have occurred in a row@>= begin print←err("Output loop---"); print←int(dead←cycles); @.Output loop...@> print(" consecutive dead cycles"); help3("I've concluded that your \output is awry; it never does a")@/ ("\shipout, so I'm shipping \box255 out myself. Next time")@/ ("increase \maxdeadcycles if you want me to be more patient!"); error; end @ @<Fire up the user's output routine and |return|@>= begin output←active:=true; incr(dead←cycles); push←nest; mode:=-vmode; prev←depth:=ignore←depth; mode←line:=-line; begin←token←list(output←routine,output←text); new←save←level(output←group); normal←paragraph; scan←left←brace; return; end @ When the user's output routine finishes, it has constructed a vlist in internal vertical mode, and \TeX\ will do the following: @<Resume the page builder after an output routine has come to an end@>= begin if loc<>null then @<Recover from an unbalanced output routine@>; end←token←list; {conserve stack space in case more outputs are triggered} end←graf; unsave; output←active:=false; insert←penalties:=0;@/ @<Ensure that box 255 is empty after output@>; if tail<>head then {current list goes after heldover insertions} begin link(page←tail):=link(head); page←tail:=tail; end; if link(page←head)<>null then {and both go before heldover contributions} begin if link(contrib←head)=null then contrib←tail:=page←tail; link(page←tail):=link(contrib←head); link(contrib←head):=link(page←head); link(page←head):=null; page←tail:=page←head; end; pop←nest; build←page; end @ @<Recover from an unbalanced output routine@>= begin print←err("Unbalanced output routine"); @.Unbalanced output routine@> help2("Your sneaky output routine has fewer real {'s than }'s.")@/ ("I can't handle that very well; good luck."); error; repeat get←token; until loc=null; end @ @<Ensure that box 255 is empty after output@>= if box(255)<>null then begin print←err("Output routine didn't use all of "); print←esc("box"); print←int(255); @.Output routine didn't use...@> help3("Your \output commands should empty \box255,")@/ ("e.g., by saying `\shipout\box255'.")@/ ("Proceed; I'll discard its present contents."); box←error(255); end @* \[46] The chief executive. We come now to the |main←control| routine, which contains the master switch that causes all the various pieces of \TeX\ to do their things, in the right order. In a sense, this is the grand climax of the program: It applies all of the tools that we have worked so hard to construct. In another sense, this is the messiest part of the program: It necessarily refers to other pieces of code all over the place, so that a person can't fully understand what is going on without paging back and forth to be reminded of conventions that are defined elsewhere. We are now at the hub of the web, the central nervous system that touches most of the other parts and ties them together. @↑brain@> The structure of |main←control| itself is quite simple. There's a label called |big←switch|, at which point the next token of input is fetched using |get←x←token|. Then the program branches at high speed into one of about 100 possible directions, based on the the value of the current mode and the newly fetched command code; the sum |abs(mode)+cur←cmd| indicates what to do next. For example, the case `|vmode+letter|' arises when a letter occurs in vertical mode (or internal vertical mode); this case leads to instructions that initialize a new paragraph and enter horizontal mode. The big |case| statement that contains this multiway switch has been labeled |reswitch|, so that the program can |goto reswitch| when the next token has already been fetched. Most of the cases are quite short; they call an ``action procedure'' that does the work for that case, and then they either |goto reswitch| or they ``fall through'' to the end of the |case| statement, which returns control back to |big←switch|. Thus, |main←control| is not an extremely large procedure, in spite of the multiplicity of things it must do; it is small enough to be handled by \PASCAL\ compilers that put severe restrictions on procedure size. @!@↑action procedure@> One case is singled out for special treatment, because it accounts for most of \TeX's activities in typical applications. The process of reading simple text and converting it into |char←node| records, while looking for ligatures and kerns, is part of \TeX's ``inner loop''; the whole program runs efficiently when its inner loop is fast, so this part has been written with particular care. @ We shall concentrate first on the inner loop of |main←control|, deferring consideration of the other cases until later. @d big←switch=60 {go here to branch on the next token of input} @d main←loop=70 {go here to typeset |cur←chr| in the current font} @d main←loop←1=71 {like |main←loop|, but |(f,c)| = current font and char} @d main←loop←2=72 {like |main←loop←1|, but |c| is known to be in range} @d main←loop←3=73 {like |main←loop←2|, but several variables are set up} @d append←normal←space=74 {go here to append a normal space between words} @p @t\4@>@<Declare action procedures for use by |main←control|@>@; @t\4@>@<Declare the procedure called |handle←right←brace|@>@; procedure main←control; {governs \TeX's activities} label big←switch,reswitch,main←loop,main←loop←1,main←loop←2,main←loop←3, append←normal←space,exit; var t:integer; {general-purpose temporary variable} @<Local variables for the inner loop of |main←control|@>@; begin if every←job<>null then begin←token←list(every←job,every←job←text); big←switch: get←x←token;@/ reswitch: @<Give diagnostic information, if requested@>; case abs(mode)+cur←cmd of hmode+letter,hmode+other←char,hmode+char←given: goto main←loop; hmode+char←num: begin scan←char←num; cur←chr:=cur←val; goto main←loop; end; hmode+spacer: if space←factor=1000 then goto append←normal←space else app←space; hmode+ex←space,mmode+ex←space: goto append←normal←space; @t\4@>@<Cases of |main←control| that are not part of the inner loop@>@; end; {of the big |case| statement} goto big←switch; main←loop:@<Append character |cur←chr| and the following characters (if~any) to the current hlist in the current font; |goto reswitch| when a non-character has been fetched@>; append←normal←space:@<Append a normal inter-word space to the current list, then |goto big←switch|@>; exit:end; @ When a new token has just been fetched at |big←switch|, we have an ideal place to monitor \TeX's activity. @↑debugging@> @<Give diagnostic information, if requested@>= if interrupt<>0 then if OK←to←interrupt then begin back←input; check←interrupt; goto big←switch; end; @!debug if panicking then check←mem(false);@+@;@+gubed if tracing←commands>0 then show←cur←cmd←chr @ In the following program, |l| is the current character or ligature; it might grow into a longer ligature. One or more characters has been used to define |l|, and the last of these was |c|. The chief use of |c| will be to modify |space←factor| and to insert discretionary nodes after explicit hyphens in the text. @<Local variables for the inner loop of |main←control|@>= @!l:quarterword; {the current character or ligature} @!c:eight←bits; {the most recent character} @!f:internal←font←number; {the current font} @!r:halfword; {the next character for ligature/kern matching} @!p:pointer; {the current |char←node|} @!k:0..font←mem←size; {index into |font←info|} @!q:pointer; {where a ligature should be detached} @!i:four←quarters; {character information bytes for |l|} @!j:four←quarters; {ligature/kern command} @!s:integer; {space factor code} @!ligature←present:boolean; {should a ligature node be made?} @ @<Append character |cur←chr| and the following characters...@>= f:=cur←font; c:=cur←chr; main←loop←1: if (c<font←bc[f])or(c>font←ec[f]) then begin char←warning(f,c); goto big←switch; end; main←loop←2: q:=tail; ligature←present:=false; l:=qi(c); main←loop←3: @<Adjust \(t)the space factor, based on its current value and |c|@>; @<Append character |l| and the following characters (if any) to the current hlist, in font |f|; if |ligature←present|, detach a ligature node starting at |link(q)|; if |c| is a hyphen, append a null |disc←node|; finally |goto reswitch|@> @ We leave |space←factor| unchanged if |sf←code(c)=0|; otherwise we set it to |sf←code(c)|, except that the space factor never changes from a value less than 1000 to a value exceeding 1000. If |c>=128|, its |sf←code| is implicitly 1000. The most common case is |sf←code(c)=1000|, so we want that case to be fast. @<Adjust \(t)the space factor...@>= if c<128 then begin s:=sf←code(c); if s=1000 then space←factor:=1000 else if s<1000 then begin if s>0 then space←factor:=s; end else if space←factor<1000 then space←factor:=1000 else space←factor:=s; end else space←factor:=1000 @ Now we come to the inner loop, in which the characters of a word are gathered at (hopefully) high speed. @↑inner loop@> @<Append character |l| and the following...@>= i:=char←info(f)(l); if char←exists(i) then begin fast←get←avail(p); font(p):=f; character(p):=qi(c); link(tail):=p; tail:=p; end else char←warning(f,qo(l)); @<Look ahead for ligature or kerning, either continuing the main loop or going to |reswitch|@> @ The result of \.{\\char} can participate in a ligature or kern, so we must look ahead for it. @↑inner loop@> @<Look ahead for ligature...@>= get←next; {set only |cur←cmd| and |cur←chr|} if cur←cmd=letter then r:=qi(cur←chr) else if cur←cmd=other←char then r:=qi(cur←chr) else if cur←cmd=char←given then r:=qi(cur←chr) else begin x←token; {set |cur←cmd|, |cur←chr|, |cur←tok|} if (cur←cmd=letter)or(cur←cmd=other←char)or(cur←cmd=char←given) then r:=qi(cur←chr) else if cur←cmd=char←num then begin scan←char←num; r:=qi(cur←val); end else r:=qi(256); {this flag means that no character follows} end; if char←tag(i)=lig←tag then if r<>qi(256) then @<Follow the lig/kern program; |goto main←loop←3| if scoring a hit@>; @<Make a ligature node, if |ligature←present|; insert a discretionary node for an explicit hyphen, if |c| is the current |hyphen←char|@>; if r=qi(256) then goto reswitch; {|cur←cmd|, |cur←chr|, |cur←tok| are untouched} c:=qo(r); goto main←loop←1 {|f| is still valid} @ Even though comparatively few characters have a lig/kern program, the |repeat| construction here counts as part of \TeX's inner loop, since it involves a potentially long sequential search. For example, tests with one commonly used font showed that about 40 per cent of all characters had a lig/kern program, and the |repeat| loop was performed about four times for every such character. @↑inner loop@> @<Follow the lig/kern...@>= begin k:=lig←kern←start(f)(i); repeat j:=font←info[k].qqqq; {fetch a lig/kern command} if next←char(j)=r then if op←bit(j)<kern←flag then @<Extend a ligature, |goto main←loop←3|@> else @<Append a kern, |goto main←loop←2|@>; incr(k); until stop←bit(j)>=stop←flag; end @ @<Append a kern,...@>= begin @<Make a ligature node,...@>; tail←append(new←kern(char←kern(f)(j))); c:=qo(r); goto main←loop←2; end @ A discretionary break is not inserted for an explicit hyphen when we are in restricted horizontal mode. In particular, this avoids putting discretionary nodes inside of other discretionaries. @↑explicit hyphens@> @<Make a ligature node,...@>= if ligature←present then begin p:=new←ligature(f,l,link(q)); link(q):=p; tail:=p; end; if c=hyphen←char[f] then if mode=hmode then tail←append(new←disc) @ @<Extend a ligature...@>= begin ligature←present:=true; l:=rem←byte(j); c:=qo(r); goto main←loop←3; end @ The occurrence of blank spaces is almost part of \TeX's inner loop, since we usually encounter about one space for every five non-blank characters. Therefore |main←control| gives second-highest priority to ordinary spaces. When a glue parameter like \.{\\spaceskip} is set to `\.{0pt}', we will see to it later that the corresponding glue specification is precisely |zero←glue|, not merely a pointer to some other specification that happens to be full of zeroes. Therefore it is simple to test whether a glue parameter is zero or not. @<Append a normal inter-word space...@>= if space←skip=zero←glue then begin @<Find the glue specification, |p|, for text spaces in the current font@>; q:=new←glue(p); end else q:=new←param←glue(space←skip←code); link(tail):=q; tail:=q; goto big←switch @ Having |font←glue| allocated for each text font saves both time and memory. If any of the three spacing parameters are subsequently changed by the use of \.{\\fontdimen}, the |find←font←dimen| procedure deallocates the |font←glue| specification allocated here. @<Find the glue specification...@>= begin p:=font←glue[cur←font]; if p=null then begin f:=cur←font; p:=new←spec(zero←glue); k:=param←base[f]+space←code; width(p):=font←info[k].sc; stretch(p):=font←info[k+1].sc; shrink(p):=font←info[k+2].sc; font←glue[f]:=p; end; end @ @<Declare act...@>= procedure app←space; {handle spaces when |space←factor<>1000|} var p:pointer; {glue specification} @!q:pointer; {glue node} @!f:internal←font←number; {the current font} @!k:0..font←mem←size; {index into |font←info|} begin if (space←factor>=2000)and(xspace←skip<>zero←glue) then q:=new←param←glue(xspace←skip←code) else begin if space←skip<>zero←glue then p:=space←skip else @<Find the glue specification...@>; p:=new←spec(p); @<Modify the glue specification in |p| according to the space factor@>; q:=new←glue(p); glue←ref←count(p):=null; end; link(tail):=q; tail:=q; end; @ @<Modify the glue specification in |p| according to the space factor@>= if space←factor>=2000 then width(p):=width(p)+extra←space(cur←font); stretch(p):=xn←over←d(stretch(p),space←factor,1000); shrink(p):=xn←over←d(shrink(p),1000,space←factor) @ Whew---that covers the main loop. We can now proceed at a leisurely pace through the other combinations of possibilities. @d any←mode(#)==vmode+#,hmode+#,mmode+# {for mode-independent commands} @<Cases of |main←control| that are not part of the inner loop@>= any←mode(relax),vmode+spacer,mmode+spacer:do←nothing; any←mode(ignore←spaces): begin @<Get the next non-blank non-call...@>; goto reswitch; end; vmode+stop: if its←all←over then return; {this is the only way out} @t\4@>@<Forbidden cases detected in |main←control|@>@+@,any←mode(mac←param): report←illegal←case; @<Math-only cases in non-math modes, or vice versa@>: insert←dollar←sign; @t\4@>@<Cases of |main←control| that build boxes and lists@>@; @t\4@>@<Cases of |main←control| that don't depend on |mode|@>@; @t\4@>@<Cases of |main←control| that are for extensions to \TeX@>@; @ Here is a list of cases where the user has probably gotten into or out of math mode by mistake. \TeX\ will insert a dollar sign and rescan the current token. @d non←math(#)==vmode+#,hmode+# @<Math-only cases in non-math modes...@>= non←math(sup←mark), non←math(sub←mark), non←math(math←char←num), non←math(math←given), non←math(math←comp), non←math(delim←num), non←math(left←right), non←math(above), non←math(radical), non←math(math←style), non←math(math←choice), non←math(vcenter), non←math(non←script), non←math(mkern), non←math(limit←switch), non←math(mskip), non←math(math←accent), mmode+endv, mmode+par←end, mmode+stop, mmode+vskip, mmode+un←vbox, mmode+valign, mmode+hrule @ @<Declare action...@>= procedure insert←dollar←sign; begin back←input; cur←tok:=math←shift←token+"$"; print←err("Missing $ inserted"); @.Missing \$ inserted@> help2("I've inserted a begin-math/end-math symbol since I think")@/ ("you left one out. Proceed, with fingers crossed."); ins←error; end; @ When erroneous situations arise, \TeX\ usually issues an error message specific to the particular error. For example, `\.{\\noalign}' should not appear in any mode, since it is recognized by the |align←peek| routine in all of its legitimate appearances; a special error message is given when `\.{\\noalign}' occurs elsewhere. But sometimes the most appropriate error message is simply that the user is not allowed to do what he or she has attempted. For example, `\.{\\moveleft}' is allowed only in vertical mode, and `\.{\\lower}' only in non-vertical modes. Such cases are enumerated here and in the other sections referred to under `See also \dots.' @<Forbidden cases...@>= vmode+vmove,hmode+hmove,mmode+hmove,any←mode(last←item), @ The `|you←cant|' procedure prints a line saying that the current command is illegal in the current mode; it identifies these things symbolically. @<Declare action...@>= procedure you←cant; begin print←err("You can't use `"); @.You can't use x in y mode@> print←cmd←chr(cur←cmd,cur←chr); print("' in "); print←mode(mode); end; @ @<Declare act...@>= procedure report←illegal←case; begin you←cant; help4("Sorry, but I'm not programmed to handle this case;")@/ ("I'll just pretend that you didn't ask for it.")@/ ("If you're in the wrong mode, you might be able to")@/ ("return to the right one by typing `I}' or `I$' or `I\par'.");@/ error; end; @ Some operations are allowed only in privileged modes, i.e., in cases that |mode>0|. The |privileged| function is used to detect violations of this rule; it issues an error message and returns |false| if the current |mode| is negative. @<Declare act...@>= function privileged:boolean; begin if mode>0 then privileged:=true else begin report←illegal←case; privileged:=false; end; end; @ Either \.{\\dump} or \.{\\end} will cause |main←control| to enter the endgame, since both of them have `|stop|' as their command code. @<Put each...@>= primitive("end",stop,0);@/ @!@:end←}{\.{\\end} primitive@> primitive("dump",stop,1);@/ @!@:dump←}{\.{\\dump} primitive@> @ @<Cases of |print←cmd←chr|...@>= stop:if chr←code=1 then print←esc("dump")@+else print←esc("end"); @ We don't want to leave |main←control| immediately when a |stop| command is sensed, because it may be necessary to invoke an \.{\\output} routine several times before things really grind to a halt. (The output routine might even say `\.{\\gdef\\end\{...\}}', to prolong the life of the job.) Therefore |its←all←over| is |true| only when the current page and contribution list are empty, and when the last output was not a ``dead cycle.'' @<Declare act...@>= function its←all←over:boolean; {do this when \.{\\end} or \.{\\dump} occurs} label exit; begin if privileged then begin if (page←head=page←tail)and(head=tail)and(dead←cycles=0) then begin its←all←over:=true; return; end; back←input; {we will try to end again after ejecting residual material} tail←append(new←null←box); width(tail):=hsize; tail←append(new←glue(fill←glue)); tail←append(new←penalty(-@'10000000000));@/ build←page; {append \.{\\hbox to \\hsize\{\}\\vss\\penalty-'10000000000}} end; its←all←over:=false; exit:end; @* \[47] Building boxes and lists. The most important parts of |main←control| are concerned with \TeX's chief mission of box-making. We need to control the activities that put entries on vlists and hlists, as well as the activities that convert those lists into boxes. All of the necessary machinery has already been developed; it remains for us to ``push the buttons'' at the right times. @ As an introduction to these routines, let's consider one of the simplest cases: What happens when `\.{\\hrule}' occurs in vertical mode, or `\.{\\vrule}' in horizontal mode or math mode? The code in |main←control| is short, since the |scan←rule←spec| routine already does most of what is required; thus, there is no need for a special action procedure. Note that baselineskip calculations are disabled after a rule in vertical mode, by setting |prev←depth:=ignore←depth|. @<Cases of |main←control| that build...@>= vmode+hrule,hmode+vrule,mmode+vrule: begin tail←append(scan←rule←spec); if abs(mode)=vmode then prev←depth:=ignore←depth else if abs(mode)=hmode then space←factor:=1000; end; @ The processing of things like \.{\\hskip} and \.{\\vskip} is slightly more complicated. But the code in |main←control| is very short, since it simply calls on the action routine |append←glue|. Similarly, \.{\\kern} activates |append←kern|. @<Cases of |main←control| that build...@>= vmode+vskip,hmode+hskip,mmode+hskip,mmode+mskip: append←glue; any←mode(kern),mmode+mkern: append←kern; @ The |hskip| and |vskip| command codes are used for control sequences like \.{\\hss} and \.{\\vfil} as well as for \.{\\hskip} and \.{\\vskip}. The difference is in the value of |cur←chr|. @d fil←code=0 {identifies \.{\\hfil} and \.{\\vfil}} @d fill←code=1 {identifies \.{\\hfill} and \.{\\vfill}} @d ss←code=2 {identifies \.{\\hss} and \.{\\vss}} @d fil←neg←code=3 {identifies \.{\\hfilneg} and \.{\\vfilneg}} @d skip←code=4 {identifies \.{\\hskip} and \.{\\vskip}} @d mskip←code=5 {identifies \.{\\mskip}} @<Put each...@>= primitive("hskip",hskip,skip←code);@/ @!@:hskip←}{\.{\\hskip} primitive@> primitive("hfil",hskip,fil←code); @!@:hfil←}{\.{\\hfil} primitive@> primitive("hfill",hskip,fill←code);@/ @!@:hfill←}{\.{\\hfill} primitive@> primitive("hss",hskip,ss←code); @!@:hss←}{\.{\\hss} primitive@> primitive("hfilneg",hskip,fil←neg←code);@/ @!@:hfil←neg←}{\.{\\hfilneg} primitive@> primitive("vskip",vskip,skip←code);@/ @!@:vskip←}{\.{\\vskip} primitive@> primitive("vfil",vskip,fil←code); @!@:vfil←}{\.{\\vfil} primitive@> primitive("vfill",vskip,fill←code);@/ @!@:vfill←}{\.{\\vfill} primitive@> primitive("vss",vskip,ss←code); @!@:vss←}{\.{\\vss} primitive@> primitive("vfilneg",vskip,fil←neg←code);@/ @!@:vfil←neg←}{\.{\\vfilneg} primitive@> primitive("mskip",mskip,mskip←code);@/ @!@:mskip←}{\.{\\mskip} primitive@> primitive("kern",kern,explicit); @!@:kern←}{\.{\\kern} primitive@> primitive("mkern",mkern,mu←glue);@/ @!@:mkern←}{\.{\\mkern} primitive@> @ @<Cases of |print←cmd←chr|...@>= hskip: case chr←code of skip←code:print←esc("hskip"); fil←code:print←esc("hfil"); fill←code:print←esc("hfill"); ss←code:print←esc("hss"); othercases print←esc("hfilneg") endcases; vskip: case chr←code of skip←code:print←esc("vskip"); fil←code:print←esc("vfil"); fill←code:print←esc("vfill"); ss←code:print←esc("vss"); othercases print←esc("vfilneg") endcases; mskip: print←esc("mskip"); kern: print←esc("kern"); mkern: print←esc("mkern"); @ All the work relating to glue creation has been relegated to the following subroutine. It does not call |build←page|, because it is used in at least one place where that would be a mistake. @<Declare action...@>= procedure append←glue; var s:small←number; {modifier of skip command} begin s:=cur←chr; case s of fil←code: cur←val:=fil←glue; fill←code: cur←val:=fill←glue; ss←code: cur←val:=ss←glue; fil←neg←code: cur←val:=fil←neg←glue; skip←code: scan←glue(glue←val); mskip←code: scan←glue(mu←val); end; {now |cur←val| points to the glue specification} tail←append(new←glue(cur←val)); if s>=skip←code then begin decr(glue←ref←count(cur←val)); if s>skip←code then subtype(tail):=mu←glue; end; end; @ @<Declare act...@>= procedure append←kern; var s:quarterword; {|subtype| of the kern node} begin s:=cur←chr; scan←dimen(s=mu←glue,false,false); tail←append(new←kern(cur←val)); subtype(tail):=s; end; @ Many of the actions related to box-making are triggered by the appearance of braces in the input. For example, when the user says `\.{\\hbox} \.{to} \.{100pt\{$\langle\,\hbox{hlist}\,\rangle$\}}' in vertical mode, the information about the box size (100pt, |exactly|) is put onto |save←stack| with a level boundary word just above it, and |cur←group:=adjusted←hbox←group|; \TeX\ enters restricted horizontal mode to process the hlist. The right brace eventually causes |save←stack| to be restored to its former state, at which time the information about the box size (100pt, |exactly|) is available once again; a box is packaged and we leave restricted horizontal mode, appending the new box to the current list of the enclosing mode (in this case to the current list of vertical mode), followed by any vertical adjustments that were removed from the box by |hpack|. The next few sections of the program are therefore concerned with the treatment of left and right curly braces. @ If a left brace occurs in the middle of a page or paragraph, it simply introduces a new level of grouping, and the matching right brace will not have such a drastic effect. Such grouping affects neither the mode nor the current list. @<Cases of |main←control| that build...@>= non←math(left←brace): new←save←level(simple←group); any←mode(begin←group): new←save←level(semi←simple←group); any←mode(end←group): if cur←group=semi←simple←group then unsave else off←save; @ We have to deal with errors in which braces and such things are not properly nested. Sometimes the user makes an error of commission by inserting an extra symbol, but sometimes the user makes an error of omission. \TeX\ can't always tell one from the other, so it makes a guess and tries to avoid getting into a loop. The |off←save| routine is called when the current group code is wrong. It tries to insert something into the user's input that will help clean off the top level. @<Declare act...@>= procedure off←save; var p:pointer; {inserted token} begin if cur←group=bottom←level then @<Drop current token and complain that it was unmatched@> else begin back←input; p:=get←avail; link(temp←head):=p; print←err("Missing "); @<Prepare to insert a token that matches |cur←group|, and print what it is@>; print(" inserted"); ins←list(link(temp←head)); help5("I've inserted something that you may have forgotten.")@/ ("(See the <inserted text> above.)")@/ ("With luck, this will get me unwedged. But if you")@/ ("really didn't forget anything, try typing `2' now; then")@/ ("my insertion and my current dilemma will both disappear."); error; end; end; @ At this point, |link(temp←head)=p|, a pointer to an empty one-word node. @<Prepare to insert a token that matches |cur←group|...@>= case cur←group of semi←simple←group: begin info(p):=cs←token←flag+frozen←end←group; print←esc("endgroup"); @.Missing \\endgroup inserted@> end; math←shift←group: begin info(p):=math←shift←token+"$"; print←char("$"); @.Missing \$ inserted@> end; math←left←group: begin info(p):=cs←token←flag+frozen←right; link(p):=get←avail; p:=link(p); info(p):=other←token+"."; print←esc("right."); @.Missing \\right\hbox{.} inserted@> @↑null delimiter@> end; othercases begin info(p):=right←brace←token+"}"; print←char("}"); @.Missing \} inserted@> end endcases @ It's very hard to get this error message; indeed, the case didn't arise until more than two years after it had been programmed. @<Drop current token and complain that it was unmatched@>= begin print←err("Extra "); print←cmd←chr(cur←cmd,cur←chr); @.Extra x@> help1("Things are pretty mixed up, but I think the worst is over.");@/ error; end @ The routine for a |right←brace| character branches into many subcases, since a variety of things may happen, depending on |cur←group|. Some types of groups are not supposed to be ended by a right brace; error messages are given in hopes of pinpointing the problem. Most branches of this routine will be filled in later, when we are ready to understand them; meanwhile, we must prepare ourselves to deal with such errors. @<Cases of |main←control| that build...@>= any←mode(right←brace): handle←right←brace; @ @<Declare the procedure called |handle←right←brace|@>= procedure handle←right←brace; var p,@!q:pointer; {for short-term use} @!d:scaled; {holds |split←max←depth| in |insert←group|} @!f:integer; {holds |floating←penalty| in |insert←group|} begin case cur←group of simple←group: unsave; bottom←level: begin print←err("Too many }'s"); @.Too many \}'s@> help2("You've closed more groups than you opened.")@/ ("Such booboos are generally harmless, so keep going."); error; end; semi←simple←group,math←shift←group,math←left←group: extra←right←brace; @t\4@>@<Cases of |handle←right←brace| where a |right←brace| triggers a delayed action@>@; othercases confusion("rightbrace") @:this can't happen rightbrace}{\quad rightbrace@> endcases; end; @ @<Declare act...@>= procedure extra←right←brace; begin print←err("Extra }, or forgotten "); @.Extra \}, or forgotten x@> case cur←group of semi←simple←group: print←esc("endgroup"); math←shift←group: print←char("$"); math←left←group: print←esc("right"); end;@/ help5("I've deleted a group-closing symbol because it seems to be")@/ ("spurious, as in `$x}$'. But perhaps the } is legitimate and")@/ ("you forgot something else, as in `\hbox{$x}'. In such cases")@/ ("the way to recover is to insert both the forgotten and the")@/ ("deleted material, e.g., by typing `I$}'."); error; incr(align←state); end; @ Here is where we clear the parameters that are supposed to revert to their default values after every paragraph and when internal vertical mode is entered. @<Declare act...@>= procedure normal←paragraph; begin if looseness<>0 then eq←word←define(int←base+looseness←code,0); if hang←indent<>0 then eq←word←define(dimen←base+hang←indent←code,0); if hang←after<>1 then eq←word←define(int←base+hang←after←code,1); if par←shape←ptr<>null then eq←define(par←shape←loc,shape←ref,null); end; @ Now let's turn to the question of how \.{\\hbox} is treated. We actually need to consider also a slightly larger context, since constructions like `\.{\\setbox3=}\penalty0\.{\\hbox...}' and `\.{\\lower3.8pt\\hbox...}' and `\.{\\leaders}\penalty0\.{\\hbox...}' are supposed to invoke quite different actions after the box has been packaged. Conversely, constructions like `\.{\\setbox3=}' can be followed by a variety of different kinds of boxes, and we would like to encode such things in an efficient way. In other words, there are two problems: To represent the context of a box, and to represent its type. The first problem is solved by putting a ``context code'' on the |save←stack|, just below the two entries that give the dimensions produced by |scan←spec|. The context code is either a (signed) shift amount, or it is a large integer |>=box←flag|, where |box←flag=@t$2↑{30}$@>|. Codes |box←flag| through |box←flag+255| represent `\.{\\setbox0}' through `\.{\\setbox255}'; codes |box←flag+256| through |box←flag+511| represent `\.{\\global\\setbox0}' through `\.{\\global\\setbox255}'; code |box←flag+512| represents `\.{\\shipout}'; and codes |box←flag+513| through |box←flag+515| represent `\.{\\leaders}', `\.{\\cleaders}', and `\.{\\xleaders}'. The second problem is solved by giving the command code |make←box| to all control sequences that produce a box, and by using the following |chr←code| values to distinguish between them: |box←code|, |copy←code|, |last←box←code|, |vsplit←code|, |vtop←code|, |vtop←code+vmode|, and |vtop←code+hmode|, where the latter two are used denote \.{\\vbox} and \.{\\hbox}, respectively. @d box←flag==@'10000000000 {context code for `\.{\\setbox0}'} @d ship←out←flag==box←flag+512 {context code for `\.{\\shipout}'} @d leader←flag==box←flag+513 {context code for `\.{\\leaders}'} @d box←code=0 {|chr←code| for `\.{\\box}'} @d copy←code=1 {|chr←code| for `\.{\\copy}'} @d last←box←code=2 {|chr←code| for `\.{\\lastbox}'} @d vsplit←code=3 {|chr←code| for `\.{\\vsplit}'} @d vtop←code=4 {|chr←code| for `\.{\\vtop}'} @<Put each...@>= primitive("moveleft",hmove,1); @!@:move←left←}{\.{\\moveleft} primitive@> primitive("moveright",hmove,0);@/ @!@:move←right←}{\.{\\moveright} primitive@> primitive("raise",vmove,1); @!@:raise←}{\.{\\raise} primitive@> primitive("lower",vmove,0); @!@:lower←}{\.{\\lower} primitive@> @# primitive("box",make←box,box←code); @!@:box←}{\.{\\box} primitive@> primitive("copy",make←box,copy←code); @!@:copy←}{\.{\\copy} primitive@> primitive("lastbox",make←box,last←box←code); @!@:last←box←}{\.{\\lastbox} primitive@> primitive("vsplit",make←box,vsplit←code); @!@:vsplit←}{\.{\\vsplit} primitive@> primitive("vtop",make←box,vtop←code);@/ @!@:vtop←}{\.{\\vtop} primitive@> primitive("vbox",make←box,vtop←code+vmode); @!@:vbox←}{\.{\\vbox} primitive@> primitive("hbox",make←box,vtop←code+hmode);@/ @!@:hbox←}{\.{\\hbox} primitive@> primitive("shipout",leader←ship,a←leaders-1); {|ship←out←flag=leader←flag-1|} @!@:ship←out←}{\.{\\shipout} primitive@> primitive("leaders",leader←ship,a←leaders); @!@:leaders←}{\.{\\leaders} primitive@> primitive("cleaders",leader←ship,c←leaders); @!@:c←leaders←}{\.{\\cleaders} primitive@> primitive("xleaders",leader←ship,x←leaders); @!@:x←leaders←}{\.{\\xleaders} primitive@> @ @<Cases of |print←cmd←chr|...@>= hmove: if chr←code=1 then print←esc("moveleft")@+else print←esc("moveright"); vmove: if chr←code=1 then print←esc("raise")@+else print←esc("lower"); make←box: case chr←code of box←code: print←esc("box"); copy←code: print←esc("copy"); last←box←code: print←esc("lastbox"); vsplit←code: print←esc("vsplit"); vtop←code: print←esc("vtop"); vtop←code+vmode: print←esc("vbox"); othercases print←esc("hbox") endcases; leader←ship: if chr←code=a←leaders then print←esc("leaders") else if chr←code=c←leaders then print←esc("cleaders") else if chr←code=x←leaders then print←esc("xleaders") else print←esc("shipout"); @ Constructions that require a box are started by placing a context code on |save←stack| and calling |scan←box|. The |scan←box| routine verifies that a |make←box| command comes next and then it calls |begin←box|. @<Cases of |main←control| that build...@>= vmode+hmove,hmode+vmove,mmode+vmove: begin t:=cur←chr; scan←normal←dimen; if t=0 then saved(0):=cur←val@+else saved(0):=-cur←val; scan←box; end; any←mode(leader←ship): begin saved(0):=leader←flag-a←leaders+cur←chr; scan←box; end; any←mode(make←box): begin saved(0):=0; begin←box; end; @ The global variable |cur←box| will point to a newly-made box. If the box is void, we will have |cur←box=null|. Otherwise we will have |type(cur←box)=hlist←node| or |vlist←node| or |rule←node|; the |rule←node| case can occur only with leaders. @<Glob...@>= @!cur←box:pointer; {box to be placed into its context} @ The |box←end| procedure does the right thing with |cur←box|, if |saved(0)| represents the context as explained above. @<Declare act...@>= procedure box←end; var p:pointer; {|ord←noad| for new box in math mode} begin if saved(0)<box←flag then @<Append box |cur←box| to the current list, shifted by |saved(0)|@> else if saved(0)<ship←out←flag then @<Store \(c)|cur←box| in a box register@> else if cur←box<>null then if saved(0)>ship←out←flag then @<Append a new leader node that uses |cur←box|@> else ship←out(cur←box); end; @ The global variable |adjust←tail| will be non-null if and only if the current box might include adjustments that should be appended to the current vertical list. @<Append box |cur←box| to the current...@>= begin if cur←box<>null then begin shift←amount(cur←box):=saved(0); if abs(mode)=vmode then begin append←to←vlist(cur←box); if adjust←tail<>null then begin if adjust←head<>adjust←tail then begin link(tail):=link(adjust←head); tail:=adjust←tail; end; adjust←tail:=null; end; if mode>0 then build←page; end else begin if abs(mode)=hmode then space←factor:=1000 else begin p:=new←noad; math←type(nucleus(p)):=sub←box; info(nucleus(p)):=cur←box; cur←box:=p; end; link(tail):=cur←box; tail:=cur←box; end; end; end @ @<Store \(c)|cur←box| in a box register@>= if saved(0)<box←flag+256 then eq←define(box←base-box←flag+saved(0),box←ref,cur←box) else geq←define(box←base-box←flag-256+saved(0),box←ref,cur←box) @ @<Append a new leader node ...@>= begin @<Get the next non-blank non-relax...@>; if ((cur←cmd=hskip)and(abs(mode)<>vmode))or@| ((cur←cmd=vskip)and(abs(mode)=vmode))or@| ((cur←cmd=mskip)and(abs(mode)=mmode)) then begin append←glue; subtype(tail):=saved(0)-(leader←flag-a←leaders); leader←ptr(tail):=cur←box; end else begin print←err("Leaders not followed by proper glue"); @.Leaders not followed by...@> help3("You should say `\leaders <box or rule><hskip or vskip>'.")@/ ("I found the <box or rule>, but there's no suitable")@/ ("<hskip or vskip>, so I'm ignoring these leaders."); back←error; flush←node←list(cur←box); end; end @ Now that we can see what eventually happens to boxes, we can consider the first steps in their creation. The |begin←box| routine is called when |saved(0)| is a context specification, |cur←chr| specifies the type of box desired, and |cur←cmd=make←box|. @<Declare act...@>= procedure begin←box; label exit; var p:pointer; {runs through the current list} @!k:halfword; {0 or |vmode| or |hmode|} @!n:eight←bits; {a box number} begin case cur←chr of box←code: begin scan←eight←bit←int; cur←box:=box(cur←val); box(cur←val):=null; {the box becomes void, at the same level} end; copy←code: begin scan←eight←bit←int; cur←box:=copy←node←list(box(cur←val)); end; last←box←code: @<If the current list ends with a box node, delete it from the list and make |cur←box| point to it; otherwise set |cur←box:=null|@>; vsplit←code: @<Split off part of a vertical box, make |cur←box| point to it@>; othercases @<Initiate the construction of an hbox or vbox, then |return|@> endcases;@/ box←end; {in simple cases, we use the box immediately} exit:end; @ Note that the condition |not is←char←node(tail)| implies that |head<>tail|, since |head| is a one-word node. @<If the current list ends with a box node, delete it...@>= begin cur←box:=null; if abs(mode)=mmode then begin you←cant; help1("Sorry; this \lastbox will be void."); error; end else if (mode=vmode)and(head=tail) then @<Apologize for inability to do \.{\\lastbox} now@> else begin if not is←char←node(tail) then if (type(tail)=hlist←node)or(type(tail)=vlist←node) then begin p:=head; cur←box:=tail; shift←amount(cur←box):=0; while link(p)<>tail do p:=link(p); tail:=p; link(p):=null; end; end; end @ @<Apologize for inability to do \.{\\lastbox} now@>= begin you←cant; help2("Sorry...I'm usually unable to take things from the current")@/ ("page. This \lastbox will therefore be void."); error; end @ Here we deal with things like `\.{\\vsplit 13 to 100pt}'. @<Split off part of a vertical box, make |cur←box| point to it@>= begin scan←eight←bit←int; n:=cur←val; if not scan←keyword("to") then @.to@> begin print←err("Missing `to' inserted"); @.Missing `to' inserted@> help2("I'm working on `\vsplit<box number> to <dimen>';")@/ ("will look for the <dimen> next."); error; end; scan←normal←dimen; cur←box:=vsplit(n,cur←val); end @ Here is where we enter restricted horizontal mode or internal vertical mode, in order to make a box. @<Initiate the construction of an hbox or vbox, then |return|@>= begin k:=cur←chr-vtop←code; incr(save←ptr); scan←spec; if k=hmode then if (saved(-3)<box←flag)and(abs(mode)=vmode) then new←save←level(adjusted←hbox←group) else new←save←level(hbox←group) else begin if k=vmode then new←save←level(vbox←group) else begin new←save←level(vtop←group); k:=vmode; end; normal←paragraph; end; push←nest; mode:=-k; if k=vmode then begin prev←depth:=ignore←depth; if every←vbox<>null then begin←token←list(every←vbox,every←vbox←text); end else begin space←factor:=1000; if every←hbox<>null then begin←token←list(every←hbox,every←hbox←text); end; return; end @ @<Declare act...@>= procedure scan←box; {the next input should specify a box or perhaps a rule} begin @<Get the next non-blank non-relax...@>; if cur←cmd=make←box then begin←box else if (saved(0)>=leader←flag)and((cur←cmd=hrule)or(cur←cmd=vrule)) then begin cur←box:=scan←rule←spec; box←end; end else begin@t@>@;@/ print←err("A <box> was supposed to be here");@/ @.A <box> was supposed to...@> help3("I was expecting to see \hbox or \vbox or \copy or \box or")@/ ("something like that. So you might find something missing in")@/ ("your output. But keep trying; you can fix this later."); back←error; end; end; @ When the right brace occurs at the end of an \.{\\hbox} or \.{\\vbox} or \.{\\vtop} construction, the |package| routine comes into action. We might also have to finish a paragraph that hasn't ended. @<Cases of |handle...@>= hbox←group: package(0); adjusted←hbox←group: begin adjust←tail:=adjust←head; package(0); end; vbox←group: begin end←graf; package(0); end; vtop←group: begin end←graf; package(vtop←code); end; @ @<Declare action...@>= procedure package(@!c:small←number); var h:scaled; {height of box} @!p:pointer; {first node in a box} @!d:scaled; {max depth} begin d:=box←max←depth; unsave; save←ptr:=save←ptr-3; if mode=-hmode then cur←box:=hpack(link(head),saved(2),saved(1)) else begin cur←box:=vpackage(link(head),saved(2),saved(1),d); if c=vtop←code then @<Readjust the height and depth of |cur←box|, for \.{\\vtop}@>; end; pop←nest; box←end; end; @ The height of a `\.{\\vtop}' box is inherited from the first item on its list, if that item is an |hlist←node|, |vlist←node|, or |rule←node|; otherwise the \.{\\vtop} height is zero. @<Readjust the height...@>= begin h:=0; p:=list←ptr(cur←box); if p<>null then if type(p)<=rule←node then h:=height(p); depth(cur←box):=depth(cur←box)-h+height(cur←box); height(cur←box):=h; end @ A paragraph begins when horizontal-mode material occurs in vertical mode, or when the paragraph is explicitly started by `\.{\\indent}' or `\.{\\noindent}'. @<Put each...@>= primitive("indent",start←par,1); @!@:indent←}{\.{\\indent} primitive@> primitive("noindent",start←par,0); @!@:no←indent←}{\.{\\noindent} primitive@> @ @<Cases of |print←cmd←chr|...@>= start←par: if chr←code=0 then print←esc("noindent")@+ else print←esc("indent"); @ @<Cases of |main←control| that build...@>= vmode+start←par: new←graf(cur←chr>0); vmode+letter,vmode+other←char,vmode+char←num,vmode+char←given, vmode+math←shift,vmode+un←hbox,vmode+vrule, vmode+accent,vmode+discretionary,vmode+hskip,vmode+valign, vmode+ex←space:@t@>@;@/ begin back←input; new←graf(true); end; @ @<Declare act...@>= procedure new←graf(@!indented:boolean); begin prev←graf:=0; if (mode=vmode)or(head<>tail) then tail←append(new←param←glue(par←skip←code)); push←nest; mode:=hmode; space←factor:=1000; if indented then begin tail:=new←null←box; link(head):=tail; width(tail):=par←indent; end; if every←par<>null then begin←token←list(every←par,every←par←text); if nest←ptr=1 then build←page; {put |par←skip| glue on current page} end; @ @<Cases of |main←control| that build...@>= hmode+start←par,mmode+start←par: indent←in←hmode; @ @<Declare act...@>= procedure indent←in←hmode; var p,@!q:pointer; begin if cur←chr>0 then {\.{\\indent}} begin p:=new←null←box; width(p):=par←indent; if abs(mode)=hmode then space←factor:=1000 else begin q:=new←noad; math←type(nucleus(q)):=sub←box; info(nucleus(q)):=p; p:=q; end; tail←append(p); end; end; @ A paragraph ends when a |par←end| command is sensed, or when we are in horizontal mode when reaching the right brace of vertical-mode routines like \.{\\vbox}, \.{\\insert}, or \.{\\output}. @<Cases of |main←control| that build...@>= vmode+par←end: begin normal←paragraph; if mode>0 then build←page; end; hmode+par←end: begin if align←state<0 then off←save; {this tries to recover from an alignment that didn't end properly} end←graf; {this takes us to the enclosing mode, if |mode>0|} if mode=vmode then build←page; end; hmode+stop,hmode+vskip,hmode+hrule,hmode+un←vbox,hmode+halign: head←for←vmode; @ @<Declare act...@>= procedure head←for←vmode; begin if mode<0 then if cur←cmd<>hrule then off←save else begin print←err("You can't use `"); print←esc("hrule"); print("' here except with leaders"); @.You can't use \\hrule...@> help2("To put a horizontal rule in an hbox or an alignment,")@/ ("you should use \leaders or \hrulefill (see The TeXbook)."); error; end else begin back←input; cur←tok:=par←token; back←input; token←type:=inserted; end; end; @ @<Declare act...@>= procedure end←graf; begin if mode=hmode then begin if head=tail then pop←nest {null paragraphs are ignored} else line←break(widow←penalty); normal←paragraph; error←count:=0; end; end; @ Insertion and adjustment and mark nodes are constructed by the following pieces of the program. @<Cases of |main←control| that build...@>= any←mode(insert),hmode+vadjust,mmode+vadjust: begin←insert←or←adjust; any←mode(mark): make←mark; @ @<Forbidden...@>= vmode+vadjust, @ @<Declare act...@>= procedure begin←insert←or←adjust; begin if cur←cmd=vadjust then cur←val:=255 else begin scan←eight←bit←int; if cur←val=255 then begin print←err("You can't "); print←esc("insert"); print←int(255); @.You can't \\insert255@> help1("I'm changing to \insert0; box 255 is special."); error; cur←val:=0; end; end; saved(0):=cur←val; incr(save←ptr); new←save←level(insert←group); scan←left←brace; normal←paragraph; push←nest; mode:=-vmode; prev←depth:=ignore←depth; end; @ @<Cases of |handle...@>= insert←group: begin end←graf; q:=split←top←skip; add←glue←ref(q); d:=split←max←depth; f:=floating←penalty; unsave; decr(save←ptr); {now |saved(0)| is the insertion number, or 255 for |vadjust|} p:=vpack(link(head),natural); pop←nest; if saved(0)<255 then begin tail←append(get←node(ins←node←size)); type(tail):=ins←node; subtype(tail):=qi(saved(0)); height(tail):=height(p)+depth(p); ins←ptr(tail):=list←ptr(p); split←top←ptr(tail):=q; depth(tail):=d; float←cost(tail):=f; end else begin tail←append(get←node(small←node←size)); type(tail):=adjust←node;@/ subtype(tail):=0; {the |subtype| is not used} adjust←ptr(tail):=list←ptr(p); delete←glue←ref(q); end; free←node(p,box←node←size); if nest←ptr=0 then build←page; end; output←group: @<Resume the page builder...@>; @ @<Declare act...@>= procedure make←mark; var p:pointer; {new node} begin p:=scan←toks(false,true); p:=get←node(small←node←size); type(p):=mark←node; subtype(p):=0; {the |subtype| is not used} mark←ptr(p):=def←ref; link(tail):=p; tail:=p; end; @ Penalty nodes get into a list via the |break←penalty| command. @↑penalties@> @<Cases of |main←control| that build...@>= any←mode(break←penalty): append←penalty; @ @<Declare action...@>= procedure append←penalty; begin scan←int; tail←append(new←penalty(cur←val)); if mode=vmode then build←page; end; @ The |remove←item| command removes a penalty, kern, or glue node if it appears at the tail of the current list, using a brute-force linear scan. Like \.{\\lastbox}, this command is not allowed in vertical mode (except internal vertical mode), since the current list in vertical mode is sent to the page builder. But if we happen to be able to implement it in vertical mode, we do. @<Cases of |main←control| that build...@>= any←mode(remove←item): delete←last; @ When |delete←last| is called, |cur←chr| is the |type| of node that will be deleted, if present. @<Declare action...@>= procedure delete←last; var p:pointer; {runs through the current list} begin if (mode=vmode)and(tail=head) then @<Apologize for inability to do the operation now, unless \.{\\unskip} follows non-glue@> else begin if not is←char←node(tail) then if type(tail)=cur←chr then begin p:=head; while link(p)<>tail do p:=link(p); link(p):=null; flush←node←list(tail); tail:=p; end; end; end; @ @<Apologize for inability to do the operation...@>= begin if (cur←chr<>glue←node)or(last←glue<>max←halfword) then begin you←cant; help2("Sorry...I'm usually unable to take things from the current")@/ ("page. Try `I\vskip-\lastskip' instead."); if cur←chr=kern←node then help←line[0]:= ("page. Try `I\kern-\lastkern' instead.") else if cur←chr<>glue←node then help←line[0]:=@| ("page. Perhaps you can make the output routine do it."); error; end; end @ @<Put each...@>= primitive("unpenalty",remove←item,penalty←node);@/ @!@:un←penalty←}{\.{\\unpenalty} primitive@> primitive("unkern",remove←item,kern←node);@/ @!@:un←kern←}{\.{\\unkern} primitive@> primitive("unskip",remove←item,glue←node);@/ @!@:un←skip←}{\.{\\unskip} primitive@> primitive("unhbox",un←hbox,box←code);@/ @!@:un←hbox←}{\.{\\unhbox} primitive@> primitive("unhcopy",un←hbox,copy←code);@/ @!@:un←hcopy←}{\.{\\unhcopy} primitive@> primitive("unvbox",un←vbox,box←code);@/ @!@:un←vbox←}{\.{\\unvbox} primitive@> primitive("unvcopy",un←vbox,copy←code);@/ @!@:un←vcopy←}{\.{\\unvcopy} primitive@> @ @<Cases of |print←cmd←chr|...@>= remove←item: if chr←code=glue←node then print←esc("unskip") else if chr←code=kern←node then print←esc("unkern") else print←esc("unpenalty"); un←hbox: if chr←code=copy←code then print←esc("unhcopy") else print←esc("unhbox"); un←vbox: if chr←code=copy←code then print←esc("unvcopy") else print←esc("unvbox"); @ The |un←hbox| and |un←vbox| commands unwrap one of the 256 current boxes. @<Cases of |main←control| that build...@>= vmode+un←vbox,hmode+un←hbox,mmode+un←hbox: unpackage; @ @<Declare act...@>= procedure unpackage; label exit; var p:pointer; {the box} @!c:box←code..copy←code; {should we copy?} begin c:=cur←chr; scan←eight←bit←int; p:=box(cur←val); if p=null then return; if (abs(mode)=mmode)or((abs(mode)=vmode)and(type(p)<>vlist←node))or@| ((abs(mode)=hmode)and(type(p)<>hlist←node)) then begin print←err("Incompatible list can't be unboxed"); @.Incompatible list...@> help3("Sorry, Pandora. (You sneaky devil.)")@/ ("I refuse to unbox an \hbox in vertical mode or vice versa.")@/ ("And I can't open any boxes in math mode.");@/ error; return; end; if c=copy←code then link(tail):=copy←node←list(list←ptr(p)) else begin link(tail):=list←ptr(p); box(cur←val):=null; free←node(p,box←node←size); end; while link(tail)<>null do tail:=link(tail); exit:end; @ @<Forbidden...@>=vmode+ital←corr, @ Italic corrections are converted to kern nodes when the |ital←corr| command follows a character. In math mode the same effect is achieved by appending a kern of zero here, since italic corrections are supplied later. @<Cases of |main←control| that build...@>= hmode+ital←corr: append←italic←correction; mmode+ital←corr: tail←append(new←kern(0)); @ @<Declare act...@>= procedure append←italic←correction; label exit; var p:pointer; {|char←node| at the tail of the current list} @!f:internal←font←number; {the font in the |char←node|} begin if tail<>head then begin if is←char←node(tail) then p:=tail else if type(tail)=ligature←node then p:=lig←char(tail) else return; f:=font(p); tail←append(new←kern(char←italic(f)(char←info(f)(character(p))))); subtype(tail):=explicit; end; exit:end; @ Discretionary nodes are easy in the common case `\.{\\-}', but in the general case we must process three braces full of items. @<Put each...@>= primitive("-",discretionary,1); @!@:Single-character primitives -}{\quad\.{\\-}@> primitive("discretionary",discretionary,0); @!@:discretionary←}{\.{\\discretionary} primitive@> @ @<Cases of |print←cmd←chr|...@>= discretionary: if chr←code=1 then print←esc("-")@+else print←esc("discretionary"); @ @<Cases of |main←control| that build...@>= hmode+discretionary,mmode+discretionary: append←discretionary; @ The space factor does not change when we append a discretionary node, but it starts out as 1000 in the subsidiary lists. @<Declare act...@>= procedure append←discretionary; var c:integer; {hyphen character} begin tail←append(new←disc); if cur←chr=1 then begin c:=hyphen←char[cur←font]; if c>=0 then if c<256 then pre←break(tail):=new←character(cur←font,c); end else begin incr(save←ptr); saved(-1):=0; scan←left←brace; new←save←level(disc←group); push←nest; mode:=-hmode; space←factor:=1000; end; end; @ The three discretionary lists are constructed somewhat as if they were hboxes. A subroutine called |build←discretionary| handles the transitions. (This is sort of fun.) @<Cases of |handle...@>= disc←group: build←discretionary; @ @<Declare act...@>= procedure build←discretionary; label done,exit; var p,@!q:pointer; {for link manipulation} @!n:integer; {length of discretionary list} begin unsave; @<Prune the current list, if necessary, until it contains only |char←node|, |kern←node|, |hlist←node|, |vlist←node|, |rule←node|, and |ligature←node| items; set |n| to the length of the list, and set |q| to the list's tail@>; p:=link(head); pop←nest; case saved(-1) of 0:pre←break(tail):=p; 1:post←break(tail):=p; 2:@<Attach list |p| to the current list, and record its length; then finish up and |return|@>; end; {there are no other cases} incr(saved(-1)); scan←left←brace; new←save←level(disc←group); push←nest; mode:=-hmode; space←factor:=1000; exit:end; @ @<Attach list |p| to the current...@>= begin if (n>0)and(abs(mode)=mmode) then begin print←err("Illegal math "); print←esc("discretionary"); @.Illegal math \\disc...@> help2("Sorry: The third part of a discretionary break must be")@/ ("empty, in math formulas. I had to delete your third part."); flush←node←list(p); n:=0; error; end else link(tail):=p; if n<128 then replace←count(tail):=n else begin print←err("Discretionary list is too long"); @.Discretionary list is too long@> help2("Wow---I never thought anybody would tweak me here.")@/ ("You can't seriously need such a huge discretionary list?"); error; end; if n>0 then tail:=q; decr(save←ptr); return; end @ During this loop, |p=link(q)| and there are |n| items preceding |p|. @<Prune the current list, if necessary...@>= q:=head; p:=link(q); n:=0; while p<>null do begin if not is←char←node(p) then if type(p)>rule←node then if type(p)<>kern←node then if type(p)<>ligature←node then begin print←err("Improper discretionary list"); @.Improper discretionary list@> help1("Discretionary lists must contain only boxes and kerns.");@/ error; begin←diagnostic; print←nl("The following discretionary sublist has been deleted:"); @.The following...deleted@> show←box(p); end←diagnostic(true); flush←node←list(p); link(q):=null; goto done; end; q:=p; p:=link(q); incr(n); end; done: @ We need only one more thing to complete the horizontal mode routines, namely the \.{\\accent} primitive. @<Cases of |main←control| that build...@>= hmode+accent: make←accent; @ The positioning of accents is straightforward but tedious. Given an accent of width |a|, designed for characters of height |x| and slant |s|; and given a character of width |w|, height |h|, and slant |t|: We will shift the accent down by |x-h|, and we will insert kern nodes that have the effect of centering the accent over the character and shifting the accent to the right by $\delta={1\over2}(w-a)+h\cdot t-x\cdot s$. If either character is absent from the font, we will simply use the other, without shifting. @<Declare act...@>= procedure make←accent; var s,@!t: real; {amount of slant} @!p,@!q,@!r:pointer; {character, box, and kern nodes} @!f:internal←font←number; {relevant font} @!a,@!h,@!x,@!w,@!delta:scaled; {heights and widths, as explained above} @!i:four←quarters; {character information} begin scan←char←num; f:=cur←font; p:=new←character(f,cur←val); if p<>null then begin x:=x←height(f); s:=slant(f)/float←constant(65536); @↑real division@> a:=char←width(f)(char←info(f)(character(p)));@/ do←assignments;@/ @<Create a character node |q| for the next character, but set |q:=null| if problems arise@>; if q<>null then @<Append the accent with appropriate kerns, then set |p:=q|@>; link(tail):=p; tail:=p; space←factor:=1000; end; end; @ @<Create a character node |q| for the next...@>= q:=null; f:=cur←font; if (cur←cmd=letter)or(cur←cmd=other←char)or(cur←cmd=char←given) then q:=new←character(f,cur←chr) else if cur←cmd=char←num then begin scan←char←num; q:=new←character(f,cur←val); end else back←input @ The kern nodes appended here must be explicit, lest they be wiped away by the hyphenation algorithm. @<Append the accent with appropriate kerns...@>= begin t:=slant(f)/float←constant(65536); @↑real division@> i:=char←info(f)(character(q)); w:=char←width(f)(i); h:=char←height(f)(height←depth(i)); if h<>x then {the accent must be shifted up or down} begin p:=hpack(p,natural); shift←amount(p):=x-h; end; delta:=round((w-a)/float←constant(2)+h*t-x*s); @↑real multiplication@> r:=new←kern(delta); subtype(r):=explicit; link(tail):=r; link(r):=p; tail:=new←kern(-a-delta); subtype(tail):=explicit; link(p):=tail; p:=q; end @ When `\.{\\cr}' or `\.{\\span}' or a tab mark comes through the scanner into |main←control|, it might be that the user has foolishly inserted one of them into something that has nothing to do with alignment. But it is far more likely that a left brace or right brace has been omitted, since |get←next| takes actions appropriate to alignment only when `\.{\\cr}' or `\.{\\span}' or tab marks occur with |align←state=0|. The following program attempts to make an appropriate recovery. @<Cases of |main←control| that build...@>= any←mode(car←ret), any←mode(tab←mark): align←error; any←mode(no←align): no←align←error; any←mode(omit): omit←error; @ @<Declare act...@>= procedure align←error; begin if abs(align←state)>2 then @<Express consternation over the fact that no alignment is in progress@> else begin back←input; if align←state<0 then begin print←err("Missing { inserted"); @.Missing \{ inserted@> incr(align←state); cur←tok:=left←brace←token+"{"; end else begin print←err("Missing } inserted"); @.Missing \} inserted@> decr(align←state); cur←tok:=right←brace←token+"}"; end; help3("I've put in what seems to be necessary to fix")@/ ("the current column of the current alignment.")@/ ("Try to go on, since this might almost work."); ins←error; end; end; @ @<Express consternation...@>= begin print←err("Misplaced "); print←cmd←chr(cur←cmd,cur←chr); @.Misplaced \&@> @.Misplaced \\span@> @.Misplaced \\cr@> if cur←tok=tab←token+"&" then begin help6("I can't figure out why you would want to use a tab mark")@/ ("here. If you just want an ampersand, the remedy is")@/ ("simple: Just type `I\&' now. But if some right brace")@/ ("up above has ended a previous alignment prematurely,")@/ ("you're probably due for more error messages, and you")@/ ("might try typing `S' now just to see what is salvageable."); end else begin help5("I can't figure out why you would want to use a tab mark")@/ ("or \cr or \span just now. If something like a right brace")@/ ("up above has ended a previous alignment prematurely,")@/ ("you're probably due for more error messages, and you")@/ ("might try typing `S' now just to see what is salvageable."); end; error; end @ The help messages here contain a little white lie, since \.{\\noalign} and \.{\\omit} are allowed also after `\.{\\noalign\{...\}}'. @<Declare act...@>= procedure no←align←error; begin print←err("Misplaced "); print←esc("noalign"); @.Misplaced \\noalign@> help2("I expect to see \noalign only after the \cr of")@/ ("an alignment. Proceed, and I'll ignore this case."); error; end; procedure omit←error; begin print←err("Misplaced "); print←esc("omit"); @.Misplaced \\omit@> help2("I expect to see \omit only after tab marks or the \cr of")@/ ("an alignment. Proceed, and I'll ignore this case."); error; end; @ We've now covered most of the abuses of \.{\\halign} and \.{\\valign}. Let's take a look at what happens when they are used correctly. @<Cases of |main←control| that build...@>= vmode+halign,hmode+valign:init←align; mmode+halign: if privileged then init←align; vmode+endv,hmode+endv: do←endv; @ An |align←group| code is supposed to remain on the |save←stack| during an entire alignment, until |fin←align| removes it. @<Declare act...@>= procedure do←endv; begin if cur←group=align←group then begin end←graf; if fin←col then fin←row; end else off←save; end; @ @<Cases of |handle←right←brace|...@>= align←group: begin back←input; cur←tok:=cs←token←flag+frozen←cr; print←err("Missing "); print←esc("cr"); print(" inserted"); @.Missing \\cr inserted@> help1("I'm guessing that you meant to end an alignment here."); ins←error; end; @ @<Cases of |handle←right←brace|...@>= no←align←group: begin end←graf; unsave; align←peek; end; @ Finally, \.{\\endcsname} is not supposed to get through to |main←control|. @<Cases of |main←control| that build...@>= any←mode(end←cs←name): cs←error; @ @<Declare act...@>= procedure cs←error; begin print←err("Extra "); print←esc("endcsname"); @.Extra \\endcsname@> help1("I'm ignoring this, since I wasn't doing a \csname."); error; end; @* \[48] Building math lists. The routines that \TeX\ uses to create mlists are similar to those we have just seen for the generation of hlists and vlists. But it is necessary to make ``noads'' as well as nodes, so the reader should review the discussion of math mode data structures before trying to make sense out of the following program. Here is a little routine that needs to be done whenever a subformula is about to be processed. The parameter is a code like |math←group|. @<Declare act...@>= procedure push←math(@!c:group←code); begin push←nest; mode:=-mmode; incompleat←noad:=null; new←save←level(c); end; @ We get into math mode from horizontal mode when a `\.\$' (i.e., a |math←shift| character) is scanned. We must check to see whether this `\.\$' is immediately followed by another, in case display math mode is called for. @<Cases of |main←control| that build...@>= hmode+math←shift:init←math; @ @<Declare act...@>= procedure init←math; label reswitch,found,not←found,done; var w:scaled; {new or partial |pre←display←size|} @!l:scaled; {new |display←width|} @!s:scaled; {new |display←indent|} @!p:pointer; {current node when calculating |pre←display←size|} @!q:pointer; {glue specification when calculating |pre←display←size|} @!f:internal←font←number; {font in current |char←node|} @!n:integer; {scope of paragraph shape specification} @!v:scaled; {|w| plus possible glue amount} @!d:scaled; {increment to |v|} begin get←token; {|get←x←token| would fail on \.{\\ifmmode}!} if (cur←cmd=math←shift)and(mode>0) then @<Go into display math mode@> else begin back←input; @<Go into ordinary math mode@>; end; end; @ @<Go into ordinary math mode@>= begin push←math(math←shift←group); eq←word←define(int←base+cur←fam←code,-1); if every←math<>null then begin←token←list(every←math,every←math←text); end @ We get into ordinary math mode from display math mode when `\.{\\eqno}' or `\.{\\leqno}' appears. In such cases |cur←chr| will be 0 or~1, respectively; the value of |cur←chr| is placed onto |save←stack| for safe keeping. @<Cases of |main←control| that build...@>= mmode+eq←no: if privileged then start←eq←no; @ @<Put each...@>= primitive("eqno",eq←no,0); @!@:eq←no←}{\.{\\eqno} primitive@> primitive("leqno",eq←no,1); @!@:leq←no←}{\.{\\leqno} primitive@> @ When \TeX\ is in display math mode, |cur←group=math←shift←group|, so it is not necessary for the |start←eq←no| procedure to test for this condition. @<Declare act...@>= procedure start←eq←no; begin saved(0):=cur←chr; incr(save←ptr); @<Go into ordinary math mode@>; end; @ @<Cases of |print←cmd←chr|...@>= eq←no:if chr←code=1 then print←esc("leqno")@+else print←esc("eqno"); @ @<Forbidden...@>=vmode+eq←no,hmode+eq←no, @ When we enter display math mode, we need to call |line←break| to process the partial paragraph that has just been interrupted by the display. Then we can set the proper values of |display←width| and |display←indent| and |pre←display←size|. @<Go into display math mode@>= begin if head=tail then {`\.{\\noindent\$\$}' or `\.{\$\${ }\$\$}'} begin pop←nest; w:=-max←dimen; end else begin line←break(display←widow←penalty);@/ @<Calculate the natural width, |w|, by which the characters of the final line extend to the right of the reference point, plus two ems; or set |w:=max←dimen| if the non-blank information on that line is affected by stretching or shrinking@>; end; {Now we are in vertical mode, working on the list that will contain the display} @<Calculate the length, |l|, and the shift amount, |s|, of the display lines@>; push←math(math←shift←group); mode:=mmode; eq←word←define(int←base+cur←fam←code,-1);@/ eq←word←define(dimen←base+pre←display←size←code,w); eq←word←define(dimen←base+display←width←code,l); eq←word←define(dimen←base+display←indent←code,s); if every←display<>null then begin←token←list(every←display,every←display←text); if nest←ptr=1 then build←page; end @ @<Calculate the natural width, |w|, by which...@>= v:=shift←amount(just←box)+2*quad(cur←font); w:=-max←dimen; p:=list←ptr(just←box); while p<>null do begin @<Let |d| be the natural width of node |p|; if the node is ``visible,'' |goto found|; if the node is glue that stretches or shrinks, set |v:=max←dimen|@>; if v<max←dimen then v:=v+d; goto not←found; found: if v<max←dimen then begin v:=v+d; w:=v; end else begin w:=max←dimen; goto done; end; not←found: p:=link(p); end; done: @ @<Let |d| be the natural width of node |p|...@>= reswitch: if is←char←node(p) then begin f:=font(p); d:=char←width(f)(char←info(f)(character(p))); goto found; end; case type(p) of hlist←node,vlist←node,rule←node: begin d:=width(p); goto found; end; ligature←node:@<Make node |p| look like a |char←node|...@>; kern←node,math←node: d:=width(p); glue←node:@<Let |d| be the natural width of this glue; if stretching or shrinking, set |v:=max←dimen|; |goto found| in the case of leaders@>; whatsit←node: @<Let |d| be the width of the whatsit |p|@>; othercases d:=0 endcases @ We need to be careful that |w|, |v|, and |d| do not depend on any |glue←set| values, since such values are subject to system-dependent rounding. System-dependent numbers are not allowed to infiltrate parameters like |pre←display←size|, since \TeX82 is supposed to make the same decisions on all machines. @<Let |d| be the natural width of this glue...@>= begin q:=glue←ptr(p); d:=width(q); if glue←sign(just←box)=stretching then begin if (glue←order(just←box)=stretch←order(q))and@| (stretch(q)<>0) then v:=max←dimen; end else if glue←sign(just←box)=shrinking then begin if (glue←order(just←box)=shrink←order(q))and@| (shrink(q)<>0) then v:=max←dimen; end; if subtype(p)>=a←leaders then goto found; end @ A displayed equation is considered to be three lines long, so we calculate the length and offset of line number |prev←graf+2|. @<Calculate the length, |l|, ...@>= if par←shape←ptr=null then if (hang←indent<>0)and@| (((hang←after>=0)and(prev←graf+2>hang←after))or@| (prev←graf+1<-hang←after)) then begin l:=hsize-abs(hang←indent); if hang←indent>0 then s:=hang←indent@+else s:=0; end else begin l:=hsize; s:=0; end else begin n:=info(par←shape←ptr); if prev←graf+2>=n then p:=par←shape←ptr+2*n else p:=par←shape←ptr+2*(prev←graf+2); s:=mem[p-1].sc; l:=mem[p].sc; end @ Subformulas of math formulas cause a new level of math mode to be entered, on the semantic nest as well as the save stack. These subformulas arise in several ways: (1)~A left brace by itself indicates the beginning of a subformula that will be put into a box, thereby freezing its glue and preventing line breaks. (2)~A subscript or superscript is treated as a subformula if it is not a single character; the same applies to the nucleus of things like \.{\\underline}. (3)~The \.{\\left} primitive initiates a subformula that will be terminated by a matching \.{\\right}. The group codes placed on |save←stack| in these three cases are |math←group|, |math←group|, and |math←left←group|, respectively. Here is the code that handles case (1); the other cases are not quite as trivial, so we shall consider them later. @<Cases of |main←control| that build...@>= mmode+left←brace: begin tail←append(new←noad); back←input; scan←math(nucleus(tail)); end; @ Recall that the |nucleus|, |subscr|, and |supscr| fields in a noad are broken down into subfields called |math←type| and either |info| or |(fam,character)|. The job of |scan←math| is to figure out what to place in one of these principal fields; it looks at the subformula that comes next in the input, and places an encoding of that subformula into a given word of |mem|. @d fam←in←range==((cur←fam>=0)and(cur←fam<16)) @<Declare act...@>= procedure scan←math(@!p:pointer); label restart,reswitch,exit; var c:integer; {math character code} begin restart:@<Get the next non-blank non-relax...@>; reswitch:case cur←cmd of letter,other←char,char←given: if cur←chr>=128 then c:=cur←chr else begin c:=ho(math←code(cur←chr)); if c=@'100000 then begin @<Treat |cur←chr| as an active character@>; goto restart; end; end; char←num: begin scan←char←num; cur←chr:=cur←val; cur←cmd:=char←given; goto reswitch; end; math←char←num: begin scan←fifteen←bit←int; c:=cur←val; end; math←given: c:=cur←chr; delim←num: begin scan←twenty←seven←bit←int; c:=cur←val div @'10000; end; othercases @<Scan a subformula enclosed in braces and |return|@> endcases;@/ math←type(p):=math←char; character(p):=qi(c mod 256); if (c>=var←code)and fam←in←range then fam(p):=cur←fam else fam(p):=(c div 256) mod 16; exit:end; @ An active character that is an |outer←call| is allowed here. @<Treat |cur←chr|...@>= begin cur←cs:=cur←chr+active←base; cur←cmd:=eq←type(cur←cs); cur←chr:=equiv(cur←cs); x←token; back←input; end @ The pointer |p| is placed on |save←stack| while a complex subformula is being scanned. @<Scan a subformula...@>= begin back←input; scan←left←brace;@/ saved(0):=p; incr(save←ptr); push←math(math←group); return; end @ The simplest math formula is, of course, `\.{\${ }\$}', when no noads are generated. The next simplest cases involve a single character, e.g., `\.{\$x\$}'. Even though such cases may not seem to be very interesting, the reader can perhaps understand how happy the author was when `\.{\$x\$}' was first properly typeset by \TeX. The code in this section was used. @↑Knuth, Donald Ervin@> @<Cases of |main←control| that build...@>= mmode+letter,mmode+other←char,mmode+char←given: if cur←chr<128 then set←math←char(ho(math←code(cur←chr))) else set←math←char(cur←chr); mmode+char←num: begin scan←char←num; cur←chr:=cur←val; if cur←chr<128 then set←math←char(ho(math←code(cur←chr))) else set←math←char(cur←chr); end; mmode+math←char←num: begin scan←fifteen←bit←int; set←math←char(cur←val); end; mmode+math←given: set←math←char(cur←chr); mmode+delim←num: begin scan←twenty←seven←bit←int; set←math←char(cur←val div @'10000); end; @ The |set←math←char| procedure creates a new noad appropriate to a given math code, and appends it to the current mlist. However, if the math code is sufficiently large, the |cur←chr| is treated as an active character and nothing is appended. @<Declare act...@>= procedure set←math←char(@!c:integer); var p:pointer; {the new noad} begin if c>=@'100000 then @<Treat |cur←chr|...@> else begin p:=new←noad; math←type(nucleus(p)):=math←char; character(nucleus(p)):=qi(c mod 256); fam(nucleus(p)):=(c div 256) mod 16; if c>=var←code then begin if fam←in←range then fam(nucleus(p)):=cur←fam; type(p):=ord←noad; end else type(p):=ord←noad+(c div @'10000); link(tail):=p; tail:=p; end; end; @ Primitive math operators like \.{\\mathop} and \.{\\underline} are given the command code |math←comp|, supplemented by the noad type that they generate. @<Put each...@>= primitive("mathord",math←comp,ord←noad); @!@:math←ord←}{\.{\\mathord} primitive@> primitive("mathop",math←comp,op←noad); @!@:math←op←}{\.{\\mathop} primitive@> primitive("mathbin",math←comp,bin←noad); @!@:math←bin←}{\.{\\mathbin} primitive@> primitive("mathrel",math←comp,rel←noad); @!@:math←rel←}{\.{\\mathrel} primitive@> primitive("mathopen",math←comp,open←noad); @!@:math←open←}{\.{\\mathopen} primitive@> primitive("mathclose",math←comp,close←noad); @!@:math←close←}{\.{\\mathclose} primitive@> primitive("mathpunct",math←comp,punct←noad); @!@:math←punct←}{\.{\\mathpunct} primitive@> primitive("mathinner",math←comp,inner←noad); @!@:math←inner←}{\.{\\mathinner} primitive@> primitive("underline",math←comp,under←noad); @!@:underline←}{\.{\\underline} primitive@> primitive("overline",math←comp,over←noad);@/ @!@:overline←}{\.{\\overline} primitive@> primitive("displaylimits",limit←switch,normal); @!@:display←limits←}{\.{\\displaylimits} primitive@> primitive("limits",limit←switch,limits); @!@:limits←}{\.{\\limits} primitive@> primitive("nolimits",limit←switch,no←limits); @!@:no←limits←}{\.{\\nolimits} primitive@> @ @<Cases of |print←cmd←chr|...@>= math←comp: case chr←code of ord←noad: print←esc("mathord"); op←noad: print←esc("mathop"); bin←noad: print←esc("mathbin"); rel←noad: print←esc("mathrel"); open←noad: print←esc("mathopen"); close←noad: print←esc("mathclose"); punct←noad: print←esc("mathpunct"); inner←noad: print←esc("mathinner"); under←noad: print←esc("underline"); othercases print←esc("overline") endcases; limit←switch: if chr←code=limits then print←esc("limits") else if chr←code=no←limits then print←esc("nolimits") else print←esc("displaylimits"); @ @<Cases of |main←control| that build...@>= mmode+math←comp: begin tail←append(new←noad); type(tail):=cur←chr; scan←math(nucleus(tail)); end; mmode+limit←switch: math←limit←switch; @ @<Declare act...@>= procedure math←limit←switch; label exit; begin if head<>tail then if type(tail)=op←noad then begin subtype(tail):=cur←chr; return; end; print←err("Limit controls must follow a math operator"); @.Limit controls must follow...@> help1("I'm ignoring this misplaced \limits or \nolimits command."); error; exit:end; @ Delimiter fields of noads are filled in by the |scan←delimiter| routine. The first parameter of this procedure is the |mem| address where the delimiter is to be placed; the second tells if this delimiter follows \.{\\radical} or not. @<Declare act...@>= procedure scan←delimiter(@!p:pointer;@!r:boolean); begin if r then scan←twenty←seven←bit←int else begin @<Get the next non-blank non-relax...@>; case cur←cmd of letter,other←char: cur←val:=del←code(cur←chr); delim←num: scan←twenty←seven←bit←int; othercases cur←val:=-1 endcases; end; if cur←val<0 then @<Report that an invalid delimiter code is being changed to null; set~|cur←val:=0|@>; small←fam(p):=(cur←val div @'4000000) mod 16; small←char(p):=qi((cur←val div @'10000) mod 256); large←fam(p):=(cur←val div 256) mod 16; large←char(p):=qi(cur←val mod 256); end; @ @<Report that an invalid delimiter...@>= begin print←err("Missing delimiter (. inserted)"); @.Missing delimiter...@> help6("I was expecting to see something like `(' or `\{' or")@/ ("`\}' here. If you typed, e.g., `{' instead of `\{', you")@/ ("should probably delete the `{' by typing `1' now, so that")@/ ("braces don't get unbalanced. Otherwise just proceed.")@/ ("Acceptable delimiters are characters whose \delcode is")@/ ("nonnegative, or you can use `\delimiter <delimiter code>'."); back←error; cur←val:=0; end @ @<Cases of |main←control| that build...@>= mmode+radical:math←radical; @ @<Declare act...@>= procedure math←radical; begin tail←append(get←node(radical←noad←size)); type(tail):=radical←noad; subtype(tail):=normal; mem[nucleus(tail)].hh:=empty←field; mem[subscr(tail)].hh:=empty←field; mem[supscr(tail)].hh:=empty←field; scan←delimiter(left←delimiter(tail),true); scan←math(nucleus(tail)); end; @ @<Cases of |main←control| that build...@>= mmode+accent,mmode+math←accent:math←ac; @ @<Declare act...@>= procedure math←ac; begin if cur←cmd=accent then @<Complain that the user should have said \.{\\mathaccent}@>; tail←append(get←node(accent←noad←size)); type(tail):=accent←noad; subtype(tail):=normal; mem[nucleus(tail)].hh:=empty←field; mem[subscr(tail)].hh:=empty←field; mem[supscr(tail)].hh:=empty←field; math←type(accent←chr(tail)):=math←char; scan←fifteen←bit←int; character(accent←chr(tail)):=qi(cur←val mod 256); if (cur←val>=var←code)and fam←in←range then fam(accent←chr(tail)):=cur←fam else fam(accent←chr(tail)):=(cur←val div 256) mod 16; scan←math(nucleus(tail)); end; @ @<Complain that the user should have said \.{\\mathaccent}@>= begin print←err("Please use "); print←esc("mathaccent"); print(" for accents in math mode"); @.Please use \\mathaccent...@> help2("I'm changing \accent to \mathaccent here; wish me luck.")@/ ("(Accents are not the same in formulas as they are in text.)"); error; end @ @<Cases of |main←control| that build...@>= mmode+vcenter: begin scan←spec; new←save←level(vcenter←group); normal←paragraph; push←nest; mode:=-vmode; prev←depth:=ignore←depth; if every←vbox<>null then begin←token←list(every←vbox,every←vbox←text); end; @ @<Cases of |handle...@>= vcenter←group: begin end←graf; unsave; save←ptr:=save←ptr-2; p:=vpack(link(head),saved(1),saved(0)); pop←nest; tail←append(new←noad); type(tail):=vcenter←noad; math←type(nucleus(tail)):=sub←box; info(nucleus(tail)):=p; end; @ The routine that inserts a |style←node| holds no surprises. @<Put each...@>= primitive("displaystyle",math←style,display←style); @!@:display←style←}{\.{\\displaystyle} primitive@> primitive("textstyle",math←style,text←style); @!@:text←style←}{\.{\\textstyle} primitive@> primitive("scriptstyle",math←style,script←style); @!@:script←style←}{\.{\\scriptstyle} primitive@> primitive("scriptscriptstyle",math←style,script←script←style); @!@:script←script←style←}{\.{\\scriptscriptstyle} primitive@> @ @<Cases of |print←cmd←chr|...@>= math←style: print←style(chr←code); @ @<Cases of |main←control| that build...@>= mmode+math←style: tail←append(new←style(cur←chr)); mmode+non←script: begin tail←append(new←glue(zero←glue)); subtype(tail):=cond←math←glue; end; mmode+math←choice: append←choices; @ The routine that scans the four mlists of a \.{\\mathchoice} is very much like the routine that builds discretionary nodes. @<Declare act...@>= procedure append←choices; begin tail←append(new←choice); incr(save←ptr); saved(-1):=0; scan←left←brace; push←math(math←choice←group); end; @ @<Cases of |handle←right←brace|...@>= math←choice←group: build←choices; @ @<Declare act...@>= @t\4@>@<Declare the function called |fin←mlist|@>@t@>@;@/ procedure build←choices; label exit; var p:pointer; {the current mlist} begin unsave; p:=fin←mlist(null); case saved(-1) of 0:display←mlist(tail):=p; 1:text←mlist(tail):=p; 2:script←mlist(tail):=p; 3:begin script←script←mlist(tail):=p; decr(save←ptr); return; end; end; {there are no other cases} incr(saved(-1)); scan←left←brace; push←math(math←choice←group); exit:end; @ Subscripts and superscripts are attached to the previous nucleus by the @↑superscripts@>@↑subscripts@> action procedure called |sub←sup|. We use the facts that |sub←mark=sup←mark+1| and |subscr(p)=supscr(p)+1|. @<Cases of |main←control| that build...@>= mmode+sub←mark,mmode+sup←mark: sub←sup; @ @<Declare act...@>= procedure sub←sup; var t:small←number; {type of previous sub/superscript} @!p:pointer; {field to be filled by |scan←math|} begin t:=empty; p:=null; if tail<>head then if scripts←allowed(tail) then begin p:=supscr(tail)+cur←cmd-sup←mark; {|supscr| or |subscr|} t:=math←type(p); end; if (p=null)or(t<>empty) then @<Insert a dummy noad to be sub/superscripted@>; scan←math(p); end; @ @<Insert a dummy...@>= begin tail←append(new←noad); p:=supscr(tail)+cur←cmd-sup←mark; {|supscr| or |subscr|} if t<>empty then begin if cur←cmd=sup←mark then begin print←err("Double superscript"); @.Double superscript@> help1("I treat `x↑1↑2' essentially like `x↑1{}↑2'."); end else begin print←err("Double subscript"); @.Double subscript@> help1("I treat `x←1←2' essentially like `x←1{}←2'."); end; error; end; end @ An operation like `\.{\\over}' causes the current mlist to go into a state of suspended animation: |incompleat←noad| points to a |fraction←noad| that contains the mlist-so-far as its numerator, while the denominator is yet to come. Finally when the mlist is finished, the denominator will go into the incompleat fraction noad, and that noad will become the whole formula, unless it is surrounded by `\.{\\left}' and `\.{\\right}' delimiters. @d above←code=0 { `\.{\\above}' } @d over←code=1 { `\.{\\over}' } @d atop←code=2 { `\.{\\atop}' } @d delimited←code=3 { `\.{\\abovewithdelims}', etc.} @<Put each...@>= primitive("above",above,above←code);@/ @!@:above←}{\.{\\above} primitive@> primitive("over",above,over←code);@/ @!@:over←}{\.{\\over} primitive@> primitive("atop",above,atop←code);@/ @!@:atop←}{\.{\\atop} primitive@> primitive("abovewithdelims",above,delimited←code+above←code);@/ @!@:above←with←delims←}{\.{\\abovewithdelims} primitive@> primitive("overwithdelims",above,delimited←code+over←code);@/ @!@:over←with←delims←}{\.{\\overwithdelims} primitive@> primitive("atopwithdelims",above,delimited←code+atop←code); @!@:atop←with←delims←}{\.{\\atopwithdelims} primitive@> @ @<Cases of |print←cmd←chr|...@>= above: case chr←code of over←code:print←esc("over"); atop←code:print←esc("atop"); delimited←code+above←code:print←esc("abovewithdelims"); delimited←code+over←code:print←esc("overwithdelims"); delimited←code+atop←code:print←esc("atopwithdelims"); othercases print←esc("above") endcases; @ @<Cases of |main←control| that build...@>= mmode+above: math←fraction; @ @<Declare act...@>= procedure math←fraction; var c:small←number; {the type of generalized fraction we are scanning} begin c:=cur←chr; if incompleat←noad<>null then @<Ignore the fraction operation and complain about this ambiguous case@> else begin incompleat←noad:=get←node(fraction←noad←size); type(incompleat←noad):=fraction←noad; subtype(incompleat←noad):=normal; math←type(numerator(incompleat←noad)):=sub←mlist; info(numerator(incompleat←noad)):=link(head); mem[denominator(incompleat←noad)].hh:=empty←field; mem[left←delimiter(incompleat←noad)].qqqq:=null←delimiter; mem[right←delimiter(incompleat←noad)].qqqq:=null←delimiter;@/ link(head):=null; tail:=head; @<Use code |c| to distinguish between generalized fractions@>; end; end; @ @<Use code |c|...@>= if c>=delimited←code then begin scan←delimiter(left←delimiter(incompleat←noad),false); scan←delimiter(right←delimiter(incompleat←noad),false); end; case c mod delimited←code of above←code: begin scan←normal←dimen; thickness(incompleat←noad):=cur←val; end; over←code: thickness(incompleat←noad):=default←code; atop←code: thickness(incompleat←noad):=0; end {there are no other cases} @ @<Ignore the fraction...@>= begin if c>=delimited←code then begin scan←delimiter(garbage,false); scan←delimiter(garbage,false); end; if c mod delimited←code=above←code then scan←normal←dimen; print←err("Ambiguous; you need another { and }"); @.Ambiguous...@> help3("I'm ignoring this fraction specification, since I don't")@/ ("know whether a construction like `x \over y \over z'")@/ ("means `{x \over y} \over z' or `x \over {y \over z}'."); error; end @ At the end of a math formula or subformula, the |fin←mlist| routine is called upon to return a pointer to the newly completed mlist, and to pop the nest back to the enclosing semantic level. The parameter to |fin←mlist|, if not null, points to a |right←noad| that ends the current mlist; this |right←noad| has not yet been appended. @<Declare the function called |fin←mlist|@>= function fin←mlist(@!p:pointer):pointer; var q:pointer; {the mlist to return} begin if incompleat←noad<>null then @<Compleat the incompleat noad@> else begin link(tail):=p; q:=link(head); end; pop←nest; fin←mlist:=q; end; @ @<Compleat...@>= begin math←type(denominator(incompleat←noad)):=sub←mlist; info(denominator(incompleat←noad)):=link(head); if p=null then q:=incompleat←noad else begin q:=info(numerator(incompleat←noad)); if type(q)<>left←noad then confusion("right"); @:this can't happen right}{\quad right@> info(numerator(incompleat←noad)):=link(q); link(q):=incompleat←noad; link(incompleat←noad):=p; end; end @ Now at last we're ready to see what happens when a right brace occurs in a math formula. Two special cases are simplified here: Braces are effectively removed when they surround a single Ord character or, when they surround an accent that is the nucleus of an Ord atom. @<Cases of |handle...@>= math←group: begin unsave; decr(save←ptr);@/ math←type(saved(0)):=sub←mlist; p:=fin←mlist(null); info(saved(0)):=p; if p<>null then if link(p)=null then if type(p)=ord←noad then begin if math←type(subscr(p))=empty then if math←type(supscr(p))=empty then begin mem[saved(0)].hh:=mem[nucleus(p)].hh; free←node(p,noad←size); end; end else if type(p)=accent←noad then if saved(0)=nucleus(tail) then if type(tail)=ord←noad then @<Replace the tail of the list by |p|@>; end; @ @<Replace the tail...@>= begin q:=head; while link(q)<>tail do q:=link(q); link(q):=p; free←node(tail,noad←size); tail:=p; end @ We have dealt with all constructions of math mode except `\.{\\left}' and `\.{\\right}', so the picture is completed by the following sections of the program. @<Put each...@>= primitive("left",left←right,left←noad); @!@:left←}{\.{\\left} primitive@> primitive("right",left←right,right←noad); @!@:right←}{\.{\\right} primitive@> text(frozen←right):="right"; eqtb[frozen←right]:=eqtb[cur←val]; @ @<Cases of |print←cmd←chr|...@>= left←right: if chr←code=left←noad then print←esc("left") else print←esc("right"); @ @<Cases of |main←control| that build...@>= mmode+left←right: math←left←right; @ @<Declare act...@>= procedure math←left←right; var t:small←number; {|left←noad| or |right←noad|} @!p:pointer; {new noad} begin t:=cur←chr; if (t=right←noad)and(cur←group<>math←left←group) then @<Try to recover from mismatched \.{\\right}@> else begin p:=new←noad; type(p):=t; scan←delimiter(delimiter(p),false); if t=left←noad then begin push←math(math←left←group); link(head):=p; tail:=p; end else begin p:=fin←mlist(p); unsave; {end of |math←left←group|} tail←append(new←noad); type(tail):=inner←noad; math←type(nucleus(tail)):=sub←mlist; info(nucleus(tail)):=p; end; end; end; @ @<Try to recover from mismatch...@>= begin if cur←group=math←shift←group then begin scan←delimiter(garbage,false); print←err("Extra "); print←esc("right"); @.Extra \\right.@> help1("I'm ignoring a \right that had no matching \left."); error; end else off←save; end @ Here is the only way out of math mode. @<Cases of |main←control| that build...@>= mmode+math←shift: if cur←group=math←shift←group then after←math else off←save; @ @<Declare act...@>= procedure after←math; var l:boolean; {`\.{\\leqno}' instead of `\.{\\eqno}'} @!danger:boolean; {not enough symbol fonts are present} @!m:integer; {|mmode| or |-mmode|} @!p:pointer; {the formula} @!a:pointer; {box containing equation number} @<Local variables for finishing a displayed formula@>@; begin danger:=false; @<Check that the necessary fonts for math symbols are present; if not, flush the current math lists and set |danger:=true|@>; m:=mode; l:=false; p:=fin←mlist(null); {this pops the nest} if mode=-m then {end of equation number} begin cur←mlist:=p; cur←style:=text←style; mlist←penalties:=false; mlist←to←hlist; a:=hpack(link(temp←head),natural); unsave; decr(save←ptr); {now |cur←group=math←shift←group|} if saved(0)=1 then l:=true; if danger then flush←math; m:=mode; p:=fin←mlist(null); end else a:=null; if m<0 then @<Finish math in text@> else begin @<Check that another \.\$ follows@>; @<Finish displayed math@>; end; end; @ @<Check that the necessary fonts...@>=@t\2@>@/ if (font←params[fam←fnt(2+text←size)]<total←mathsy←params)or@| (font←params[fam←fnt(2+script←size)]<total←mathsy←params)or@| (font←params[fam←fnt(2+script←script←size)]<total←mathsy←params) then begin print←err("Math formula deleted: Insufficient symbol fonts");@/ @.Math formula deleted...@> help3("Sorry, but I can't typeset math unless \textfont 2")@/ ("and \scriptfont 2 and \scriptscriptfont 2 have all")@/ ("the \fontdimen values needed in math symbol fonts."); error; flush←math; danger:=true; end else if (font←params[fam←fnt(3+text←size)]<total←mathex←params)or@| (font←params[fam←fnt(3+script←size)]<total←mathex←params)or@| (font←params[fam←fnt(3+script←script←size)]<total←mathex←params) then begin print←err("Math formula deleted: Insufficient extension fonts");@/ help3("Sorry, but I can't typeset math unless \textfont 3")@/ ("and \scriptfont 3 and \scriptscriptfont 3 have all")@/ ("the \fontdimen values needed in math extension fonts."); error; flush←math; danger:=true; end @ The |unsave| is done after everything else here; therefore an appearance of `\.{\\mathsurround}' inside of `\.{\$...\$}' affects the spacing at these particular \.\$'s. This is consistent with the conventions of `\.{\$\$...\$\$}', since `\.{\\abovedisplayskip}' inside a display affects the space above that display. @<Finish math in text@>= begin tail←append(new←math(math←surround,before)); cur←mlist:=p; cur←style:=text←style; mlist←penalties:=(mode>0); mlist←to←hlist; link(tail):=link(temp←head); while link(tail)<>null do tail:=link(tail); tail←append(new←math(math←surround,after)); space←factor:=1000; unsave; end @ \TeX\ gets to the following part of the program when the first `\.\$' ending a display has been scanned. @<Check that another \.\$ follows@>= begin get←x←token; if cur←cmd<>math←shift then begin print←err("Display math should end with $$"); @.Display math...with \$\$@> help2("The `$' that I just saw supposedly matches a previous `$$'.")@/ ("So I shall assume that you typed `$$' both times."); back←error; end; end @ We have saved the worst for last: The fussiest part of math mode processing occurs when a displayed formula is being centered and placed with an optional equation number. @<Local variables for finishing...@>= @!b:pointer; {box containing the equation} @!w:scaled; {width of the equation} @!z:scaled; {width of the line} @!e:scaled; {width of equation number} @!q:scaled; {width of equation number plus space to separate from equation} @!d:scaled; {displacement of equation in the line} @!s:scaled; {move the line right this much} @!g1,@!g2:small←number; {glue parameter codes for before and after} @!r:pointer; {kern node used to position the display} @!t:pointer; {tail of adjustment list} @ At this time |p| points to the mlist for the formula; |a| is either |null| or it points to a box containing the equation number; and we are in vertical mode (or internal vertical mode). @<Finish displayed math@>= cur←mlist:=p; cur←style:=display←style; mlist←penalties:=false; mlist←to←hlist; p:=link(temp←head);@/ adjust←tail:=adjust←head; b:=hpack(p,natural); t:=adjust←tail; adjust←tail:=null;@/ w:=width(b); z:=display←width; s:=display←indent; if (a=null)or danger then begin e:=0; q:=0; end else begin e:=width(a); q:=e+math←quad(text←size); end; if w+q>z then @<Squeeze the equation as much as possible; if there is an equation number that should go on a separate line by itself, set~|e:=0|@>; @<Determine the displacement, |d|, of the left edge of the equation, with respect to the line size |z|, assuming that |l=false|@>; @<Append the glue or equation number preceding the display@>; @<Append the display and perhaps also the equation number@>; @<Append the glue or equation number following the display@>; resume←after←display @ @<Declare act...@>= procedure resume←after←display; begin if cur←group<>math←shift←group then confusion("display"); @:this can't happen display}{\quad display@> unsave; prev←graf:=prev←graf+3; push←nest; mode:=hmode; space←factor:=1000; @<Scan an optional space@>; if nest←ptr=1 then build←page; end; @ The user can force the equation number to go on a separate line by making its width zero. @<Squeeze the equation as much as possible...@>= begin if (e<>0)and((w-total←shrink[normal]+q<=z)or@| (total←shrink[fil]<>0)or(total←shrink[fill]<>0)or (total←shrink[filll]<>0)) then begin free←node(b,box←node←size); b:=hpack(p,z-q,exactly); end else begin e:=0; if w>z then begin free←node(b,box←node←size); b:=hpack(p,z,exactly); end; end; w:=width(b); end @ We try first to center the display without regard to the existence of the equation number. If that would make it too close (where ``too close'' means that the space between display and equation number is less than the width of the equation number), we either center it in the remaining space or move it as far from the equation number as possible. The latter alternative is taken only if the display begins with glue, since we assume that the user put glue there to control the spacing precisely. @<Determine the displacement, |d|, of the left edge of the equation...@>= d:=half(z-w); if (e>0)and(d<2*e) then {too close} begin d:=half(z-w-e); if p<>null then if type(p)=glue←node then d:=0; end @ If the equation number is set on a line by itself, either before or after the formula, we append an infinite penalty so that no page break will separate the display from its number; and we use the same size and displacement for all three potential lines of the display, even though `\.{\\parshape}' may specify them differently. @<Append the glue or equation number preceding the display@>= tail←append(new←penalty(pre←display←penalty));@/ if (d+s<=pre←display←size)or l then {not enough clearance} begin g1:=above←display←skip←code; g2:=below←display←skip←code; end else begin g1:=above←display←short←skip←code; g2:=below←display←short←skip←code; end; if l and(e=0) then {it follows that |type(a)=hlist←node|} begin shift←amount(a):=s; append←to←vlist(a); tail←append(new←penalty(inf←penalty)); end else tail←append(new←param←glue(g1)) @ @<Append the display and perhaps also the equation number@>= if e<>0 then begin r:=new←kern(z-w-e-d); if l then begin link(a):=r; link(r):=b; b:=a; d:=0; end else begin link(b):=r; link(r):=a; end; b:=hpack(b,natural); end; shift←amount(b):=s+d; append←to←vlist(b); if t<>adjust←head then begin link(tail):=link(adjust←head); tail:=t; end @ @<Append the glue or equation number following the display@>= if (a<>null)and(e=0)and not l then begin tail←append(new←penalty(inf←penalty)); shift←amount(a):=s+z-width(a); append←to←vlist(a); tail←append(new←penalty(post←display←penalty)); end else begin tail←append(new←penalty(post←display←penalty)); tail←append(new←param←glue(g2)); end @ When \.{\\halign} appears in a display, the alignment routines operate essentially as they do in vertical mode. Then the following program is activated, with |p| and |q| pointing to the beginning and end of the resulting list, and with |t| the |prev←depth| value. @<Finish an alignment in a display@>= begin do←assignments; if cur←cmd<>math←shift then @<Pontificate about improper alignment in display@> else @<Check that another \.\$ follows@>; pop←nest; tail←append(new←penalty(pre←display←penalty)); tail←append(new←param←glue(above←display←skip←code)); link(tail):=p; if p<>null then tail:=q; tail←append(new←penalty(post←display←penalty)); tail←append(new←param←glue(below←display←skip←code)); prev←depth:=t; resume←after←display; end @ @<Pontificate...@>= begin print←err("Missing $$ inserted"); @.Missing {\$\$} inserted@> help2("Displays can use special alignments (like \eqalignno)")@/ ("only if nothing but the alignment itself is between $$'s."); back←error; end @* \[49] Mode-independent processing. The long |main←control| procedure has now been fully specified, except for certain activities that are independent of the current mode. These activities do not change the current vlist or hlist or mlist; if they change anything, it is the value of a parameter or the meaning of a control sequence. Assignments to values in |eqtb| can be global or local. Furthermore, a control sequence can be defined to be `\.{\\long}' or `\.{\\outer}', and it might or might not be expanded. The prefixes `\.{\\global}', `\.{\\long}', and `\.{\\outer}' can occur in any order. Therefore we assign binary numeric codes, making it possible to accumulate the union of all specified prefixes by adding the corresponding codes. (\PASCAL's |set| operations could also have been used.) @<Put each...@>= primitive("long",prefix,1); @!@:long←}{\.{\\long} primitive@> primitive("outer",prefix,2); @!@:outer←}{\.{\\outer} primitive@> primitive("global",prefix,4); @!@:global←}{\.{\\global} primitive@> primitive("def",def,0); @!@:def←}{\.{\\def} primitive@> primitive("gdef",def,1); @!@:gdef←}{\.{\\gdef} primitive@> primitive("edef",def,2); @!@:edef←}{\.{\\edef} primitive@> primitive("xdef",def,3); @!@:xdef←}{\.{\\xdef} primitive@> @ @<Cases of |print←cmd←chr|...@>= prefix: if chr←code=1 then print←esc("long") else if chr←code=2 then print←esc("outer") else print←esc("global"); def: if chr←code=0 then print←esc("def") else if chr←code=1 then print←esc("gdef") else if chr←code=2 then print←esc("edef") else print←esc("xdef"); @ Every prefix, and every command code that might or might not be prefixed, calls the action procedure |prefixed←command|. This routine accumulates a sequence of prefixes until coming to a non-prefix, then it carries out the command. @<Cases of |main←control| that don't...@>= any←mode(toks←register), any←mode(assign←toks), any←mode(assign←int), any←mode(assign←dimen), any←mode(assign←glue), any←mode(assign←mu←glue), any←mode(assign←font←dimen), any←mode(assign←font←int), any←mode(set←aux), any←mode(set←prev←graf), any←mode(set←page←dimen), any←mode(set←page←int), any←mode(set←box←dimen), any←mode(set←shape), any←mode(def←code), any←mode(def←family), any←mode(set←font), any←mode(def←font), any←mode(register), any←mode(advance), any←mode(multiply), any←mode(divide), any←mode(prefix), any←mode(let), any←mode(shorthand←def), any←mode(read←to←cs), any←mode(def), any←mode(set←box), any←mode(hyph←data), any←mode(set←interaction):prefixed←command; @ If the user says, e.g., `\.{\\global\\global}', the redundancy is silently accepted. @<Declare act...@>= @t\4@>@<Declare subprocedures for |prefixed←command|@>@t@>@;@/ procedure prefixed←command; label done,exit; var a:small←number; {accumulated prefix codes so far} @!f:internal←font←number; {identifies a font} @!j:halfword; {index into a \.{\\parshape} specification} @!k:0..font←mem←size; {index into |font←info|} @!p,@!q:pointer; {for temporary short-term use} @!n:integer; {ditto} @!e:boolean; {should a definition be expanded? or was \.{\\let} not done?} begin a:=0; while cur←cmd=prefix do begin if not odd(a div cur←chr) then a:=a+cur←chr; @<Get the next non-blank non-relax...@>; if cur←cmd<=max←non←prefixed←command then @<Discard erroneous prefixes and |return|@>; end; @<Discard the prefixes \.{\\long} and \.{\\outer} if they are irrelevant@>; @<Adjust \(f)for the setting of \.{\\globaldefs}@>; case cur←cmd of @t\4@>@<Assignments@>@; othercases confusion("prefix") @:this can't happen prefix}{\quad prefix@> endcases; done: @<Insert a token saved by \.{\\afterassignment}, if any@>; exit:end; @ @<Discard erroneous...@>= begin print←err("You can't use a prefix with `"); @.You can't use a prefix with x@> print←cmd←chr(cur←cmd,cur←chr); print←char("'"); help1("I'll pretend you didn't say \long or \outer or \global."); back←error; return; end @ @<Discard the prefixes...@>= if (cur←cmd<>def)and(a mod 4<>0) then begin print←err("You can't use `"); print←esc("long"); print("' or `"); print←esc("outer"); print("' with `"); @.You can't use \\long...@> print←cmd←chr(cur←cmd,cur←chr); print←char("'"); help1("I'll pretend you didn't say \long or \outer here."); error; end @ The previous routine does not have to adjust |a| so that |a mod 4=0|, since the following routines test for the \.{\\global} prefix as follows. @d global==(a>=4) @d define(#)==if global then geq←define(#)@+else eq←define(#) @d word←define(#)==if global then geq←word←define(#)@+else eq←word←define(#) @<Adjust \(f)for the setting of \.{\\globaldefs}@>= if global←defs<>0 then if global←defs<0 then begin if global then a:=a-4; end else begin if not global then a:=a+4; end @ When a control sequence is to be defined, by \.{\\def} or \.{\\let} or something similar, the |get←r←token| routine will substitute a special control sequence for a token that is not redefinable. @<Declare subprocedures for |prefixed←command|@>= procedure get←r←token; label restart; begin restart: repeat get←token; until cur←tok<>space←token; if (cur←cs=0)or(cur←cs>frozen←control←sequence) then begin print←err("Missing control sequence inserted"); @.Missing control...@> help5("Please don't say `\def cs{...}', say `\def\cs{...}'.")@/ ("I've inserted an inaccessible control sequence so that your")@/ ("definition will be completed without mixing me up too badly.")@/ ("You can recover graciously from this error, if you're")@/ ("careful; see exercise 27.2 in The TeXbook."); @:TeXbook}{\sl The \TeX book@> if cur←cs=0 then back←input; cur←tok:=cs←token←flag+frozen←protection; ins←error; goto restart; end; end; @ @<Initialize table entries...@>= text(frozen←protection):="inaccessible"; @ Here's an example of the way many of the following routines operate. (Unfortunately, they aren't all as simple as this.) @<Assignments@>= set←font: define(cur←font←loc,data,cur←chr); @ When a |def| command has been scanned, |cur←chr| is odd if the definition is supposed to be global, and |cur←chr>=2| if the definition is supposed to be expanded. @<Assignments@>= def: begin if odd(cur←chr)and not global and(global←defs>=0) then a:=a+4; e:=(cur←chr>=2); get←r←token; p:=cur←cs; q:=scan←toks(true,e); define(p,call+(a mod 4),def←ref); end; @ Both \.{\\let} and \.{\\futurelet} share the command code |let|. @<Put each...@>= primitive("let",let,normal);@/ @!@:let←}{\.{\\let} primitive@> primitive("futurelet",let,normal+1);@/ @!@:future←let←}{\.{\\futurelet} primitive@> @ @<Cases of |print←cmd←chr|...@>= let: if chr←code<>normal then print←esc("futurelet")@+else print←esc("let"); @ @<Assignments@>= let: begin n:=cur←chr; get←r←token; p:=cur←cs; if n=normal then begin repeat get←token; until cur←cmd<>spacer; if cur←tok=other←token+"=" then begin get←token; if cur←cmd=spacer then get←token; end; end else begin get←token; q:=cur←tok; get←token; back←input; cur←tok:=q; back←input; {look ahead, then back up} end; {note that |back←input| doesn't affect |cur←cmd|, |cur←chr|} if cur←cmd>=call then add←token←ref(cur←chr); define(p,cur←cmd,cur←chr); end; @ A \.{\\chardef} creates a control sequence whose |cmd| is |char←given|; a \.{\\mathchardef} creates a control sequence whose |cmd| is |math←given|; and the corresponding |chr| is the character code or math code. A \.{\\countdef} or \.{\\dimendef} or \.{\\skipdef} or \.{\\muskipdef} creates a control sequence whose |cmd| is |assign←int| or \dots\ or |assign←mu←glue|, and the corresponding |chr| is the |eqtb| location of the internal register in question. @d char←def←code=0 {|shorthand←def| for \.{\\chardef}} @d math←char←def←code=1 {|shorthand←def| for \.{\\mathchardef}} @d count←def←code=2 {|shorthand←def| for \.{\\countdef}} @d dimen←def←code=3 {|shorthand←def| for \.{\\dimendef}} @d skip←def←code=4 {|shorthand←def| for \.{\\skipdef}} @d mu←skip←def←code=5 {|shorthand←def| for \.{\\muskipdef}} @d toks←def←code=6 {|shorthand←def| for \.{\\toksdef}} @<Put each...@>= primitive("chardef",shorthand←def,char←def←code);@/ @!@:char←def←}{\.{\\chardef} primitive@> primitive("mathchardef",shorthand←def,math←char←def←code);@/ @!@:math←char←def←}{\.{\\mathchardef} primitive@> primitive("countdef",shorthand←def,count←def←code);@/ @!@:count←def←}{\.{\\countdef} primitive@> primitive("dimendef",shorthand←def,dimen←def←code);@/ @!@:dimen←def←}{\.{\\dimendef} primitive@> primitive("skipdef",shorthand←def,skip←def←code);@/ @!@:skip←def←}{\.{\\skipdef} primitive@> primitive("muskipdef",shorthand←def,mu←skip←def←code);@/ @!@:mu←skip←def←}{\.{\\muskipdef} primitive@> primitive("toksdef",shorthand←def,toks←def←code);@/ @!@:toks←def←}{\.{\\toksdef} primitive@> @ @<Cases of |print←cmd←chr|...@>= shorthand←def: case chr←code of char←def←code: print←esc("chardef"); math←char←def←code: print←esc("mathchardef"); count←def←code: print←esc("countdef"); dimen←def←code: print←esc("dimendef"); skip←def←code: print←esc("skipdef"); mu←skip←def←code: print←esc("muskipdef"); othercases print←esc("toksdef") endcases; char←given: begin print←esc("char"); print←hex(chr←code); end; math←given: begin print←esc("mathchar"); print←hex(chr←code); end; @ @<Assignments@>= shorthand←def: begin n:=cur←chr; get←r←token; p:=cur←cs; define(p,relax,256); scan←optional←equals; case n of char←def←code: begin scan←char←num; define(p,char←given,cur←val); end; math←char←def←code: begin scan←fifteen←bit←int; define(p,math←given,cur←val); end; othercases begin scan←eight←bit←int; case n of char←def←code: define(p,char←given,cur←val); math←char←def←code: define(p,math←given,cur←val); count←def←code: define(p,assign←int,count←base+cur←val); dimen←def←code: define(p,assign←dimen,scaled←base+cur←val); skip←def←code: define(p,assign←glue,skip←base+cur←val); mu←skip←def←code: define(p,assign←mu←glue,mu←skip←base+cur←val); toks←def←code: define(p,assign←toks,toks←base+cur←val); end; {there are no other cases} end endcases; end; @ @<Assignments@>= read←to←cs: begin scan←int; n:=cur←val; if not scan←keyword("to") then begin print←err("Missing `to' inserted"); @.Missing `to'...@> help2("You should have said `\read<number> to \cs'.")@/ ("I'm going to look for the \cs now."); error; end; get←r←token; p:=cur←cs; read←toks(n,p); define(p,call,cur←val); end; @ The token-list parameters, \.{\\output} and \.{\\everypar}, etc., receive their values in the following way. (For safety's sake, we place an enclosing pair of braces around an \.{\\output} list.) @<Assignments@>= toks←register,assign←toks: begin q:=cur←cs; if cur←cmd=toks←register then begin scan←eight←bit←int; p:=toks←base+cur←val; end else p:=cur←chr; {|p=every←par←loc| or |output←routine←loc| or \dots} scan←optional←equals; @<Get the next non-blank non-relax non-call token@>; if cur←cmd<>left←brace then @<If the right-hand side is a token parameter or token register, finish the assignment and |goto done|@>; back←input; cur←cs:=q; q:=scan←toks(false,false); if link(def←ref)=null then {empty list: revert to the default} begin define(p,undefined←cs,null); free←avail(def←ref); end else begin if p=output←routine←loc then {enclose in curlies} begin link(q):=get←avail; q:=link(q); info(q):=right←brace←token+"}"; q:=get←avail; info(q):=left←brace←token+"{"; link(q):=link(def←ref); link(def←ref):=q; end; define(p,call,def←ref); end; end; @ @<If the right-hand side is a token parameter...@>= begin if cur←cmd=toks←register then begin scan←eight←bit←int; cur←cmd:=assign←toks; cur←chr:=toks←base+cur←val; end; if cur←cmd=assign←toks then begin q:=equiv(cur←chr); if q=null then define(p,undefined←cs,null) else begin add←token←ref(q); define(p,call,q); end; goto done; end; end @ Similar routines are used to assign values to the numeric parameters. @<Assignments@>= assign←int: begin p:=cur←chr; scan←optional←equals; scan←int; word←define(p,cur←val); end; assign←dimen: begin p:=cur←chr; scan←optional←equals; scan←normal←dimen; word←define(p,cur←val); end; assign←glue,assign←mu←glue: begin p:=cur←chr; n:=cur←cmd; scan←optional←equals; if n=assign←mu←glue then scan←glue(mu←val)@+else scan←glue(glue←val); trap←zero←glue; define(p,glue←ref,cur←val); end; @ When a glue register or parameter becomes zero, it will always point to |zero←glue| because of the following procedure. @<Declare subprocedures for |prefixed←command|@>= procedure trap←zero←glue; begin if (width(cur←val)=0)and(stretch(cur←val)=0)and(shrink(cur←val)=0) then begin add←glue←ref(zero←glue); delete←glue←ref(cur←val); cur←val:=zero←glue; end; end; @ The various character code tables are changed by the |def←code| commands, and the font families are declared by |def←family|. @<Put each...@>= primitive("catcode",def←code,cat←code←base); @!@:cat←code←}{\.{\\catcode} primitive@> primitive("mathcode",def←code,math←code←base); @!@:math←code←}{\.{\\mathcode} primitive@> primitive("lccode",def←code,lc←code←base); @!@:lc←code←}{\.{\\lccode} primitive@> primitive("uccode",def←code,uc←code←base); @!@:uc←code←}{\.{\\uccode} primitive@> primitive("sfcode",def←code,sf←code←base); @!@:sf←code←}{\.{\\sfcode} primitive@> primitive("delcode",def←code,del←code←base); @!@:del←code←}{\.{\\delcode} primitive@> primitive("textfont",def←family,math←font←base); @!@:text←font←}{\.{\\textfont} primitive@> primitive("scriptfont",def←family,math←font←base+script←size); @!@:script←font←}{\.{\\scriptfont} primitive@> primitive("scriptscriptfont",def←family,math←font←base+script←script←size); @!@:script←script←font←}{\.{\\scriptscriptfont} primitive@> @ @<Cases of |print←cmd←chr|...@>= def←code: if chr←code=cat←code←base then print←esc("catcode") else if chr←code=math←code←base then print←esc("mathcode") else if chr←code=lc←code←base then print←esc("lccode") else if chr←code=uc←code←base then print←esc("uccode") else if chr←code=sf←code←base then print←esc("sfcode") else print←esc("delcode"); def←family: print←size(chr←code-math←font←base); @ The different types of code values have different legal ranges; the following program is careful to check each case properly. @<Assignments@>= def←code: begin @<Let |n| be the largest legal code value, based on |cur←chr|@>; p:=cur←chr; scan←seven←bit←int; p:=p+cur←val; scan←optional←equals; scan←int; if ((cur←val<0)and(p<del←code←base))or(cur←val>n) then begin print←err("Invalid code ("); print←int(cur←val); @.Invalid code@> if p<del←code←base then print("), should be in the range 0..") else print("), should be at most "); print←int(n); help1("I'm going to use 0 instead of that illegal code value.");@/ error; cur←val:=0; end; if p<math←code←base then define(p,data,cur←val) else if p<del←code←base then define(p,data,hi(cur←val)) else word←define(p,cur←val); end; @ @<Let |n| be the largest...@>= if cur←chr=cat←code←base then n:=max←char←code else if cur←chr=math←code←base then n:=@'100000 else if cur←chr=sf←code←base then n:=@'77777 else if cur←chr=del←code←base then n:=@'77777777 else n:=127 @ @<Assignments@>= def←family: begin p:=cur←chr; scan←four←bit←int; p:=p+cur←val; scan←optional←equals; scan←font←ident; define(p,data,cur←val); end; @ Next we consider changes to \TeX's numeric registers. @<Assignments@>= register,advance,multiply,divide: do←register←command(a); @ We use the fact that |register<advance<multiply<divide|. @<Declare subprocedures for |prefixed←command|@>= procedure do←register←command(@!a:small←number); label found,exit; var l,@!q,@!r,@!s:pointer; {for list manipulation} @!p:int←val..mu←val; {type of register involved} begin q:=cur←cmd; @<Compute the register location |l| and its type |p|; but |return| if invalid@>; if q=register then scan←optional←equals else if scan←keyword("by") then do←nothing; {optional `\.{by}'} arith←error:=false; if q<multiply then @<Compute result of |register| or |advance|, put it in |cur←val|@> else @<Compute result of |multiply| or |divide|, put it in |cur←val|@>; if arith←error then begin print←err("Arithmetic overflow"); @.Arithmetic overflow@> help2("I can't carry out that multiplication or division,")@/ ("since the result is out of range."); error; return; end; if p<glue←val then word←define(l,cur←val) else begin trap←zero←glue; define(l,glue←ref,cur←val); end; exit: end; @ Here we use the fact that the consecutive codes |int←val...mu←val| and |assign←int..assign←mu←glue| correspond to each other nicely. @<Compute the register location |l| and its type |p|...@>= begin if q<>register then begin get←x←token; if (cur←cmd>=assign←int)and(cur←cmd<=assign←mu←glue) then begin l:=cur←chr; p:=cur←cmd-assign←int; goto found; end; if cur←cmd<>register then begin print←err("You can't use `"); print←cmd←chr(cur←cmd,cur←chr); @.You can't use x after ...@> print("' after "); print←cmd←chr(q,0); help1("I'm forgetting what you said and not changing anything."); error; return; end; end; p:=cur←chr; scan←eight←bit←int; case p of int←val: l:=cur←val+count←base; dimen←val: l:=cur←val+scaled←base; glue←val: l:=cur←val+skip←base; mu←val: l:=cur←val+mu←skip←base; end; {there are no other cases} end; found: @ @<Compute result of |register| or |advance|...@>= if p<glue←val then begin if p=int←val then scan←int@+else scan←normal←dimen; if q=advance then cur←val:=cur←val+eqtb[l].int; end else begin scan←glue(p); if q=advance then @<Compute the sum of two glue specs@>; end @ @<Compute the sum of two glue specs@>= begin q:=new←spec(cur←val); r:=equiv(l); delete←glue←ref(cur←val); width(q):=width(q)+width(r); if stretch(q)=0 then stretch←order(q):=normal; if stretch←order(q)=stretch←order(r) then stretch(q):=stretch(q)+stretch(r) else if (stretch←order(q)<stretch←order(r))and(stretch(r)<>0) then begin stretch(q):=stretch(r); stretch←order(q):=stretch←order(r); end; if shrink(q)=0 then shrink←order(q):=normal; if shrink←order(q)=shrink←order(r) then shrink(q):=shrink(q)+shrink(r) else if (shrink←order(q)<shrink←order(r))and(shrink(r)<>0) then begin shrink(q):=shrink(r); shrink←order(q):=shrink←order(r); end; cur←val:=q; end @ @<Compute result of |multiply| or |divide|...@>= begin scan←int; if p<glue←val then if q=multiply then cur←val:=nx←plus←y(eqtb[l].int,cur←val,0) else cur←val:=x←over←n(eqtb[l].int,cur←val) else begin s:=equiv(l); r:=new←spec(s); if q=multiply then begin width(r):=nx←plus←y(width(s),cur←val,0); stretch(r):=nx←plus←y(stretch(s),cur←val,0); shrink(r):=nx←plus←y(shrink(s),cur←val,0); end else begin width(r):=x←over←n(width(s),cur←val); stretch(r):=x←over←n(stretch(s),cur←val); shrink(r):=x←over←n(shrink(s),cur←val); end; cur←val:=r; end; end @ The processing of boxes is somewhat different, because it may be necessary to scan and create an entire box before we actually change the value of the old one. @<Assignments@>= set←box: begin scan←eight←bit←int; if global then saved(0):=box←flag+256+cur←val else saved(0):=box←flag+cur←val; scan←optional←equals; scan←box; end; @ The |space←factor| or |prev←depth| settings are changed when a |set←aux| command is sensed. Similarly, |prev←graf| is changed in the presence of |set←prev←graf|, and |dead←cycles| or |insert←penalties| in the presence of |set←page←int|. These definitions are always global. When some dimension of a box register is changed, the change isn't exactly global; but \TeX\ does not look at the \.{\\global} switch. @<Assignments@>= set←aux:alter←aux; set←prev←graf:alter←prev←graf; set←page←dimen:alter←page←so←far; set←page←int:alter←integer; set←box←dimen:alter←box←dimen; @ @<Declare subprocedures for |prefixed←command|@>= procedure alter←aux; var c:halfword; {|hmode| or |vmode|} begin if cur←chr<>abs(mode) then report←illegal←case else begin c:=cur←chr; scan←optional←equals; if c=vmode then begin scan←normal←dimen; prev←depth:=cur←val; end else begin scan←int; if (cur←val<=0)or(cur←val>32767) then begin print←err("Bad space factor"); @.Bad space factor@> help1("I allow only values in the range 1..32767 here."); int←error(cur←val); end else space←factor:=cur←val; end; end; end; @ @<Declare subprocedures for |prefixed←command|@>= procedure alter←prev←graf; var p:0..nest←size; {index into |nest|} begin nest[nest←ptr]:=cur←list; p:=nest←ptr; while abs(nest[p].mode←field)<>vmode do decr(p); scan←optional←equals; scan←int; if cur←val<0 then begin print←err("Bad "); print←esc("prevgraf"); @.Bad \\prevgraf@> help1("I allow only nonnegative values here."); int←error(cur←val); end else begin nest[p].pg←field:=cur←val; cur←list:=nest[nest←ptr]; end; end; @ @<Declare subprocedures for |prefixed←command|@>= procedure alter←page←so←far; var c:0..7; {index into |page←so←far|} begin c:=cur←chr; scan←optional←equals; scan←normal←dimen; page←so←far[c]:=cur←val; end; @ @<Declare subprocedures for |prefixed←command|@>= procedure alter←integer; var c:0..1; {0 for \.{\\deadcycles}, 1 for \.{\\insertpenalties}} begin c:=cur←chr; scan←optional←equals; scan←int; if c=0 then dead←cycles:=cur←val else insert←penalties:=cur←val; end; @ @<Declare subprocedures for |prefixed←command|@>= procedure alter←box←dimen; var c:small←number; {|width←offset| or |height←offset| or |depth←offset|} @!b:eight←bits; {box number} begin c:=cur←chr; scan←eight←bit←int; b:=cur←val; scan←optional←equals; scan←normal←dimen; if box(b)<>null then mem[box(b)+c].sc:=cur←val; end; @ Paragraph shapes are set up in the obvious way. @<Assignments@>= set←shape: begin scan←optional←equals; scan←int; n:=cur←val; if n<=0 then p:=null else begin p:=get←node(2*n+1); info(p):=n; for j:=1 to n do begin scan←normal←dimen; mem[p+2*j-1].sc:=cur←val; {indentation} scan←normal←dimen; mem[p+2*j].sc:=cur←val; {width} end; end; define(par←shape←loc,shape←ref,p); end; @ Here's something that isn't quite so obvious. It guarantees that |info(par←shape←ptr)| can hold any positive~|n| such |get←node(2*n+1)| doesn't overflow the memory capacity. @<Check the ``constant''...@>= if 2*max←halfword<mem←top-mem←min then bad:=41; @ New hyphenation data is loaded by the |hyph←data| command. @<Put each...@>= primitive("hyphenation",hyph←data,0); @!@:hyphenation←}{\.{\\hyphenation} primitive@> primitive("patterns",hyph←data,1); @!@:patterns←}{\.{\\patterns} primitive@> @ @<Cases of |print←cmd←chr|...@>= hyph←data: if chr←code=1 then print←esc("patterns") else print←esc("hyphenation"); @ @<Assignments@>= hyph←data: if cur←chr=1 then begin @!init new←patterns; goto done;@;@+tini@/ print←err("Patterns can be loaded only by INITEX"); @.Patterns can be...@> help0; error; repeat get←token; until cur←cmd=right←brace; {flush the patterns} return; end else begin new←hyph←exceptions; goto done; end; @ All of \TeX's parameters are kept in |eqtb| except the font information, the interaction mode, and the hyphenation tables; these are strictly global. @<Assignments@>= assign←font←dimen: begin find←font←dimen(true); k:=cur←val; scan←optional←equals; scan←normal←dimen; font←info[k].sc:=cur←val; end; assign←font←int: begin n:=cur←chr; scan←font←ident; f:=cur←val; scan←optional←equals; scan←int; if n=0 then hyphen←char[f]:=cur←val@+else skew←char[f]:=cur←val; end; @ @<Put each...@>= primitive("hyphenchar",assign←font←int,0); @!@:hyphen←char←}{\.{\\hyphenchar} primitive@> primitive("skewchar",assign←font←int,1); @!@:skew←char←}{\.{\\skewchar} primitive@> @ @<Cases of |print←cmd←chr|...@>= assign←font←int: if chr←code=0 then print←esc("hyphenchar") else print←esc("skewchar"); @ Here is where the information for a new font gets loaded. @<Assignments@>= def←font: new←font(a); @ @<Declare subprocedures for |prefixed←command|@>= procedure new←font(@!a:small←number); label common←ending; var u:pointer; {user's font identifier} @!s:scaled; {stated ``at'' size, or negative of scaled magnification} @!f:internal←font←number; {runs through existing fonts} @!t:str←number; {name for the frozen font identifier} @!old←setting:0..max←selector; {holds |selector| setting} begin if job←name=0 then open←log←file; {avoid confusing \.{texput} with the font name} @.texput@> get←r←token; u:=cur←cs; if u>=hash←base then t:=text(u) else if u>=single←base then if u=null←cs then t:="FONT"@+else t:=u-single←base else begin old←setting:=selector; selector:=new←string; print("FONT"); print(u-active←base); selector:=old←setting; @.FONTx@> str←room(1); t:=make←string; end; define(u,set←font,null←font); scan←optional←equals; scan←file←name; @<Scan the font size specification@>; @<If this font has already been loaded, set |f| to the internal font number and |goto common←ending|@>; f:=read←font←info(u,cur←name,cur←area,s); common←ending: equiv(u):=f; eqtb[font←id←base+f]:=eqtb[u]; font←id←text(f):=t; end; @ @<Scan the font size specification@>= name←in←progress:=true; {this keeps |cur←name| from being changed} if scan←keyword("at") then @<Put the (positive) `at' size into |s|@> @.at@> else if scan←keyword("scaled") then @.scaled@> begin scan←int; s:=-cur←val; if (cur←val<=0)or(cur←val>32768) then begin print←err("Illegal magnification has been changed to 1000");@/ @.Illegal magnification...@> help1("The magnification ratio must be between 1 and 32768."); int←error(cur←val); s:=-1000; end; end else s:=-1000; name←in←progress:=false @ @<Put the (positive) `at' size into |s|@>= begin scan←normal←dimen; s:=cur←val; if (s<=0)or(s>=@'1000000000) then begin print←err("Improper `at' size ("); print←scaled(s); print("pt), replaced by 10pt"); @.Improper `at' size...@> help2("I can only handle fonts at positive sizes that are")@/ ("less than 2048pt, so I've changed what you said to 10pt."); error; s:=10*unity; end; end @ When the user gives a new identifier to a font that was previously loaded, the new name becomes the font identifier of record. Font names `\.{xyz}' and `\.{XYZ}' are considered to be different. @<If this font has already been loaded...@>= for f:=font←base+1 to font←ptr do if str←eq←str(font←name[f],cur←name)and str←eq←str(font←area[f],cur←area) then begin if s>0 then begin if s=font←size[f] then goto common←ending; end else if font←size[f]=xn←over←d(font←dsize[f],-s,1000) then goto common←ending; end @ @<Cases of |print←cmd←chr|...@>= set←font:begin print("select font "); print(font←name[chr←code]); if font←size[chr←code]<>font←dsize[chr←code] then begin print(" at "); print←scaled(font←size[chr←code]); print("pt"); end; end; @ @<Put each...@>= primitive("batchmode",set←interaction,batch←mode); @!@:batch←mode←}{\.{\\batchmode} primitive@> primitive("nonstopmode",set←interaction,nonstop←mode); @!@:nonstop←mode←}{\.{\\nonstopmode} primitive@> primitive("scrollmode",set←interaction,scroll←mode); @!@:scroll←mode←}{\.{\\scrollmode} primitive@> primitive("errorstopmode",set←interaction,error←stop←mode); @!@:error←stop←mode←}{\.{\\errorstopmode} primitive@> @ @<Cases of |print←cmd←chr|...@>= set←interaction: case chr←code of batch←mode: print←esc("batchmode"); nonstop←mode: print←esc("nonstopmode"); scroll←mode: print←esc("scrollmode"); othercases print←esc("errorstopmode") endcases; @ @<Assignments@>= set←interaction: new←interaction; @ @<Declare subprocedures for |prefixed←command|@>= procedure new←interaction; begin print←ln; interaction:=cur←chr; @<Initialize the print |selector| based on |interaction|@>; if job←name<>0 then selector:=selector+2; end; @ The \.{\\afterassignment} command puts a token into the global variable |after←token|. This global variable is examined just after every assignment has been performed. @<Glob...@>= @!after←token:halfword; {zero, or a saved token} @ @<Set init...@>= after←token:=0; @ @<Cases of |main←control| that don't...@>= any←mode(after←assignment):begin get←token; after←token:=cur←tok; end; @ @<Insert a token saved by \.{\\afterassignment}, if any@>= if after←token<>0 then begin cur←tok:=after←token; back←input; after←token:=0; end @ Here is a procedure that might be called `Get the next non-blank non-relax non-call non-assignment token'. @<Declare act...@>= procedure do←assignments; label exit; begin loop begin @<Get the next non-blank non-relax...@>; if cur←cmd<=max←non←prefixed←command then return; prefixed←command; end; exit:end; @ @<Cases of |main←control| that don't...@>= any←mode(after←group):begin get←token; save←for←after(cur←tok); end; @ Files for \.{\\read} are opened and closed by the |in←stream| command. @<Put each...@>= primitive("openin",in←stream,1); @!@:open←in←}{\.{\\openin} primitive@> primitive("closein",in←stream,0); @!@:close←in←}{\.{\\closein} primitive@> @ @<Cases of |print←cmd←chr|...@>= in←stream: if chr←code=0 then print←esc("closein") else print←esc("openin"); @ @<Cases of |main←control| that don't...@>= any←mode(in←stream): open←or←close←in; @ @<Declare act...@>= procedure open←or←close←in; var c:0..1; {1 for \.{\\openin}, 0 for \.{\\closein}} @!n:0..15; {stream number} begin c:=cur←chr; scan←four←bit←int; n:=cur←val; if read←open[n]<>closed then begin a←close(read←file[n]); read←open[n]:=closed; end; if c<>0 then begin scan←optional←equals; scan←file←name; if cur←ext="" then cur←ext:=".tex"; pack←cur←name; if a←open←in(read←file[n]) then read←open[n]:=just←open; end; end; @ The user can issue messages to the terminal, regardless of the current mode. @<Cases of |main←control| that don't...@>= any←mode(message):issue←message; @ @<Put each...@>= primitive("message",message,0); @!@:message←}{\.{\\message} primitive@> primitive("errmessage",message,1); @!@:err←message←}{\.{\\errmessage} primitive@> @ @<Cases of |print←cmd←chr|...@>= message: if chr←code=0 then print←esc("message") else print←esc("errmessage"); @ @<Declare act...@>= procedure issue←message; var old←setting:0..max←selector; {holds |selector| setting} @!c:0..1; {identifies \.{\\message} and \.{\\errmessage}} @!s:str←number; {the message} begin c:=cur←chr; link(garbage):=scan←toks(false,true); old←setting:=selector; selector:=new←string; token←show(def←ref); selector:=old←setting; flush←list(def←ref); str←room(1); s:=make←string; if c=0 then @<Print string |s| on the terminal@> else @<Print string |s| as an error message@>; flush←string; end; @ @<Print string |s| on the terminal@>= begin if term←offset+length(s)>max←print←line-2 then print←ln else if (term←offset>0)or(file←offset>0) then print←char(" "); print(s); update←terminal; end @ If \.{\\errmessage} occurs often in |scroll←mode|, without user-defined \.{\\errhelp}, we don't want to give a long help message each time. So we give a verbose explanation only once. @<Glob...@>= @!long←help←seen:boolean; {has the long \.{\\errmessage} help been used?} @ @<Set init...@>=long←help←seen:=false; @ @<Print string |s| as an error message@>= begin print←err(s); if err←help<>null then use←err←help:=true else if long←help←seen then help1("(That was another \errmessage.)") else begin if interaction<error←stop←mode then long←help←seen:=true; help4("This error message was generated by an \errmessage")@/ ("command, so I can't give any explicit help.")@/ ("Pretend that you're Hercule Poirot, examine all clues,")@/ @↑Poirot, Hercule@> ("and deduce the truth by order and method."); end; error; use←err←help:=false; end @ The |error| routine calls on |give←err←help| if help is requested from the |err←help| parameter. @p procedure give←err←help; begin token←show(err←help); end; @ The \.{\\uppercase} and \.{\\lowercase} commands are implemented by building a token list and then changing the cases of the letters in it. @<Cases of |main←control| that don't...@>= any←mode(case←shift):shift←case; @ @<Put each...@>= primitive("lowercase",case←shift,lc←code←base); @!@:lowercase←}{\.{\\lowercase} primitive@> primitive("uppercase",case←shift,uc←code←base); @!@:uppercase←}{\.{\\uppercase} primitive@> @ @<Cases of |print←cmd←chr|...@>= case←shift:if chr←code=lc←code←base then print←esc("lowercase") else print←esc("uppercase"); @ @<Declare act...@>= procedure shift←case; var b:pointer; {|lc←code←base| or |uc←code←base|} @!p:pointer; {runs through the token list} @!t:halfword; {token} @!c:eight←bits; {character code} begin b:=cur←chr; p:=scan←toks(false,false); p:=link(def←ref); while p<>null do begin @<Change the case of the token in |p|, if a change is appropriate@>; p:=link(p); end; back←list(link(def←ref)); free←avail(def←ref); {omit reference count} end; @ When the case of a |chr←code| changes, we don't change the |cmd|. We also change active characters, using the fact that |cs←token←flag| is a multiple of~256. @<Change the case of the token in |p|, if a change is appropriate@>= t:=info(p); if t<cs←token←flag+single←base then begin if t>=cs←token←flag then t:=t-active←base; c:=t mod 256; if c<128 then if equiv(b+c)<>0 then t:=256*(t div 256)+equiv(b+c); if t>=cs←token←flag then info(p):=t+active←base else info(p):=t; end @ We come finally to the last pieces missing from |main←control|, namely the `\.{\\show}' commands that are useful when debugging. @<Cases of |main←control| that don't...@>= any←mode(xray): show←whatever; @ @d show←code=0 { \.{\\show} } @d show←box←code=1 { \.{\\showbox} } @d show←the←code=2 { \.{\\showthe} } @d show←lists=3 { \.{\\showlists} } @<Put each...@>= primitive("show",xray,show←code); @!@:show←}{\.{\\show} primitive@> primitive("showbox",xray,show←box←code); @!@:show←box←}{\.{\\showbox} primitive@> primitive("showthe",xray,show←the←code); @!@:show←the←}{\.{\\showthe} primitive@> primitive("showlists",xray,show←lists); @!@:show←lists←}{\.{\\showlists} primitive@> @ @<Cases of |print←cmd←chr|...@>= xray: case chr←code of show←box←code:print←esc("showbox"); show←the←code:print←esc("showthe"); show←lists:print←esc("showlists"); othercases print←esc("show") endcases; @ @<Declare act...@>= procedure show←whatever; label common←ending; var p:pointer; {tail of a token list to show} begin case cur←chr of show←lists: begin begin←diagnostic; show←activities; end; show←box←code: @<Show the current contents of a box@>; show←code: @<Show the current meaning of a token, then |goto common←ending|@>; othercases @<Show the current value of some parameter or register, then |goto common←ending|@> endcases;@/ @<Complete a potentially long \.{\\show} command@>; common←ending: if interaction<error←stop←mode then begin help0; decr(error←count); end else if tracing←online>0 then begin@t@>@;@/ help3("This isn't an error message; I'm just \showing something.")@/ ("Type `I\show...' to show more (e.g., \show\cs,")@/ ("\showthe\count10, \showbox255, \showlists)."); end else begin@t@>@;@/ help5("This isn't an error message; I'm just \showing something.")@/ ("Type `I\show...' to show more (e.g., \show\cs,")@/ ("\showthe\count10, \showbox255, \showlists).")@/ ("And type `I\tracingonline=1\show...' to show boxes and")@/ ("lists on your terminal as well as in the transcript file."); end; error; end; @ @<Show the current meaning of a token...@>= begin get←token; print←nl("> "); if cur←cs<>0 then begin sprint←cs(cur←cs); print←char("="); end; print←meaning; goto common←ending; end @ @<Cases of |print←cmd←chr|...@>= undefined←cs: print("undefined"); call: print("macro"); long←call: print←esc("long macro"); outer←call: print←esc("outer macro"); long←outer←call: begin print←esc("long"); print←esc("outer macro"); end; end←template: print←esc("outer endtemplate"); @ @<Show the current contents of a box@>= begin scan←eight←bit←int; begin←diagnostic; print←nl("> \box"); print←int(cur←val); print←char("="); if box(cur←val)=null then print("void") else show←box(box(cur←val)); end @ @<Show the current value of some parameter...@>= begin p:=the←toks; print←nl("> "); token←show(temp←head); flush←list(link(temp←head)); goto common←ending; end @ @<Complete a potentially long \.{\\show} command@>= end←diagnostic(true); print←err("OK"); @.OK@> if selector=term←and←log then if tracing←online<=0 then begin selector:=term←only; print(" (see the transcript file)"); selector:=term←and←log; end @* \[50] Dumping and undumping the tables. After \.{INITEX} has seen a collection of fonts and macros, it can write all the necessary information on an auxiliary file so that production versions of \TeX\ are able to initialize their memory at high speed. The present section of the program takes care of such output and input. We shall consider simultaneously the processes of storing and restoring, so that the inverse relation between them is clear. @.INITEX@> The global variable |format←ident| is a string that is printed right after the |banner| line when \TeX\ is ready to start. For \.{INITEX} this string says simply `\.{(INITEX)}'; for other versions of \TeX\ it says, for example, `\.{(preloaded format=PLAIN 82.11.19)}', showing the year, month, and day that the format file was created. We have |format←ident=0| before \TeX's tables are loaded. @<Glob...@>= @!format←ident:str←number; @ @<Set init...@>= format←ident:=0; @ @<Initialize table entries...@>= format←ident:=" (INITEX)"; @ @<Declare act...@>= @!init procedure store←fmt←file; label found1,found2,done1,done2; var j,@!k,@!l:integer; {all-purpose indices} @!p,@!q: pointer; {all-purpose pointers} @!x: integer; {something to dump} @!w: four←quarters; {four ASCII codes} begin @<If dumping is not allowed, abort@>; @<Create the |format←ident|, open the format file, and inform the user that dumping has begun@>; @<Dump constants for consistency check@>; @<Dump the string pool@>; @<Dump the dynamic memory@>; @<Dump the table of equivalents@>; @<Dump the font information@>; @<Dump the hyphenation tables@>; @<Dump a couple more things and the closing check word@>; @<Close the format file@>; end; tini @ Corresponding to the procedure that dumps a format file, we have a function that reads one in. The function returns |false| if the dumped format is incompatible with the present \TeX\ table sizes, etc. @d bad←fmt=6666 {go here if the format file is unacceptable} @d too←small(#)==begin wake←up←terminal; wterm←ln('---! Must increase the ',#); @.Must increase the x@> goto bad←fmt; end @p @t\4@>@<Declare the function called |open←fmt←file|@>@; function load←fmt←file:boolean; label bad←fmt,exit; var j,@!k:integer; {all-purpose indices} @!p,@!q: pointer; {all-purpose pointers} @!x: integer; {something undumped} @!w: four←quarters; {four ASCII codes} begin @<Undump constants for consistency check@>; @<Undump the string pool@>; @<Undump the dynamic memory@>; @<Undump the table of equivalents@>; @<Undump the font information@>; @<Undump the hyphenation tables@>; @<Undump a couple more things and the closing check word@>; load←fmt←file:=true; return; {it worked!} bad←fmt: wake←up←terminal; wterm←ln('(Fatal format file error; I''m stymied)'); @.Fatal format file error@> load←fmt←file:=false; exit:end; @ The user is not allowed to dump a format file unless |save←ptr=0|. This condition implies that |cur←level=level←one|, hence the |xeq←level| array is constant and it need not be dumped. @<If dumping is not allowed, abort@>= if save←ptr<>0 then begin print←err("You can't dump inside a group"); @.You can't dump...@> help1("`{...\dump}' is a no-no."); succumb; end @ Format files consist of |memory←word| items, and we use the following macros to dump words of different types: @d dump←wd(#)==begin fmt←file↑:=#; put(fmt←file);@+end @d dump←int(#)==begin fmt←file↑.int:=#; put(fmt←file);@+end @d dump←hh(#)==begin fmt←file↑.hh:=#; put(fmt←file);@+end @d dump←qqqq(#)==begin fmt←file↑.qqqq:=#; put(fmt←file);@+end @<Glob...@>= @!fmt←file:word←file; {for input or output of format information} @ The inverse macros are slightly more complicated, since we need to check the range of the values we are reading in. We say `|undump(a)(b)(x)|' to read an integer value |x| that is supposed to be in the range |a<=x<=b|. @d undump←wd(#)==begin get(fmt←file); #:=fmt←file↑;@+end @d undump←int(#)==begin get(fmt←file); #:=fmt←file↑.int;@+end @d undump←hh(#)==begin get(fmt←file); #:=fmt←file↑.hh;@+end @d undump←qqqq(#)==begin get(fmt←file); #:=fmt←file↑.qqqq;@+end @d undump←end←end(#)==#:=x;@+end @d undump←end(#)==(x>#) then goto bad←fmt@+else undump←end←end @d undump(#)==begin undump←int(x); if (x<#) or undump←end @d undump←size←end←end(#)==too←small(#)@+else undump←end←end @d undump←size←end(#)==if x># then undump←size←end←end @d undump←size(#)==begin undump←int(x); if x<# then goto bad←fmt; undump←size←end @ The next few sections of the program should make it clear how we use the dump/undump macros. @<Dump constants for consistency check@>= dump←int(@$);@/ dump←int(mem←bot);@/ dump←int(mem←top);@/ dump←int(eqtb←size);@/ dump←int(hash←prime);@/ dump←int(hyph←size) @ Sections of a \.{WEB} program that are ``commented out'' still contribute strings to the string pool; therefore \.{INITEX} and \TeX\ will have the same strings. (And it is, of course, a good thing that they do.) @.WEB@> @↑string pool@> @<Undump constants for consistency check@>= x:=fmt←file↑.int; if x<>@$ then goto bad←fmt; {check that strings are the same} undump←int(x); if x<>mem←bot then goto bad←fmt; undump←int(x); if x<>mem←top then goto bad←fmt; undump←int(x); if x<>eqtb←size then goto bad←fmt; undump←int(x); if x<>hash←prime then goto bad←fmt; undump←int(x); if x<>hyph←size then goto bad←fmt @ @d dump←four←ASCII== w.b0:=str←pool[k]; w.b1:=str←pool[k+1]; w.b2:=str←pool[k+2]; w.b3:=str←pool[k+3]; dump←qqqq(w) @<Dump the string pool@>= dump←int(pool←ptr); dump←int(str←ptr); for k:=0 to str←ptr do dump←int(str←start[k]); k:=0; while k+4<pool←ptr do begin dump←four←ASCII; k:=k+4; end; k:=pool←ptr-4; dump←four←ASCII; print←ln; print←int(str←ptr); print(" strings of total length "); print←int(pool←ptr) @ @d undump←four←ASCII== undump←qqqq(w); str←pool[k]:=w.b0; str←pool[k+1]:=w.b1; str←pool[k+2]:=w.b2; str←pool[k+3]:=w.b3 @<Undump the string pool@>= undump←size(0)(pool←size)('string pool size')(pool←ptr); undump←size(0)(max←strings)('max strings')(str←ptr); for k:=0 to str←ptr do undump(0)(pool←ptr)(str←start[k]); k:=0; while k+4<pool←ptr do begin undump←four←ASCII; k:=k+4; end; k:=pool←ptr-4; undump←four←ASCII @ By sorting the list of available spaces in the variable-size portion of |mem|, we are usually able to get by without having to dump very much of the dynamic memory. We recompute |var←used| and |dyn←used|, so that \.{INITEX} dumps valid information even when it has not been gathering statistics. @<Dump the dynamic memory@>= sort←avail; var←used:=0; dump←int(lo←mem←max); dump←int(rover); p:=mem←bot; q:=rover; x:=0; repeat for k:=p to q+1 do dump←wd(mem[k]); x:=x+q+2-p; var←used:=var←used+q-p; p:=q+node←size(q); q:=rlink(q); until q=rover; var←used:=var←used+lo←mem←max-p; dyn←used:=mem←end+1-hi←mem←min;@/ for k:=p to lo←mem←max do dump←wd(mem[k]); x:=x+lo←mem←max+1-p; dump←int(hi←mem←min); dump←int(avail); for k:=hi←mem←min to mem←end do dump←wd(mem[k]); x:=x+mem←end+1-hi←mem←min; p:=avail; while p<>null do begin decr(dyn←used); p:=link(p); end; dump←int(var←used); dump←int(dyn←used); print←ln; print←int(x); print(" memory locations dumped; current usage is "); print←int(var←used); print←char("&"); print←int(dyn←used) @ @<Undump the dynamic memory@>= undump(lo←mem←stat←max+1000)(hi←mem←stat←min-1)(lo←mem←max); undump(lo←mem←stat←max+1)(lo←mem←max)(rover); p:=mem←bot; q:=rover; x:=0; repeat for k:=p to q+1 do undump←wd(mem[k]); p:=q+node←size(q); if (p>lo←mem←max)or((q>=rlink(q))and(rlink(q)<>rover)) then goto bad←fmt; q:=rlink(q); until q=rover; for k:=p to lo←mem←max do undump←wd(mem[k]); if mem←min<mem←bot-2 then {make more low memory available} begin p:=llink(rover); q:=mem←min+1; link(mem←min):=null; info(mem←min):=null; {we don't use the bottom word} rlink(p):=q; llink(rover):=q;@/ rlink(q):=rover; llink(q):=p; link(q):=empty←flag; node←size(q):=mem←bot-q; end; undump(lo←mem←max+1)(hi←mem←stat←min)(hi←mem←min); undump(null)(mem←top)(avail); mem←end:=mem←top; for k:=hi←mem←min to mem←end do undump←wd(mem[k]); undump←int(var←used); undump←int(dyn←used) @ @<Dump the table of equivalents@>= @<Dump regions 1 to 4 of |eqtb|@>; @<Dump regions 5 and 6 of |eqtb|@>; dump←int(par←loc); dump←int(write←loc);@/ @<Dump the hash table@> @ @<Undump the table of equivalents@>= @<Undump regions 1 to 6 of |eqtb|@>; undump(hash←base)(frozen←control←sequence)(par←loc); par←token:=cs←token←flag+par←loc;@/ undump(hash←base)(frozen←control←sequence)(write←loc);@/ @<Undump the hash table@> @ The table of equivalents usually contains repeated information, so we dump it in compressed form: The sequence of $n+2$ values $(n,x←1,\ldots,x←n,m)$ in the format file represents $n+m$ consecutive entries of |eqtb|, with |m| extra copies of $x←n$, namely $(x←1,\ldots,x←n,x←n,\ldots,x←n)$. @<Dump regions 1 to 4 of |eqtb|@>= k:=active←base; repeat j:=k; while j<int←base-1 do begin if (equiv(j)=equiv(j+1))and(eq←type(j)=eq←type(j+1))and@| (eq←level(j)=eq←level(j+1)) then goto found1; incr(j); end; l:=int←base; goto done1; {|j=int←base-1|} found1: incr(j); l:=j; while j<int←base-1 do begin if (equiv(j)<>equiv(j+1))or(eq←type(j)<>eq←type(j+1))or@| (eq←level(j)<>eq←level(j+1)) then goto done1; incr(j); end; done1:dump←int(l-k); while k<l do begin dump←wd(eqtb[k]); incr(k); end; k:=j+1; dump←int(k-l); until k=int←base @ @<Dump regions 5 and 6 of |eqtb|@>= repeat j:=k; while j<eqtb←size do begin if eqtb[j].int=eqtb[j+1].int then goto found2; incr(j); end; l:=eqtb←size+1; goto done2; {|j=eqtb←size|} found2: incr(j); l:=j; while j<eqtb←size do begin if eqtb[j].int<>eqtb[j+1].int then goto done2; incr(j); end; done2:dump←int(l-k); while k<l do begin dump←wd(eqtb[k]); incr(k); end; k:=j+1; dump←int(k-l); until k>eqtb←size @ @<Undump regions 1 to 6 of |eqtb|@>= k:=active←base; repeat undump←int(x); if (x<1)or(k+x>eqtb←size+1) then goto bad←fmt; for j:=k to k+x-1 do undump←wd(eqtb[j]); k:=k+x; undump←int(x); if (x<0)or(k+x>eqtb←size+1) then goto bad←fmt; for j:=k to k+x-1 do eqtb[j]:=eqtb[k-1]; k:=k+x; until k>eqtb←size @ A different scheme is used to compress the hash table, since its lower region is usually sparse. When |text(p)<>0| for |p<=hash←used|, we output two words, |p| and |hash[p]|. The hash table is, of course, densely packed for |p>=hash←used|, so the remaining entries are output in a block. @<Dump the hash table@>= dump←int(hash←used); cs←count:=frozen←control←sequence-1-hash←used; for p:=hash←base to hash←used do if text(p)<>0 then begin dump←int(p); dump←hh(hash[p]); incr(cs←count); end; for p:=hash←used+1 to undefined←control←sequence-1 do dump←hh(hash[p]); dump←int(cs←count);@/ print←ln; print←int(cs←count); print(" multiletter control sequences") @ @<Undump the hash table@>= undump(hash←base)(frozen←control←sequence)(hash←used); p:=hash←base-1; repeat undump(p+1)(hash←used)(p); undump←hh(hash[p]); until p=hash←used; for p:=hash←used+1 to undefined←control←sequence-1 do undump←hh(hash[p]); undump←int(cs←count) @ @<Dump the font information@>= dump←int(fmem←ptr); for k:=0 to fmem←ptr-1 do dump←wd(font←info[k]); dump←int(font←ptr); for k:=null←font to font←ptr do @<Dump the array info for internal font number |k|@>; print←ln; print←int(fmem←ptr-7); print(" words of font info for "); print←int(font←ptr-font←base); print(" preloaded font"); if font←ptr<>font←base+1 then print←char("s") @ @<Undump the font information@>= undump←size(7)(font←mem←size)('font mem size')(fmem←ptr); for k:=0 to fmem←ptr-1 do undump←wd(font←info[k]); undump←size(font←base)(font←max)('font max')(font←ptr); for k:=null←font to font←ptr do @<Undump the array info for internal font number |k|@> @ @<Dump the array info for internal font number |k|@>= begin dump←qqqq(font←check[k]); dump←int(font←size[k]); dump←int(font←dsize[k]); dump←int(font←params[k]);@/ dump←int(hyphen←char[k]); dump←int(skew←char[k]);@/ dump←int(font←name[k]); dump←int(font←area[k]);@/ dump←int(font←bc[k]); dump←int(font←ec[k]);@/ dump←int(char←base[k]); dump←int(width←base[k]); dump←int(height←base[k]);@/ dump←int(depth←base[k]); dump←int(italic←base[k]); dump←int(lig←kern←base[k]);@/ dump←int(kern←base[k]); dump←int(exten←base[k]); dump←int(param←base[k]);@/ dump←int(font←glue[k]);@/ print←nl("\font"); print←esc(font←id←text(k)); print←char("="); print←file←name(font←name[k],font←area[k],""); if font←size[k]<>font←dsize[k] then begin print(" at "); print←scaled(font←size[k]); print("pt"); end; end @ @<Undump the array info for internal font number |k|@>= begin undump←qqqq(font←check[k]);@/ undump←int(font←size[k]); undump←int(font←dsize[k]); undump(min←halfword)(max←halfword)(font←params[k]);@/ undump←int(hyphen←char[k]); undump←int(skew←char[k]);@/ undump(0)(str←ptr)(font←name[k]); undump(0)(str←ptr)(font←area[k]);@/ undump(0)(255)(font←bc[k]); undump(0)(255)(font←ec[k]);@/ undump←int(char←base[k]); undump←int(width←base[k]); undump←int(height←base[k]);@/ undump←int(depth←base[k]); undump←int(italic←base[k]); undump←int(lig←kern←base[k]);@/ undump←int(kern←base[k]); undump←int(exten←base[k]); undump←int(param←base[k]);@/ undump(min←halfword)(lo←mem←max)(font←glue[k]); end @ @<Dump the hyphenation tables@>= dump←int(hyph←count); for k:=0 to hyph←size do if hyph←word[k]<>0 then begin dump←int(k); dump←int(hyph←word[k]); dump←int(hyph←list[k]); end; dump←int(trie←max); for k:=0 to trie←max do dump←hh(trie[k]); dump←int(trie←op←ptr); for k:=min←quarterword+1 to trie←op←ptr do begin dump←int(hyf←distance[k]); dump←int(hyf←num[k]); dump←int(hyf←next[k]); end; print←ln; print←int(hyph←count); print(" hyphenation exception"); if hyph←count<>1 then print←char("s"); print←nl("Hyphenation trie of length "); print←int(trie←max); @.Hyphenation trie...@> print(" has "); print←int(qo(trie←op←ptr)); print(" op"); if trie←op←ptr<>min←quarterword+1 then print←char("s") @ @<Undump the hyphenation tables@>= undump(0)(hyph←size)(hyph←count); for k:=1 to hyph←count do begin undump(0)(hyph←size)(j); undump(0)(str←ptr)(hyph←word[j]); undump(min←halfword)(max←halfword)(hyph←list[j]); end; undump←size(0)(trie←size)('trie size')(trie←max); for k:=0 to trie←max do undump←hh(trie[k]); undump(min←quarterword)(max←quarterword)(trie←op←ptr); for k:=min←quarterword+1 to trie←op←ptr do begin undump(0)(63)(hyf←distance[k]); {a |small←number|} undump(0)(63)(hyf←num[k]); undump(min←quarterword)(max←quarterword)(hyf←next[k]); end @ We have already printed a lot of statistics, so we set |tracing←stats:=0| to prevent them appearing again. @<Dump a couple more things and the closing check word@>= dump←int(interaction); dump←int(format←ident); dump←int(69069); tracing←stats:=0 @ @<Undump a couple more things and the closing check word@>= undump(batch←mode)(error←stop←mode)(interaction); undump(0)(str←ptr)(format←ident); undump←int(x); if (x<>69069)or eof(fmt←file) then goto bad←fmt @ @<Create the |format←ident|...@>= selector:=new←string; print(" (preloaded format="); print(job←name); print←char(" "); print←int(year mod 100); print←char("."); print←int(month); print←char("."); print←int(day); print←char(")"); if interaction=batch←mode then selector:=log←only else selector:=term←and←log; str←room(1); format←ident:=make←string; pack←job←name(".fmt"); while not w←open←out(fmt←file) do prompt←file←name("format file name",".fmt"); print←nl("Beginning to dump on file "); @.Beginning to dump...@> print(w←make←name←string(fmt←file)); flush←string; print←nl(format←ident) @ @<Close the format file@>= w←close(fmt←file) @* \[51] The main program. This is it: the part of \TeX\ that executes all those procedures we have written. Well---almost. Let's leave space for a few more routines that we may have forgotten. @p @<Last-minute procedures@> @ We have noted that there are two versions of \TeX82. One, called \.{INITEX}, @.INITEX@> has to be run first; it initializes everything from scratch, without reading a format file, and it has the capability of dumping a format file. The other one is called `\.{VIRTEX}'; it is a ``virgin'' program that needs @.VIRTEX@> to input a format file in order to get started. \.{VIRTEX} typically has more memory capacity than \.{INITEX}, because it does not need the space consumed by the auxiliary hyphenation tables and the numerous calls on |primitive|, etc. The \.{VIRTEX} program cannot read a format file instantaneously, of course; the best implementations therefore allow for production versions of \TeX\ that not only avoid the loading routine for \PASCAL\ object code, they also have a format file pre-loaded. This is impossible to do if we stick to standard \PASCAL, but there is a simple way to fool many systems into avoiding the initialization, as follows.\quad(1)~We declare a global integer variable called |ready←already|. The probability is negligible that this variable holds any particular value like 314159 when \.{VIRTEX} is first loaded.\quad(2)~After we have read in a format file and initialized everything, we set |ready←already:=314159|.\quad(3)~Soon \.{VIRTEX} will print `\.*', waiting for more input; and at this point we interrupt the program and save its core image in some form that the operating system can reload speedily.\quad(4)~When that core image is activated, the program starts again at the beginning; but now |ready←already=314159| and all the other global variables have their initial values too. The former chastity has vanished! In other words, if we allow ourselves to test the condition |ready←already=314159|, before |ready←already| has been assigned a value, we can avoid the lengthy initialization. Dirty tricks rarely pay off so handsomely. @↑dirty \PASCAL@> @↑system dependencies@> On systems that allow such preloading, the standard program called \.{TeX} should be the one that has \.{PLAIN} format preloaded, since that agrees with {\sl The \TeX book}. Other versions, e.g., \.{AmSTeX}, should also @:TeXbook}{\sl The \TeX book@> @.AmSTeX@> @.PLAIN@> be provided for commonly used formats. @<Glob...@>= @!ready←already:integer; {a sacrifice of purity for economy} @ Now this is really it: \TeX\ starts and ends here. The initial test involving |ready←already| should be deleted if the \PASCAL\ runtime system is smart enough to detect such a ``mistake.'' @↑system dependencies@> @p begin @!{|start←here|} history:=fatal←error←stop; {in case we quit during initialization} t←open←out; {open the terminal for output} if ready←already=314159 then goto start←of←TEX; @<Check the ``constant'' values...@>@; if bad>0 then begin t←open←out; wterm('Ouch---my internal constants have been', ' clobbered!---case ',bad:1); @.Ouch...clobbered@> goto final←end; end; initialize; {set global variables to their starting values} @!init if not get←strings←started then goto final←end; init←prim; {call |primitive| for each primitive} tini@/ ready←already:=314159; start←of←TEX: @<Initialize the output routines@>; @<Get the first line of input and prepare to start@>; init←str←ptr:=str←ptr; init←pool←ptr:=pool←ptr;@/ history:=spotless; {ready to go!} main←control; {come to life} final←cleanup; {prepare for death} end←of←TEX: close←files←and←terminate; final←end: ready←already:=0; end. @ Here we do whatever is needed to complete \TeX's job gracefully on the local operating system. The code here might come into play after a fatal error; it must therefore consist entirely of ``safe'' operations that cannot produce error messages. For example, it would be a mistake to call |str←room| or |make←string| at this time, because a call on |overflow| might lead to an infinite loop. @↑system dependencies@> @<Last-minute...@>= procedure close←files←and←terminate; var k:integer; {all-purpose index} begin @<Finish the extensions@>; @!stat if tracing←stats>0 then @<Output statistics about this job@>;@;@+tats@/ wake←up←terminal; @<Finish the \.{DVI} file@>; if job←name>0 then begin wlog←cr; a←close(log←file); selector:=selector-2; if selector=term←only then begin print←nl("Transcript written on "); @.Transcript written...@> print(log←name); print←char("."); end; end; end; @ The present section goes directly to the log file instead of using |print| commands, because there's no need for these strings to take up |str←pool| memory when a non-{\bf stat} version of \TeX\ is being used. @<Output statistics...@>= if job←name>0 then {the log file is open} begin wlog←ln(' '); wlog←ln('Here is how much of TeX''s memory',' you used:'); @.Here is how much...@> wlog(' ',str←ptr-init←str←ptr:1,' string'); if str←ptr<>init←str←ptr+1 then wlog('s'); wlog←ln(' out of ', max←strings-init←str←ptr:1);@/ wlog←ln(' ',pool←ptr-init←pool←ptr:1,' string characters out of ', pool←size-init←pool←ptr:1);@/ wlog←ln(' ',lo←mem←max-mem←min+mem←end-hi←mem←min+2:1,@| ' words of memory out of ',mem←end+1-mem←min:1);@/ wlog←ln(' ',cs←count:1,' multiletter control sequences out of ', hash←size:1);@/ wlog(' ',fmem←ptr:1,' words of font info for ', font←ptr-font←base:1,' font'); if font←ptr<>font←base+1 then wlog('s'); wlog←ln(', out of ',font←mem←size:1,' for ',font←max-font←base:1);@/ wlog(' ',hyph←count:1,' hyphenation exception'); if hyph←count<>1 then wlog('s'); wlog←ln(' out of ',hyph←size:1);@/ wlog←ln(' ',max←in←stack:1,'i,',max←nest←stack:1,'n,',@| max←param←stack:1,'p,',@| max←buf←stack+1:1,'b,',@| max←save←stack+6:1,'s stack positions out of ',@| stack←size:1,'i,', nest←size:1,'n,', param←size:1,'p,', buf←size:1,'b,', save←size:1,'s'); end @ We get to the |final←cleanup| routine when \.{\\end} or \.{\\dump} has been scanned and |its←all←over|. @<Last-minute...@>= procedure final←cleanup; label exit; var c:small←number; {0 for \.{\\end}, 1 for \.{\\dump}} begin c:=cur←chr; if job←name=0 then open←log←file; if cur←level>level←one then begin print←nl("("); print←esc("end occurred "); print("inside a group at level "); @:end←}{\.{(\\end occurred...)}@> print←int(cur←level-level←one); print←char(")"); end; while cond←ptr<>null do begin print←nl("("); print←esc("end occurred "); print("when "); print←cmd←chr(if←test,cur←if); if if←line<>0 then begin print(" on line "); print←int(if←line); end; print(" was incomplete)"); if←line:=if←line←field(cond←ptr); cur←if:=subtype(cond←ptr); cond←ptr:=link(cond←ptr); end; if history<>spotless then if ((history=warning←issued)or(interaction<error←stop←mode)) then if selector=term←and←log then begin selector:=term←only; print←nl("(see the transcript file for additional information)"); @.see the transcript file...@> selector:=term←and←log; end; if c=1 then begin @!init store←fmt←file; return;@+tini@/ print←nl("(\dump is performed only by INITEX)"); return; @:dump←}{\.{\\dump...only by INITEX}@> end; exit:end; @ @<Last-minute...@>= @!init procedure init←prim; {initialize all the primitives} begin no←new←control←sequence:=false; @<Put each...@>; no←new←control←sequence:=true; end; tini @ When we begin the following code, \TeX's tables may still contain garbage; the strings might not even be present. Thus we must proceed cautiously to get bootstrapped in. But when we finish this part of the program, \TeX\ is ready to call on the |main←control| routine to do its work. @<Get the first line...@>= begin @<Initialize the input routines@>; if (format←ident=0)or(buffer[loc]="&") then begin if format←ident<>0 then initialize; {erase preloaded format} if not open←fmt←file then goto final←end; if not load←fmt←file then begin w←close(fmt←file); goto final←end; end; w←close(fmt←file); while (loc<limit)and(buffer[loc]=" ") do incr(loc); end; if (end←line←char<0)or(end←line←char>127) then decr(limit) else buffer[limit]:=end←line←char; fix←date←and←time;@/ @<Compute the magic offset@>; @<Initialize the print |selector|...@>; if (loc<limit)and(cat←code(buffer[loc])<>escape) then start←input; {\.{\\input} assumed} end @* \[52] Debugging. Once \TeX\ is working, it should be possible to diagnose most errors with the \.{\\show} commands and other diagnostic features. But for the initial stages of debugging, and for the revelation of really deep mysteries, you can compile \TeX\ with a few more aids including the \PASCAL\ runtime checks and its debugger. An additional routine called |debug←help| will also come into play when you type `\.D' after an error message; |debug←help| also occurs just before a fatal error causes \TeX\ to succumb. @↑debugging@> @↑system dependencies@> The interface to |debug←help| is primitive, but it is good enough when used with a \PASCAL\ debugger that allows you to set breakpoints and to read variables and change their values. After getting the prompt `\.{debug \#}', you type either a negative number (this exits |debug←help|), or zero (this goes to a location where you can set a breakpoint, thereby entering into dialog with the \PASCAL\ debugger), or a positive number |m| followed by an argument |n|. The meaning of |m| and |n| will be clear from the program below. (If |m=13|, there is an additional argument, |l|.) @.debug \#@> @d breakpoint=888 {place where a breakpoint is desirable} @<Last-minute...@>= @!debug procedure debug←help; {routine to display various things} label breakpoint,exit; var k,@!l,@!m,@!n:integer; begin loop begin wake←up←terminal; print←nl("debug # (-1 to exit):"); update←terminal; @.debug \#@> read(term←in,m); if m<0 then return else if m=0 then begin goto breakpoint;@\ {go to every label at least once} breakpoint: m:=0; @{'BREAKPOINT'@}@\ end else begin read(term←in,n); case m of @t\4@>@<Numbered cases for |debug←help|@>@; othercases print("?") endcases; end; end; exit:end; gubed @ @<Numbered cases...@>= 1: print←word(mem[n]); {display |mem[n]| in all forms} 2: print←int(info(n)); 3: print←int(link(n)); 4: print←word(eqtb[n]); 5: print←word(font←info[n]); 6: print←word(save←stack[n]); 7: show←box(n); {show a box, abbreviated by |show←box←depth| and |show←box←breadth|} 8: begin breadth←max:=10000; depth←threshold:=pool←size-pool←ptr-10; show←node←list(n); {show a box in its entirety} end; 9: show←token←list(n,null,1000); 10: print(n); 11: check←mem(n>0); {check wellformedness; print new busy locations if |n>0|} 12: search←mem(n); {look for pointers to |n|} 13: begin read(term←in,l); print←cmd←chr(n,l); end; 14: for k:=0 to n do print(buffer[k]); 15: begin font←in←short←display:=null←font; short←display(n); end; 16: panicking:=not panicking; @* \[53] Extensions. The program above includes a bunch of ``hooks'' that allow further capabilities to be added without upsetting \TeX's basic structure. Most of these hooks are concerned with ``whatsit'' nodes, which are intended to be used for special purposes; whenever a new extension to \TeX\ involves a new kind of whatsit node, a corresponding change needs to be made to the routines below that deal with such nodes, but it will usually be unnecessary to make many changes to the other parts of this program. In order to demonstrate how extensions can be made, we shall treat `\.{\\write}', `\.{\\openout}', `\.{\\closeout}', `\.{\\immediate}, and `\.{\\special}' as if they were extensions. These commands are actually primitives of \TeX82, and they should appear in all implementations of the system; but let's try to imagine that they aren't. Then the program below illustrates how a person could add them. Sometimes, of course, an extension will require changes to \TeX\ itself; no system of hooks could be complete enough for all conceivable extensions. The features associated with `\.{\\write}' are almost all confined to the following paragraphs, but there are small parts of the |print←ln| and |print←char| procedures that were introduced specifically to \.{\\write} characters. Furthermore one of the token lists recognized by the scanner is a |write←text|; and there are a few other miscellaneous places where we have already provided for some aspect of \.{\\write}. The goal of a \TeX\ extender should be to minimize alterations to the standard parts of the program, and to avoid them completely if possible. He or she should also be quite sure that there's no easy way to accomplish the desired goals with the standard features that \TeX\ already has. ``Think thrice before extending,'' because that may save a lot of work, and it will also keep incompatible extensions of \TeX\ from proliferating. @↑system dependencies@> @↑extensions to \TeX@> @ First let's consider the format of whatsit nodes that are used to represent the data associated with \.{\\write} and its relatives. Recall that a whatsit has |type=whatsit←node|, and the |subtype| is supposed to distinguish different kinds of whatsits. Each node occupies two or more words; the exact number is immaterial, as long as it is readily determined from the |subtype| or other data. We shall introduce four |subtype| values here, corresponding to the control sequences \.{\\openout}, \.{\\write}, \.{\\closeout}, and \.{\\special}. The second word of such whatsits has a |write←stream| field that identifies the write-stream number (0 to 15, or 16 for out-of-range and positive, or 17 for out-of-range and negative). In the case of \.{\\write} and \.{\\special}, there is also a field that points to the reference count of a token list that should be sent. In the case of \.{\\openout}, we need three words and three auxiliary subfields to hold the string numbers for name, area, and extension. @d write←node←size=2 {number of words in a write/whatsit node} @d open←node←size=3 {number of words in an open/whatsit node} @d open←node=0 {|subtype| in whatsits that represent files to \.{\\openout}} @d write←node=1 {|subtype| in whatsits that represent things to \.{\\write}} @d close←node=2 {|subtype| in whatsits that represent streams to \.{\\closeout}} @d special←node=3 {|subtype| in whatsits that represent \.{\\special} things} @d write←tokens(#) == link(#+1) {reference count of token list to write} @d write←stream(#) == info(#+1) {stream number (0 to 16)} @d open←name(#) == link(#+1) {string number of file name to open} @d open←area(#) == info(#+2) {string number of file area for |open←name|} @d open←ext(#) == link(#+2) {string number of file extension for |open←name|} @ The sixteen possible \.{\\write} streams are represented by the |write←file| array. The |j|th file is open if and only if |write←open[j]=true|. The last two streams are special; |write←open[16]| represents a stream number greater than 15, while |write←open[17]| represents a negative stream number, and both of these variables are always |false|. @<Glob...@>= @!write←file:array[0..15] of alpha←file; @!write←open:array[0..17] of boolean; @ @<Set init...@>= for k:=0 to 17 do write←open[k]:=false; @ Extensions might introduce new command codes; but it's best to use |extension| with a modifier, whenever possible, so that |main←control| stays the same. @d immediate←code=4 {command modifier for \.{\\immediate}} @<Put each...@>= primitive("openout",extension,open←node);@/ @!@:open←out←}{\.{\\openout} primitive@> primitive("write",extension,write←node); write←loc:=cur←val;@/ @!@:write←}{\.{\\write} primitive@> primitive("closeout",extension,close←node);@/ @!@:close←out←}{\.{\\closeout} primitive@> primitive("special",extension,special←node);@/ @!@:special←}{\.{\\special} primitive@> primitive("immediate",extension,immediate←code);@/ @!@:immediate←}{\.{\\immediate} primitive@> @ The variable |write←loc| just introduced is used to provide an appropriate error message in case of ``runaway'' write texts. @<Glob...@>= @!write←loc:pointer; {|eqtb| address of \.{\\write}} @ @<Cases of |print←cmd←chr|...@>= extension: case chr←code of open←node:print←esc("openout"); write←node:print←esc("write"); close←node:print←esc("closeout"); special←node:print←esc("special"); immediate←code:print←esc("immediate"); othercases print("[unknown extension!]") endcases; @ When an |extension| command occurs in |main←control|, in any mode, the |do←extension| routine is called. @<Cases of |main←control| that are for extensions...@>= any←mode(extension):do←extension; @ @<Declare act...@>= @t\4@>@<Declare procedures needed in |do←extension|@>@; procedure do←extension; var i,@!j,@!k:integer; {all-purpose integers} @!p,@!q,@!r:pointer; {all-purpose pointers} begin case cur←chr of open←node:@<Implement \.{\\openout}@>; write←node:@<Implement \.{\\write}@>; close←node:@<Implement \.{\\closeout}@>; special←node:@<Implement \.{\\special}@>; immediate←code:@<Implement \.{\\immediate}@>; othercases confusion("ext1") @:this can't happen ext1}{\quad ext1@> endcases; end; @ Here is a subroutine that creates a whatsit node having a given |subtype| and a given number of words. It initializes only the first word of the whatsit, and appends it to the current list. @<Declare procedures needed in |do←extension|@>= procedure new←whatsit(@!s:small←number;@!w:small←number); var p:pointer; {the new node} begin p:=get←node(w); type(p):=whatsit←node; subtype(p):=s; link(tail):=p; tail:=p; end; @ The next subroutine uses |cur←chr| to decide what sort of whatsit is involved, and also inserts a |write←stream| number. @<Declare procedures needed in |do←ext...@>= procedure new←write←whatsit(@!w:small←number); begin new←whatsit(cur←chr,w); if w<>write←node←size then scan←four←bit←int else begin scan←int; if cur←val<0 then cur←val:=17 else if cur←val>15 then cur←val:=16; end; write←stream(tail):=cur←val; end; @ @<Implement \.{\\openout}@>= begin new←write←whatsit(open←node←size); scan←optional←equals; scan←file←name;@/ open←name(tail):=cur←name; open←area(tail):=cur←area; open←ext(tail):=cur←ext; end @ When `\.{\\write 12\{...\}}' appears, we scan the token list `\.{\{...\}}' without expanding its macros; the macros will be expanded later when this token list is rescanned. @<Implement \.{\\write}@>= begin k:=cur←cs; new←write←whatsit(write←node←size);@/ cur←cs:=k; p:=scan←toks(false,false); write←tokens(tail):=def←ref; end @ @<Implement \.{\\closeout}@>= begin new←write←whatsit(write←node←size); write←tokens(tail):=null; end @ When `\.{\\special\{...\}}' appears, we expand the macros in the token list as in \.{\\xdef} and \.{\\mark}. @<Implement \.{\\special}@>= begin new←whatsit(special←node,write←node←size); write←stream(tail):=null; p:=scan←toks(false,true); write←tokens(tail):=def←ref; end @ Each new type of node that appears in our data structure must be capable of being displayed, copied, destroyed, and so on. The routines that we need for write-oriented whatsits are somewhat like those for mark nodes; other extensions might, of course, involve more subtlety here. @<Basic printing...@>= procedure print←write←whatsit(@!s:str←number;@!p:pointer); begin print←esc(s); if write←stream(p)<16 then print←int(write←stream(p)) else if write←stream(p)=16 then print←char("*") else print←char("-"); end; @ @<Display the whatsit...@>= case subtype(p) of open←node:begin print←write←whatsit("openout",p); print←char("="); print←file←name(open←name(p),open←area(p),open←ext(p)); end; write←node:begin print←write←whatsit("write",p); print←mark(write←tokens(p)); end; close←node:print←write←whatsit("closeout",p); special←node:begin print←esc("special"); print←mark(write←tokens(p)); end; othercases print("whatsit?") endcases @ @<Make a partial copy of the whatsit...@>= case subtype(p) of open←node: begin r:=get←node(open←node←size); words:=open←node←size; end; write←node,special←node: begin r:=get←node(write←node←size); add←token←ref(write←tokens(p)); words:=write←node←size; end; close←node: begin r:=get←node(small←node←size); words:=small←node←size; end; othercases confusion("ext2") @:this can't happen ext2}{\quad ext2@> endcases @ @<Wipe out the whatsit...@>= begin case subtype(p) of open←node: free←node(p,open←node←size); write←node,special←node: begin delete←token←ref(write←tokens(p)); free←node(p,write←node←size); goto done; end; close←node: free←node(p,small←node←size); othercases confusion("ext3") @:this can't happen ext3}{\quad ext3@> endcases;@/ goto done; end @ @<Incorporate a whatsit node into a vbox@>=do←nothing @ @<Incorporate a whatsit node into an hbox@>=do←nothing @ @<Let |d| be the width of the whatsit |p|@>=d:=0 @ @<Advance \(p)past a whatsit node in the |line←break| loop@>=do←nothing @ @<Prepare to move whatsit |p| to the current page, then |goto contribute|@>= goto contribute @ @<Process whatsit |p| in |vert←break| loop, |goto not←found|@>= goto not←found @ @<Output the whatsit node |p| in a vlist@>= out←what(p) @ @<Output the whatsit node |p| in an hlist@>= out←what(p) @ @<Finish the extensions@>= for k:=0 to 15 do if write←open[k] then a←close(write←file[k]) @ After all this preliminary shuffling, we come finally to the routines that actually send out the requested data. Let's do \.{\\special} first (it's easier). @<Declare procedures needed in |hlist←out|, |vlist←out|@>= procedure special←out(@!p:pointer); var old←setting:0..max←selector; {holds print |selector|} @!k:pool←pointer; {index into |str←pool|} begin synch←h; synch←v;@/ old←setting:=selector; selector:=new←string; show←token←list(link(write←tokens(p)),null,pool←size-pool←ptr); selector:=old←setting; str←room(1); if cur←length<256 then begin dvi←out(xxx1); dvi←out(cur←length); end else begin dvi←out(xxx4); dvi←four(cur←length); end; for k:=str←start[str←ptr] to pool←ptr-1 do dvi←out(str←pool[k]); pool←ptr:=str←start[str←ptr]; {erase the string} end; @ To write a token list, we must run it through \TeX's scanner, expanding macros and \.{\\the} and \.{\\number}, etc. This might cause runaways, if a delimited macro parameter isn't matched, and runaways would be extremely confusing since we are calling on \TeX's scanner in the middle of a \.{\\shipout} command. Therefore we will put a dummy control sequence as a ``stopper,'' right after the token list. This control sequence is artificially defined to be \.{\\outer}. @:end←write←}{\.{\\endwrite}@> @<Initialize table...@>= text(end←write):="endwrite"; eq←level(end←write):=level←one; eq←type(end←write):=outer←call; equiv(end←write):=null; @ @<Declare procedures needed in |hlist←out|, |vlist←out|@>= procedure write←out(@!p:pointer); var old←setting:0..max←selector; {holds print |selector|} @!old←mode:integer; {saved |mode|} @!j:small←number; {write stream number} @!q,@!r:pointer; {temporary variables for list manipulation} begin @<Expand macros in the token list and make |link(def←ref)| point to the result@>; old←setting:=selector; j:=write←stream(p); if write←open[j] then selector:=j else begin {write to the terminal if file isn't open} if (j=17)and(selector=term←and←log) then selector:=log←only; print←nl(""); end; show←token←list(link(def←ref),null,buf←size-10); print←ln; flush←list(def←ref); selector:=old←setting; end; @ The final line of this routine is slightly subtle; at least, the author didn't think about it until getting burnt! There is a used-up token list @↑Knuth, Donald Ervin@> on the stack, namely the one that contained |end←write←token|. (We insert this artificial `\.{\\endwrite}' to prevent runaways, as explained above.) If it were not removed, and if there were numerous writes on a single page, the stack would overflow. @d end←write←token==cs←token←flag+end←write @<Expand macros in the token list and...@>= q:=get←avail; info(q):=right←brace←token+"}";@/ r:=get←avail; link(q):=r; info(r):=end←write←token; ins←list(q);@/ begin←token←list(write←tokens(p),write←text);@/ q:=get←avail; info(q):=left←brace←token+"{"; ins←list(q); {now we're ready to scan `\.\{$\langle\,$token list$\,\rangle$\.{\} \\endwrite}'} old←mode:=mode; mode:=0; {disable \.{\\prevdepth}, \.{\\spacefactor}, \.{\\lastskip}} cur←cs:=write←loc; q:=scan←toks(false,true); {expand macros, etc.} get←token;@+if cur←tok<>end←write←token then @<Recover from an unbalanced write command@>; mode:=old←mode; end←token←list {conserve stack space} @ @<Recover from an unbalanced write command@>= begin print←err("Unbalanced write command"); @.Unbalanced write...@> help2("On this page there's a \write with fewer real {'s than }'s.")@/ ("I can't handle that very well; good luck."); error; repeat get←token; until cur←tok=end←write←token; end @ The |out←what| procedure takes care of outputting whatsit nodes for |vlist←out| and |hlist←out|. @<Declare procedures needed in |hlist←out|, |vlist←out|@>= procedure out←what(@!p:pointer); var j:small←number; {write stream number} begin case subtype(p) of open←node,write←node,close←node:@<Do some work that has been queued up for \.{\\write}@>; special←node:special←out(p); othercases confusion("ext4") @:this can't happen ext4}{\quad ext4@> endcases; end; @ We don't implement \.{\\write} inside of leaders. (The reason is that the number of times a leader box appears might be different in different implementations, due to machine-dependent rounding in the glue calculations.) @↑leaders@> @<Do some work that has been queued up...@>= if not doing←leaders then begin j:=write←stream(p); if subtype(p)=write←node then write←out(p) else begin if write←open[j] then a←close(write←file[j]); if subtype(p)=close←node then write←open[j]:=false else if j<16 then begin cur←name:=open←name(p); cur←area:=open←area(p); cur←ext:=open←ext(p); if cur←ext="" then cur←ext:=".tex"; pack←cur←name; while not a←open←out(write←file[j]) do prompt←file←name("output file name",".tex"); write←open[j]:=true; end; end; end @ The presence of `\.{\\immediate}' causes the |do←extension| procedure to descend to one level of recursion. Nothing happens unless \.{\\immediate} is followed by `\.{\\openout}', `\.{\\write}', or `\.{\\closeout}'. @↑recursion@> @<Implement \.{\\immediate}@>= begin get←x←token; if (cur←cmd=extension)and(cur←chr<=close←node) then begin p:=tail; do←extension; {append a whatsit node} out←what(tail); {do the action immediately} flush←node←list(tail); tail:=p; link(p):=null; end else back←input; end @* \[54] System-dependent changes. This section should be replaced, if necessary, by changes to the program that are necessary to make \TeX\ work at a particular installation. It is usually best to design your change file so that all changes to previous sections preserve the section numbering; then everybody's version will be consistent with the published program. More extensive changes, which introduce new sections, can be inserted here; then only the index itself will get a new section number. @↑system dependencies@> @* \[55] Index. Here is where you can find all uses of each identifier in the program, with underlined entries pointing to where the identifier was defined. If the identifier is only one letter long, however, you get to see only the underlined entries. All references are to section numbers instead of page numbers. This index also lists error messages and other aspects of the program that you might want to look up some day. For example, the entry for ``system dependencies'' lists all sections that should receive special attention from people who are installing \TeX\ in a new operating environment. A list of various things that can't happen appears under ``this can't happen''. Approximately 40 sections are listed under ``inner loop''; these account for about 60\% of \TeX's running time, exclusive of input and output.