XEROX	SINGLEFILEINDEX
2

4

1

SINGLEFILEINDEX
1

4

By:  Christopher Tong and Jon L. White
Revision by: Bill van Melle (vanMelle.PA@Xerox.com)
INTRODUCTION
SINGLEFILEINDEX is a program for generating indexed listings for Interlisp source files, intended to augment the behavior of Interlisp's LISTFILES.  SINGLEFILEINDEX operates by scanning the file as it lists it, building up an index consisting of all "interesting" objects on the file, identified by type (e.g. FNS, VARS, etc.) and the page number on which their definitions appear in the listing.  At the end of the file listing, SINGLEFILEINDEX lists this index, sorted by type, and within type, alphabetically by name.  The user is expected to manually transpose the index to the front of the collection of paper that emerges from the printer.  
For normal use, it suffices to know that loading SINGLEFILEINDEX.LCOM  redefines the function LISTFILES1 so that calling LISTFILES (or using the File Browser's Hardcopy command) instead invokes SINGLEFILEINDEX if the file is a Lisp source file.  This is the manner in which most people use SINGLEFILEINDEX.  The listing is created by default in a background process.  The file being indexed needn't be loaded, or even noticed (in the file package sense).  However, the file must reside on a random-access device.
The remainder of this document describes the programmer's interface to SINGLEFILEINDEX.
CREATING INDEXED FILE LISTINGS
Ordinarily, one calls LISTFILES (or uses the File Browser) to create listings.  However, for special cases, such as when the output of the indexing process is not to be sent to the printer directly, one can call SINGLEFILEINDEX directly:
(SINGLEFILEINDEX  file outputfile)	[Function]
Lists file, an Interlisp source file, printing it and its index to outputfile.  outputfile can either be the name of a file to be created or a stream that is already open for output (in which case it is left open afterwards).  If outputfile is NIL, the output goes to the value of PRINTERDEVICEFILENAME  (initially {LPT}). 
HOW TO SPECIFY INDEXING TYPES
Initially, SINGLEFILEINDEX knows about most of the standard file package types.  You can augment the set of types that it will index by adding elements to the following variable:
SINGLEFILEINDEX.TYPES	[Variable]
A list of types to be indexed.  Each type entry names the type and a way of testing whether an expression on the file is of the desired type.  The elements of SINGLEFILEINDEX.TYPES are lists of up to 4 elements of the form (type dumpfn testfn ambiguous?):
type	The name of the type, e.g., BITMAP.  This name will appear as the name of the index for this type, e.g., "BITMAP INDEX".  The type can also be a list of synonyms for this type, in which case the first element of the list is the name that will appear in the index.  type is usually the name of a file package type, though it need not be.
dumpfn	The name, or a list of names, of the function that appears as the CAR of the form that defines objects of type type on the file.  E.g., for type TEMPLATE it is SETTEMPLATE; for type VAR it is (RPAQ RPAQQ RPAQ? ADDTOVAR).
testfn	A function that tests whether the expression that starts with dumpfn really is of the desired type, and returns the name of the object defined in the expression.  The function takes as arguments (stream dumpfn entry), where stream is the stream open for input on the file being indexed, with the file pointer positioned following dumpfn (so that (READ stream) would read the CADR of the expression); dumpfn is the CAR of the form being tested; entry is the member of SINGLEFILEINDEX.TYPES being used.  The testfn should return one of the following:
NIL	the expression is not of the desired type.
name	the expression defines a single object of this name and of the type given in the entry.
a list	the value is either a single list or a list of lists, each of the form (type . names), meaning that the expression defines each of the names as having the specified type.
If the testfn is NIL or omitted, the name of the object is assumed to be the CADR of the expression on the file, and the only test that it need pass is that it be LITATOM.
ambiguous?	True if the type is ambiguous, in the sense that even if testfn returns a non-NIL value, it is possible for this expression to also satisfy other entries in SINGLEFILEINDEX.TYPES.  E.g., the VAR type is ambiguous, because (RPAQ --) expressions can also define bitmaps.  If ambiguous is true, you usually want a corresponding entry on SINGLEFILEINDEX.FILTERS (below).
SINGLEFILEINDEX handles the type FUNCTION independently of the entries on SINGLEFILEINDEX.TYPES, due to the special way functions appear on files.  You could, however, add additional entries to SINGLEFILEINDEX.TYPES with type FUNCTION if there were some other kind of object, not dumped in a DEFINEQ expression, that you wanted to treat as a function.  In addition to the types listed explicitly on SINGLEFILEINDEX.TYPES. SINGLEFILEINDEX automatically handles expressions defined by the DEFDEFINER facility.
The order of types in SINGLEFILEINDEX.TYPES also dictates the default order in which types appear on the index page(s).
SINGLEFILEINDEX.PROPERTIES	[Variable]
A list used by the default handler for the PUTPROPS form.  It associates property names with a type (something more specific than the type PROPERTY) under which objects having this property should be indexed.  Each element is of the form (propname type).  If type is NIL or omitted, then objects having this property are ignored.  In addition, the default PUTPROPS handler treats all elements of the list MACROPROPS as implying type MACRO.
The initial value of SINGLEFILEINDEX.PROPERTIES is
((COPYRIGHT)
 (READVICE ADVICE)),
meaning that the COPYRIGHT property should be ignored, and the READVICE property implies that the object should be indexed as type ADVICE.
SINGLEFILEINDEX reads the input file, one line at a time, copying it to the output file.  Whenever it finds a line that starts with a left parenthesis or bracket, it reads the next symbol and looks it up in SINGLEFILEINDEX.TYPES.  If it finds one or more matching entries, it applies the entry's testfn and creates index entries in compliance with the result returned.  Note that in order to be indexed, the expression defining an object must be printed so that the opening parenthesis/bracket is printed at the very start of the line.  SINGLEFILEINDEX knows about makefile environments; during its operation, *READTABLE* and *PACKAGE* are bound according to the DEFINE-FILE-INFO expression, if any, at the top of the file (or to the old FILERDTBL for old files lacking such an expression).   Thus, an entry's testfn can call READ with no explicit read table or package and have the right thing happen.
The following functions may be of use when defining testfns:
(TestForQuotedType stream dumpfn entry)	[Function]
A suitable testfn for types where the second element of the form is the quoted name of the object.
(TestForGenericDefinition stream dumpfn entry)	[Function]
A suitable testfn for types that are dumped using PUTDEF.  TestForGenericDefinition tests that the expression is of the form (PUTDEF 'name 'type --), where type is the type given in entry.   E.g., if you wanted to index objects of type FILEPKGCOMS, which is a type that happens to be dumped this way, you could add (FILEPKGCOMS PUTDEF TestForGenericDefinition) to SINGLEFILEINDEX.TYPES.
(SFI.WHOLE.EXPRESSION stream)	[Function]
Intended to be called from within a testfn.  Reads from stream and returns the entire expression currently being examined (recall that when the testfn is called, the left parenthesis and CAR of form have already been skipped over).
After SINGLEFILEINDEX finishes reading the input file, it tests for index entries that should be removed.   This operation makes use of the following list:
SINGLEFILEINDEX.FILTERS	[Variable]
A list describing potential index entries that should be filtered out of the final index.  Each element of SINGLEFILEINDEX.FILTERS is a list (type filterfn), where type is one of the types in SINGLEFILEINDEX.TYPES and filterfn is a function of one argument, an index entry.  If filterfn returns true, then the index entry is discarded.  An index entry is of the form (name. pagenumbers).  For convenience, an element of SINGLEFILEINDEX.FILTERS can also take the form (type . specific-type), meaning that if an object is already indexed as a specific-type then it should not also be indexed as a type.
For example, SINGLEFILEINDEX.FILTERS initially contains
((VAR . BITMAP)
 (VAR . CONSTANTS)),
meaning that "variables" that successfully index as constants or bitmaps should not also be listed in the VAR index.  This extra pass is needed because both bitmaps and constants cause expressions of the form (RPAQ var value) to be dumped on the file, and it is inconvenient to eliminate the duplication at the time the form is first indexed.
Filter functions may want to call the following function:
(SFI.LOOKUP.NAME name type)	[Function]
Looks up name in the index being built for type type.  If it finds an entry, it returns it.  Index entries are of the form (name . pagenumbers).  It is permissible for a filter function as a side effect to destructively change another index entry by adding page numbers to it.  You might want to do so, for example, in the case where there is a kind of object that dumps two expressions on a file, each of which is a different type (from SINGLEFILEINDEX.TYPES), but you want both occurrences indexed as a single type. 
OTHER SINGLEFILEINDEX VARIABLES
SINGLEFILEINDEX.TWOSIDED	[Variable]
If true, SINGLEFILEINDEX assumes you are printing on a two-sided printer.  The only effect of this variable currently is that it forces SINGLEFILEINDEX to insert a blank page before the index if necessary to force the index to be on an odd page (rather than on the back side of the last page of the source file).
SINGLEFILEINDEX.DONTSPAWN	[Variable]
If true, SINGLEFILEINDEX makes the listing directly, rather than spawning a process to do it.  If your CLEANUPOPTIONS contains LIST, you should make sure that LIST occurs last in that list, or set SINGLEFILEINDEX.DONTSPAWN to true, so that the compiler and SINGLEFILEINDEX do not fight with each other for access to the source file.
FILELINELENGTH	[Variable]
The length of a line in the output file (OPENFILE initially sets the linelength of a new stream to this value).  This affects the appearance of the index page(s) and the placement of the page number on each listing page.
PRINTERDEVICEFILENAME	[Variable]
Output of SINGLEFILEINDEX defaultly goes to this file.  Initially {LPT}, which causes the output to be sent to the default printer.
LINESPERPAGE	[Variable]
The number of lines per printed page, initially 65.  SINGLEFILEINDEX inserts page breaks this often, and avoids starting index entries or function definitions when close to the end of a page.  This variable should be set somewhat smaller than the number of lines that fit when all the text is in the default font, to allow space for the occasional taller font.
RELATIVEINDEXFLG	[Variable]
An older version of SINGLEFILEINDEX only indexed functions, used relative labels instead of absolute page numbers, and printed the function index before listing the file. Setting RELATIVEINDEXFLG to T restores this behavior, should you want it (e.g., if you are printing on fanfold paper rather than cut sheets, you may want the index to appear before the file listing).
(LIST ((PAGE NIL (PAPERSIZE LETTER STARTINGPAGE# 244) (0 0 612 792) ((FOLIO NIL (PARALOOKS (QUAD CENTERED) CHARLOOKS (SUPERSCRIPT 0 INVISIBLE OFF SELECTPOINT OFF PROTECTED OFF SIZE 10 FAMILY MODERN OVERLINE OFF STRIKEOUT OFF UNDERLINE OFF EXPANSION REGULAR SLOPE REGULAR WEIGHT MEDIUM INVERTED OFF USERINFO NIL STYLE NIL) FORMATINFO NIL) (174 36 288 36) NIL) (HEADING NIL (HEADINGTYPE RUNNINGHEAD) (84 744 528 36) NIL) (TEXT NIL NIL (84 96 456 600) NIL))) (PAGE NIL (PAPERSIZE NIL . LETTER) (0 0 612 792) ((FOLIO NIL (PARALOOKS (QUAD CENTERED) CHARLOOKS (SUPERSCRIPT 0 INVISIBLE OFF SELECTPOINT OFF PROTECTED OFF SIZE 10 FAMILY MODERN OVERLINE OFF STRIKEOUT OFF UNDERLINE OFF EXPANSION REGULAR SLOPE REGULAR WEIGHT MEDIUM INVERTED OFF USERINFO NIL STYLE NIL) FORMATINFO NIL) (174 36 288 36) NIL) (HEADING NIL (HEADINGTYPE RUNNINGHEAD) (84 744 528 36) NIL) (TEXT NIL NIL (84 96 456 600) NIL))) (PAGE NIL (PAPERSIZE NIL . LETTER) (0 0 612 792) ((FOLIO NIL (PARALOOKS (QUAD CENTERED) CHARLOOKS (SUPERSCRIPT 0 INVISIBLE OFF SELECTPOINT OFF PROTECTED OFF SIZE 10 FAMILY MODERN OVERLINE OFF STRIKEOUT OFF UNDERLINE OFF EXPANSION REGULAR SLOPE REGULAR WEIGHT MEDIUM INVERTED OFF USERINFO NIL STYLE NIL) FORMATINFO NIL) (174 36 288 36) NIL) (HEADING NIL (HEADINGTYPE RUNNINGHEAD) (84 744 528 36) NIL) (TEXT NIL NIL (84 96 456 600) NIL)))))(��
8(��
82


�

T+$$+$$+1


�
(
�(
�D
�

�
PAGEHEADINGRUNNINGHEAD++1

�
(+HH.`�
�.H
HGACHA
MODERN
MODERN
MODERNMODERNMODERN

LOGO



HRULE.GETFN
MODERN


HRULE.GETFN
MODERN


HRULE.GETFN
MODERN




HRULE.GETFN
MODERN

HRULE.GETFN
MODERN
	'4

�


X

�








9

�

"




�

�
,




\�Dp



?�

	d*&
%
,]�
�

:^
,	
*	4

J
c
M
d
C


M

+
X
\

V
)






	
%
<
�

~
	

;	
O
I=







c







2

*	


1
D-





8��

k

	
4R#

5
)



j
d



v:





	#
�
<




9


g



"
o


)
�


B
=



i


�


�
,�
�z�