{Begin SubSec Symbolic File Format}
{Title Symbolic File Format}
{Text


The file package manipulates symbolic files in a particular format.  This format is defined so that the information in the file is easily readable when the file is listed, as well as being easily manipulated by the file package functions.  In general, there is no reason for the user to manually change the contents of a symbolic file.  However, in order to allow users to extend the file package, this section describes some of the functions used to write symbolic files, and other matters related to their format.



{note should all of the following functions be documented in the manual??  Some of them (FILECREATED, PRINTDATE, PRINTFNS) seem pretty low-level functions.  What are they useful for, besides writting the file package???}



{FnDef {FnName PRETTYDEF} {FnArgs PRTTYFNS PRTTYFILE PRTTYCOMS REPRINTFNS SOURCEFILE CHANGES}
{Text
Writes a symbolic file in {fn PRETTYPRINT} format for loading, using {var FILERDTBL}{index FILERDTBL Var} as its read table.  {fn PRETTYDEF} returns the name of the symbolic file that was created.

{fn PRETTYDEF} operates under a {fn RESETLST} (see {PageRef Fn RESETLST}), so if an error occurs, or a control-D is typed, all files that {fn PRETTYDEF} has opened will be closed, the (partially complete) file being written will be deleted, and any undoable operations executed will be undone.  The {fn RESETLST} also means that any {fn RESETSAVE}s executed in the file package commands will also be protected.

{arg PRTTYFNS} is an optional list of function names.  It is equivalent to including {lisp (FNS * {arg PRTTYFNS})} in the file package commands in {arg PRTTYCOMS}.  {arg PRTTYFNS} is an anachronism from when {fn PRETTYDEF} did not use a list of file package commands, and should be specified as {lisp NIL}.


{arg PRTTYFILE} is the name of the file on which the output is to be written.  If {arg PRTTYFILE}={lisp NIL}, the primary output file is used.  If {arg PRTTYFILE} is atomic the file is opened if not already open, and it becomes the primary output file.  {arg PRTTYFILE} is closed at end of {fn PRETTYDEF}, and the primary output file is restored.  Finally, if {arg PRTTYFILE} is a list, {fn CAR} of {arg PRTTYFILE} is assumed to be the file name, and is opened if not already open.  In this case, the file is left open at end of {fn PRETTYDEF}.

{arg PRTTYCOMS} is a list of file package commands interpreted as described on {PageRef Tag FilePkgComs}.  If {arg PRTTYCOMS} is atomic, its top level value is used and an {index RPAQQ FN}{fn RPAQQ} is written which will set that atom to the list of commands when the file is subsequently loaded.  A {fn PRETTYCOMPRINT} expression (see below) will also be written which informs the user of the named atom or list of commands when the file is subsequently loaded.  In addition, if any of the functions in the file are nlambda functions, {fn PRETTYDEF} will automatically print a {filecom DECLARE:} expression suitable for informing the compiler about these functions, in case the user recompiles the file without having first loaded the nlambda functions (see {PageRef Tag CompilingNLAMBDAs}).

{arg REPRINTFNS} and {arg SOURCEFILE} are for use in conjunction with remaking a file (see {PageRef Tag RemakingFiles}).  {arg REPRINTFNS} can be a list of functions to be prettyprinted, or {atom EXPRS}, meaning prettyprint all functions with {prop EXPR} definitions, or {atom ALL} meaning prettyprint all functions either defined as {lisp EXPR}s, or with {prop EXPR} properties.  Note that doing a remake with {arg REPRINTFNS}={lisp NIL} makes sense if there have been changes in the file, but not to any of the functions, e.g., changes to variables or property lists.  {arg SOURCEFILE} is the name of the file from which to copy the definitions for those functions that are {it not} going to be prettyprinted, i.e., those not specified by {arg REPRINTFNS}.  {arg SOURCEFILE}={lisp T} means to use most recent version (i.e., highest number) of {arg PRTTYFILE}, the second argument to {fn PRETTYDEF}.  If {arg SOURCEFILE} cannot be found, {fn PRETTYDEF} prints the message {lisp "{arg FILE} NOT FOUND, SO IT WILL BE WRITTEN ANEW"}{index NOT FOUND, SO IT WILL BE WRITTEN ANEW Error}, and proceeds as it does when {arg REPRINTFNS} and {arg SOURCEFILE} are both {lisp NIL}.


{fn PRETTYDEF} calls {fn PRETTYPRINT} with its second argument {arg PRETTYDEFLG}={lisp T}, so whenever {fn PRETTYPRINT} starts a new function, it prints (on the terminal) the name of that function if more than 30 seconds (real time) have elapsed since the last time it printed the name of a function.

Note that normally if {fn PRETTYPRINT} is given a litatom which is not defined as a function but is known to be on one of the files noticed by the file package, {fn PRETTYPRINT} will load in the definition (using {fn LOADFNS}) and print it.  This is not done when {fn PRETTYPRINT} is called from {fn PRETTYDEF}.
}}



{FnDef {FnName PRINTFNS} {FnArgs X {anonarg}}
{Text
{arg X} is a list of functions.  {fn PRINTFNS} prettyprints a {lisp DEFINEQ} epression that defines the functions to the primary output stream using the primary read table.  Used by {fn PRETTYDEF} to implement the {filecom FNS} file package command.
}}


{FnDef {FnName PRINTDATE} {FnArgs FILE CHANGES}
{Text
Prints the {lisp FILECREATED} expression at beginning of {fn PRETTYDEF}
files.  {arg CHANGES} used by the file package.
}}


{FnDef {FnName FILECREATED} {FnArgs X}
{Type NLAMBDA NOSPREAD}
{Text
Prints a message (using {fn LISPXPRINT}) followed by the time and date the file was made, which is {lisp (CAR {arg X})}.  The message is the value of {var PRETTYHEADER}{index PRETTYHEADER Var}, initially {lisp "FILE CREATED"}.  If {var PRETTYHEADER}={lisp NIL}, nothing is printed.  {lisp (CDR {arg X})} contains information about the file, e.g., full name, address of file map, list of changed items, etc.  {fn FILECREATED} also stores the time and date the file was made on the property list of the file under the property {prop FILEDATES}{index FILEDATES Prop} and performs other initialization for the file package.

{note should this function be flushed from manual??  what good is this write-up, unless you document the full format of X??}
}}


{FnDef {FnName PRETTYCOMPRINT} {FnArgs X}
{Type NLAMBDA}
{Text
Prints {arg X} (unevaluated) using {fn LISPXPRINT}, unless {var PRETTYHEADER}={lisp NIL}.
}}


{VarDef {Name PRETTYHEADER}
{Text
{index PRETTYHEADER Var}Value is the message printed by {fn FILECREATED}.  {var PRETTYHEADER} is initially {lisp "FILE CREATED"}.
If {var PRETTYHEADER}={lisp NIL}, neither {fn FILECREATED} nor {fn PRETTYCOMPRINT} will print anything.  Thus, setting {var PRETTYHEADER} to {lisp NIL} will result in "silent loads".  {var PRETTYHEADER} is reset to {lisp NIL} during greeting ({PageRef Tag Greeting}).
}}



{FnDef {FnName FILECHANGES} {FnArgs FILE TYPE}
{Text
Returns a list of the changed objects of file package type {arg TYPE} from the {lisp FILECREATED} expression of {arg FILE}.  If {arg TYPE}={lisp NIL}, returns an alist of all of the changes, with the file package types as the {fn CAR}s of the elements..
}}

{FnDef {FnName FILEDATE} {FnArgs FILE {anonarg}}
{Text
Returns the file date contained in the {lisp FILECREATED} expression of {arg FILE}.
}}

{FnDef {FnName LISPSOURCEFILEP} {FnArgs FILE}
{Text
Returns a non-{lisp NIL} value if {arg FILE} is an Interlisp source file, {lisp NIL} otherwise.
}}



{Begin SubSec Copyright Notices}
{Title Copyright Notices}
{Text

The system has a facility for automatically printing a copyright notice near the front of files, right after the {lisp FILECREATED} expression, specifying the years it was edited and the copyright owner.  The format of the copyright notice is:

{lispcode
(* Copyright (c) 1981 by Foo Bars Corporation)}

Once a file has a copyright notice then every version will have a new copyright notice inserted into the file without user intervention.  (The copyright information necessary to keep the copyright up to date is stored at the end of the file.).

Any year the file has been edited is considered a "copyright year" and therefore kept with the copyright information.  For example, if a file has been edited in 1981, 1982, and 1984, then the copyright notice would look like:

{lispcode
(* Copyright (c) 1981,1982,1984 by Foo Bars Corporation)}



When a file is made, if it has no copyright information, the system will ask the user to specify the copyright owner (if {var COPYRIGHTFLG}={lisp T}).  The user may specify one of the names from {var COPYRIGHTOWNERS}, or give one of the following responses:

(1) Type a left-square-bracket.  The system will then prompt for an arbitrary string which will be used as the owner-string

(2) Type a right-square-bracket, which specifies that the user really does not want a copyright notice.

(3) Type "{lisp NONE}" which specifies that this file should never have a copyright notice.

For example, if COPYRIGHTOWNERS has the value

{lispcode
((BBN "Bolt Beranek and Newman Inc.")
 (XEROX "Xerox Corporation"))}

then for a new file {lisp FOO} the following interaction will take place:

{lispcode
Do you want to Copyright FOO? Yes
Copyright owner:   {it (user typed ?)}
one of:
BBN - Bolt Beranek and Newman Inc.
XEROX - Xerox Corporation
NONE - no copyright ever for this file
[ - new copyright owner -- type one line of text
] - no copyright notice for this file now

Copyright owner: BBN}

Then "Foo Bars Corporation" in the above copyright notice example would have been "Bolt Beranek and Newman Inc."



The following variables control the operation of the copyright facility:


{VarDef {Name COPYRIGHTFLG}
{Text
The value of {var COPYRIGHTFLG} determines whether copyright information is maintained in files.  Its value is interpreted as follows:

{Begin Labeledlist COPYRIGHTFLG}

{Label {lisp NIL}}
{Text
The system will preserve old copyright information, but will not ask the user about copyrighting new files.  This is the default value of {var COPYRIGHTFLG}.
}

{Label {lisp T}}
{Text
When a file is made, if it has no copyright information, the system will ask the user to specify the copyright owner.
}

{Label {lisp NEVER}}
{Text
The system will neither prompt for new copyright information nor preserve old copyright information.
}

{Label {lisp DEFAULT}}
{Text
The value of {var DEFAULTCOPYRIGHTOWNER} (below) is used for putting copyright information in files that don't have any other copyright.  The prompt "{lisp Copyright owner for file xx:}" will still be printed, but the default will be filled in immediately.
}

{End Labeledlist COPYRIGHTFLG}

}}


{VarDef {Name COPYRIGHTOWNERS}
{Text
{var COPYRIGHTOWNERS} is a list of entries of the form {lisp ({arg KEY} {arg OWNERSTRING})}, where {arg KEY} is used as a response to {fn ASKUSER} and {arg OWNERSTRING} is a string which is the full identification of the owner.  
}}

{VarDef {Name DEFAULTCOPYRIGHTOWNER}
{Text
If the user does not respond in {var DWIMWAIT} seconds to the copyright
query, the value of {var DEFAULTCOPYRIGHTOWNER} is used.
}}



{Begin Note}
doc the following??

Implementation notes

The copyright information for a file is stored on the property list of the file, under the COPYRIGHT property, which is a list of the form (OWNER year1 year2 ...). To remove all current copyright information about a file, call (REMPROP FILE 'COPYRIGHT).

Copyright information included via the previous <LISPUSERS>COPYRIGHT package will be preserved (i.e., the copyright facility looks at the COPYRIGHTOWNER and COPYRIGHTYEARS property of a file.)

= = = = = = = = = = = = = = = = = = =

In addition, there is available on <LISPUSERS>COPYRIGHT & .COM an 
extension of the copyright facility: if you load in this file, then copyright
notices will be printed as

(*                      Copyright (c) 1981, 1982, 1984
                                    by
                           Larry M. Masinter                                    *)
<<<do you want pointers to lispusers functions here?  Why not make this part of the standard system, selectable with a flg?   ---mjs>>>>>
{End Note}

}{End SubSec Copyright Notices}



{Begin SubSec Functions Used Within Source Files}
{Title Functions Used Within Source Files}
{Text

The following functions are normally only used within symbolic files, to set variable values, property values, etc.  Most of these have special behavior depending on file package variables.

{index Variable bindings}

{FnDef {FnName RPAQ} {FnArgs VAR VALUE}
{Type NLAMBDA}
{Text
An nlambda function like {fn SETQ} that sets the top level binding of {arg VAR} (unevaluated) to {arg VALUE}.

{note isn't RPAQ = nlambda version of SETTOPVAL ??}
}}


{FnDef {FnName RPAQQ} {FnArgs VAR VALUE}
{Type NLAMBDA}
{Text
An nlambda function like {fn SETQQ} that sets the top level binding of {arg VAR} (unevaluated) to {arg VALUE} (unevaluated).
}}


{FnDef {FnName RPAQ?} {FnArgs VAR VALUE}
{Type NLAMBDA}
{Text
Similar to {fn RPAQ}, except that it does nothing if {arg VAR} already has a top level value other than {atom NOBIND}.  Returns {arg VALUE} if {arg VAR} is reset, otherwise {lisp NIL}.
}}


{fn RPAQ}, {fn RPAQQ}, and {fn RPAQ?} generate errors if {arg X} is not a litatom.{index ARG NOT LITATOM Error}  All are affected by the value of {var DFNFLG} ({PageRef Var DFNFLG}).  If {var DFNFLG}={lisp ALLPROP}{index ALLPROP Litatom} (and the value of {arg VAR} is other than {atom NOBIND}), instead of setting {arg X}, the corresponding value is stored on the property list of {arg VAR} under the property {prop VALUE}.  All are undoable.



{FnDef {FnName ADDTOVAR}
{FnArgs VAR X{SUB 1} X{SUB 2} {ellipsis} X{SUB N}}
{Type NOSPREAD NLAMBDA}
{Text
Each {arg X{sub i}} that is not a member of the value of {arg VAR} is added to it, i.e. after {fn ADDTOVAR} completes, the value of {arg VAR} will be {lisp (UNION (LIST {arg X{sub 1}} {arg X{sub 2}} {ellipsis} {arg X{sub N}}) {arg VAR})}.  {fn ADDTOVAR} is used by {fn PRETTYDEF} for implementing the {lisp ADDVARS} command.  It performs some file package related operations, i.e. "notices" that {arg VAR} has been changed.  Returns the atom {arg VAR} (not the value of {arg VAR}).
}}

{FnDef {FnName APPENDTOVAR}
{FnArgs VAR X{SUB 1} X{SUB 2} {ellipsis} X{SUB N}}
{Type NOSPREAD NLAMBDA}
{Text
Similar to {fn ADDTOVAR}, except that the values are added to the end tof the list, rather than at the beginning.
}}

{FnDef {FnName PUTPROPS}
{FnArgs ATM PROP{SUB 1} VAL{SUB 1} {ellipsis} PROP{SUB N} VAL{SUB N}}
{Type NOSPREAD NLAMBDA}
{Text
Nlambda nospread version of {fn PUTPROP} (none of the arguments are evaluated).  For {arg i}={lisp 1{ellipsis}{arg N}}, puts property {arg PROP{sub i}}, value {arg VAL{sub i}}, on the property list of {arg ATM}.  Performs some file package related operations, i.e., "notices" that the corresponding properties have been changed.
}}


{FnDef {FnName SAVEPUT} {FnArgs ATM PROP VAL}
{Text
Same as {fn PUTPROP}, but marks the corresponding property value as having been changed (used by the file package).
}}


}{End SubSec Functions Used Within Source Files}



{Begin SubSec File Maps}
{Title File Maps}
{Text

{Tag FileMaps}

{index *PRIMARY* File maps}

A file map is a data structure which contains a symbolic 'map' of the contents of a file.  Currently, this consists of the begin and end byte address (see {fn GETFILEPTR}, {PageRef Fn GETFILEPTR}) for each {fn DEFINEQ} expression in the file, the begin and end address for each function definition within the {fn DEFINEQ}, and the begin and end address for each compiled function.


{fn MAKEFILE}, {fn PRETTYDEF}, {fn LOADFNS}, {fn RECOMPILE}, and numerous other system functions depend heavily on the file map for efficient operation.  For example, the file map enables {fn LOADFNS} to load selected function definitions simply by setting the file pointer to the corresponding
address using {fn SETFILEPTR}, and then performing a single {fn READ}.
Similarly, the file map is heavily used by the "remake" option of {fn MAKEFILE} ({PageRef Tag RemakingFiles}): those function definitions that have been changed since the previous version are prettyprinted; the rest are simply copied from the old file to the new one, resulting in a considerable speedup.


Whenever a file is written by {fn MAKEFILE}, a file map for the new file is built.  Building the map in this case essentially comes for free, since it requires only reading the current file pointer before and after each definition is written or copied.  However, building the map does require that {fn PRETTYPRINT} {it know} that it is printing a {lisp DEFINEQ} expression.  For this reason, the user should never print a {lisp DEFINEQ} expression onto a file himself, but should instead always use the {filecom FNS} file package command ({PageRef FileCom FNS}).


The file map is stored on the property list of the root name of the file, under the property {prop FILEMAP}.{index *PRIMARY* FILEMAP Prop}  In addition, {fn MAKEFILE} writes the file map on the file itself.  For cosmetic reasons, the file map is written as the last expression in the file.  However, the {it address} of the file map in the file is (over)written into the {lisp FILECREATED} expression that appears at the beginning of the file so that the file map can be rapidly accessed without having to scan the entire file.  In most cases, {fn LOAD} and {fn LOADFNS} do not have to build the file map at all, since a file map will usually appear in the corresponding file, unless the file was written with {var BUILDMAPFLG}={lisp NIL}, or was written outside of Interlisp.



Currently, file maps for {it compiled} files are not written onto the files themselves.  However, {fn LOAD} and {fn LOADFNS} will build maps for a compiled file when it is loaded, and store it on the property {prop FILEMAP}.
Similary, {fn LOADFNS} will obtain and use the file map for a compiled file, when available.



The use and creation of file maps is controlled by the following variables:


{note all reference to GETFILEMAP deleted ...users don't need to know about it, right??}

{VarDef {Name BUILDMAPFLG}
{Text
Whenever a file is read by {fn LOAD} or {fn LOADFNS}, or written by {fn MAKEFILE}, a file map is automatically built unless {var BUILDMAPFLG}={lisp NIL}.{index *PRIMARY* BUILDMAPFLG Var} ({var BUILDMAPFLG} is initially {lisp T}.)


While building the map will not help the first reference to a file, it will help in future references.  For example, if the user performs {lisp (LOADFROM 'FOO)} where {lisp FOO} does not contain a file map, the {fn LOADFROM} will be (slightly) slower than if {lisp FOO} did contain a file map, but subsequent calls to {fn LOADFNS} for this version of {lisp FOO} will be able to use the map that was built as the result of the {fn LOADFROM}, since it will be stored on {lisp FOO}'s {prop FILEMAP} property.
}}



{VarDef {Name USEMAPFLG}
{Text
If {var USEMAPFLG}={lisp T} (the initial setting), the functions that use file maps will first check the {prop FILEMAP} property to see if a file map for this file was previously obtained or built.  If not, the first expression on the file is checked to see if it is a {lisp FILECREATED} expression that also contains the address of a file map.  If the file map is not on the {prop FILEMAP} property or in the file, a file map will be built (unless {var BUILDMAPFLG}={lisp NIL}).

If {var USEMAPFLG}={lisp NIL}, the {prop FILEMAP} property and the file will not be checked for the file map.  This allows the user to recover in those cases where the file and its map for some reason do not agree.  For example, if the user uses a text editor to change a symbolic file that contains a map (not recommended), inserting or deleting just one character will throw that map off.  The functions which use file maps contain various integrity checks to enable them to detect that something is wrong, and to generate the error {lisp FILEMAP DOES NOT AGREE WITH CONTENTS OF {arg FILE}}{index FILEMAP DOES NOT AGREE WITH CONTENTS OF Error}.  In such cases, the user can set {var USEMAPFLG} to {lisp NIL}, causing the map contained in the file to be ignored, and then reexecute the operation.
}}


{note put in index to error message "rewriting file map...."}



}{End SubSec File Maps}



}{End SubSec Symbolic File Format}