{Begin SubSec Macro Expansion}
{Title Macro Expansion}
{Text

{index *BEGIN* Macro Expansion (in Masterscope) Term}

As part of analysis, Masterscope will expand the macro definition of called functions, if they are not otherwise defined (see {PageRef Tag Macros}).  Masterscope macro expansion is controlled by the variable {var MSMACROPROPS}:

{VarDef {Name MSMACROPROPS}
{Text
Value is an ordered list of macro-property names that Masterscope will search to find a macro definition.  Only the kinds of macros that appear on {var MSMACROPROPS} will be expanded.  All others will be treated as function calls and left unexpanded.

Initially {lisp (MACRO)}.

Note:  {var MSMACROPROPS} initially contains only {lisp MACRO} (and not {lisp 10MACRO}, {lisp DMACRO}, etc.) in the theory that the machine-dependent macro definitions are more likely "optimizers".
}}


Note that if you edit a macro, Masterscope will know to reanalyze the functions which call that macro. However, if your macro is of the "computed-macro" style, and it calls functions which you edit, Masterscope will not notice.  You must be careful to tell masterscope to {lisp REANALYZE} the appropriate functions (e.g., if you edit {lisp FOOEXPANDER} which is used to expand {lisp FOO} macros, you have to {lisp . REANALYZE ANY CALLING FOO}.

{index *END* Macro Expansion (in Masterscope) Term}


}{End SubSec Macro Expansion}





{Begin SubSec Affecting Masterscope Analysis}
{Title Affecting Masterscope Analysis}
{Text

{index *BEGIN* Templates (in Masterscope) Term}

{Tag Templates}

Masterscope analyzes the {lisp EXPR} definitions of functions and notes in its database the relations that function has with other functions and with variables.  To perform this analysis, Masterscope uses {it templates} which describe the behavior of functions.  For example, the information that {fn SORT} {lisp SMASH}es its first argument is contained in the template for {lisp SORT}.  Masterscope initially contains templates for most system functions which set variables, test their arguments, or perform destructive operations.


A template is a list structure containing any of the following atoms:

{note better defn of a template needed!}


{Def {Type (in Masterscope template)}  {Name PPE}
{Text
If an expression appears in this location, there is most likely a parenthesis error.

Masterscope notes this as a "call" to the function "{lisp ppe}" (lowercase).{index ppe (used in Masterscope)}  Therefore, {lisp SHOW WHERE ANY CALLS ppe} will print out all possible parenthesis errors.  When Masterscope finds a possible parenthesis error in the course of analyzing a function definition, rather than printing the usual "{lisp .}", it prints out a "{lisp ?}" instead.{index ? (printed by Masterscope)}
}}


{Def {Type (in Masterscope template)}  {Name NIL}
{Text
The expression occuring at this location is not evaluated.
}}


{Def {Type (in Masterscope template)}  {Name SET}
{Text
A variable appearing at this place is set.
}}


{Def {Type (in Masterscope template)}  {Name SMASH}
{Text
The value of this expression is smashed.
}}


{Def {Type (in Masterscope template)}  {Name TEST}
{Text
This expression is used as a predicate (that is, the only use of the value of the expression is whether it is {lisp NIL} or non-{lisp NIL}).
}}


{Def {Type (in Masterscope template)}  {Name PROP}
{Text
The value of this expression is used as a property name.  If the expression is of the form {lisp (QUOTE {arg ATOM})}, Masterscope will note that {arg ATOM} is {lisp USED AS A PROPERTY NAME}.  For example, the template for {fn GETPROP} is {lisp (EVAL PROP . PPE)}.
}}


{Def {Type (in Masterscope template)}  {Name FUNCTION}
{Text
The expression at this point is used as a functional argument.
For example, the template for {lisp MAPC} is {lisp (SMASH FUNCTION FUNCTION . PPE)}.
}}


{Def {Type (in Masterscope template)}  {Name FUNCTIONAL}
{Text
The expression at this point is used as a functional argument.  This is like {lisp FUNCTION}, except that Masterscope distinguishes between functional arguments to functions which "compile open" from those that do not.  For the latter (e.g. {lisp SORT} and {lisp APPLY}), {lisp FUNCTIONAL} should be used rather than {lisp FUNCTION}.
}}



{Def {Type (in Masterscope template)}  {Name EVAL}
{Text
The expression at this location is evaluated (but not set, smashed, tested, used as a functional argument, etc.).
}}


{Def {Type (in Masterscope template)}  {Name RETURN}
{Text
The value of the function (of which this is the template) is the value of this expression.
}}


{Def {Type (in Masterscope template)}  {Name TESTRETURN}
{Text
A combination of {lisp TEST} and {lisp RETURN}:  If the value of the function is non-{lisp NIL}, then it is returned. For instance, a one-element {lisp COND} clause is this way.
}}


{Def {Type (in Masterscope template)}  {Name EFFECT}
{Text
The expression at this location is evaluated, but the value is not used.
}}


{Def {Type (in Masterscope template)}  {Name FETCH}
{Text
An atom at this location is a field which is fetched.
}}


{Def {Type (in Masterscope template)}  {Name REPLACE}
{Text
An atom at this location is a field which is replaced.
}}


{Def {Type (in Masterscope template)}  {Name RECORD}
{Text
An atom at this location is used as a record name.
}}


{Def {Type (in Masterscope template)}  {Name CREATE}
{Text
An atom at this location is a record which is created.
}}


{Def {Type (in Masterscope template)}  {Name BIND}
{Text
An atom at this location is a variable which is bound.
}}


{Def {Type (in Masterscope template)}  {Name CALL}
{Text
An atom at this location is a function which is called.
}}


{Def {Type (in Masterscope template)}  {Name CLISP}
{Text
An atom at this location is used as a CLISP word.
}}



{Def {Type (in Masterscope template)}  {Name !}
{Text
This atom, which can only occur as the first element of a template, allows one to specify a template for the {fn CAR} of the function form.  If {lisp !} doesn't appear, the {fn CAR} of the form is treated as if it had a {lisp CALL} specified for it.  In other words, the templates {lisp (.. EVAL)} and {lisp (! CALL .. EVAL)} are equivalent.

If the next atom after a {lisp !} is {lisp NIL}, this specifies that the function name should not be remembered.  For example, the template for {fn AND} is {lisp (! NIL .. TEST RETURN)}, which means that if you see an "{fn AND}", don't remember it as being called.  This keeps the Masterscope database from being cluttered by too many uninteresting relations; Masterscope also throws away relations for {fn COND}, {fn CAR}, {fn CDR}, and a couple of others.
}}



In addition to the above atoms which occur in templates, there are some "special forms" which are lists keyed by their {fn CAR}.



{Def {Type (in Masterscope template)}
{Name ..} {Args TEMPLATE} {NoParens}
{Text
Any part of a template may be preceded by the atom {lisp ..} (two periods) which specifies that the template should be repeated an indefinite number
({arg N}{GE}0) of times to fill out the expression.  For example, the template for {lisp COND} might be {lisp (.. (TEST .. EFFECT RETURN))} while the template for {lisp SELECTQ} is {lisp (EVAL .. (NIL .. EFFECT RETURN) RETURN)}.
}}


{Def {Type (in Masterscope template)}
{Name BOTH} {Args TEMPLATE{sub 1} TEMPLATE{sub 2}}
{Text
Analyze the current expression twice, using the each of the templates in turn.
}}


{Def {Type (in Masterscope template)}
{Name IF} {Args EXPRESSION TEMPLATE{sub 1} TEMPLATE{sub 2}}
{Text
Evaluate {arg EXPRESSION} at analysis time (the variable {var EXPR}{index EXPR Var} will be bound to the expression which corresponds to the {lisp IF}), and if the result is non-{lisp NIL}, use {arg TEMPLATE{sub 1}}, otherwise {arg TEMPLATE{sub 2}}.  If {arg EXPRESSION} is a literal atom, it is {fn APPLY}'d to {var EXPR}.  For example, {lisp (IF LISTP (RECORD FETCH) FETCH)} specifies that if the current expression is a list, then the first element is a record name and the second element a field name, otherwise it is a field name.  
}}


{Def {Type (in Masterscope template)}
{Name @} {Args EXPRFORM TEMPLATEFORM}
{Text
Evaluate {arg EXPRFORM} giving {arg EXPR}, evaluate {arg TEMPLATEFORM} giving {arg TEMPLATE}.  Then analyze {arg EXPR} with {arg TEMPLATE}.  {lisp @} lets the user compute on the fly both a template and an expression to analyze with it.  The forms can use the variable {var EXPR}{index EXPR Var}, which is bound to the current expression.
}}


{Def {Type (in Masterscope template)}
{Name MACRO} {Args {lisp .} MACRO}
{Text
{arg MACRO} is interpreted in the same way as a macro (see {PageRef Tag Macros}) and the resulting form is analyzed.  If the template is the atom {lisp MACRO} alone, Masterscope will use the {prop MACRO} property of the function itself.  This is useful when analyzing code which contains calls to user-defined macros.  If the user changes a macro property (e.g. by editing it) of an atom which has template of {lisp MACRO}, Masterscope will mark any function which used that macro as needing to be reanalyzed.
}}



Some examples of templates:


{Begin LabeledList examples}

{Label function}		{Item template}

{Label {lisp DREVERSE}}	{Item {lisp (SMASH . PPE)}}

{Label {lisp AND}}	{Item {lisp (! NIL TEST .. RETURN)}}

{Label {lisp MAPCAR}}	{Item {lisp (EVAL FUNCTION FUNCTION)}}

{Label {lisp COND}}	{Item {lisp (! NIL .. (IF CDR (TEST .. EFFECT RETURN) (TESTRETURN . PPE)))}}

{End LabeledList examples}



{Note unexplained template atoms: STACK  EVALQT}




Templates may be changed and new templates defined using the functions:


{FnDef {Name GETTEMPLATE} {Args FN}
{Text
Returns the current template of {arg FN}.
}}



{FnDef {Name SETTEMPLATE} {Args FN TEMPLATE}
{Text
Changes the template for the function {arg FN} and returns the old value.  If any functions in the database are marked as calling {arg FN}, they will be marked as needing re-analysis.
}}



{Begin Note}
Where should this go????   ---mjs

---For now, I think we want to keep this under our hats, until and unless Larry wants to provide public interfaces to the world---JDS 7/15/83

Date: 21 NOV 1979 0057-PST
From: KAPLAN

I added a hook to the masterscope function analyzer so that the user could specify his own computations.

After a function has been analyzed, masterscope looks at the variable ANALYZEUSERFNS.  If it is non-nil, it is assumed to be a list of functions.  Each of them is applied in turn to the function-name, function-definition, and analysis data list for the function just analyzed.  The function should return a new data list (or perhaps just the old one if all that is intened is side-effecting).

This is the hook that is needed to add various user defined relations to masterscope.  The MSHASH package will be able to store the argument list of a function so that DESCRIBE won't have to rummage around for it if the function is not loaded.  Also, DECL will be able to store the RETURNS declaration for a function so that inter-function declarations can be checked.
{End Note}


{index *END* Templates (in Masterscope) Term}

}{End SubSec Affecting Masterscope Analysis}



{Begin SubSec Data Base Updating}
{Title Data Base Updating}
{Text

Masterscope is interfaced to the editor and file package so that it notes whenever a function has been changed, either through editing or loading in a new definition.  Whenever a command is given which requires knowing the information about a specific function, if that function has been noted as being changed, the function is automatically re-analyzed before the command is interpreted.  If the command requires that all the information in the database be consistent (e.g., the user asks {lisp WHO CALLS X)} then {it all} functions which have been marked as changed are re-analyzed.

}{End SubSec Data Base Updating}



{Begin SubSec Masterscope Entries}
{Title Masterscope Entries}
{Text


{FnDef {Name CALLS} {Args FN USEDATABASE {anonarg}}
{Text
{arg FN} can be a function name, a definition, or a form.  Note: {fn CALLS} will also work on compiled code.  {fn CALLS} returns a list of four elements: a list of all the functions called by {arg FN},{foot
Functions called via "linked" calls from compiled code are indicated by semicolons {fn PACK}ed around their name; e.g.
{lisp (CALLS 'MASTERSCOPE)} might return
{lisp ((;MASTERSCOPEXEC; ;MSINTERPRET; ;PRINT; HELP) --)}.
This feature can be suppressed by setting {var NOPACKCALLSFLG}{index NOPACKCALLSFLG Var} to {lisp T}.
}{comment endfootnote}
a list of all the variables bound in {arg FN}, a list of all the variables used freely in {arg FN}, and a list of the variables used globally in {arg FN}.
For the purpose of {fn CALLS}, variables used freely which are on {var GLOBALVARS} or have a property {prop GLOBALVAR} value {lisp T} are considered to be used globally.  If {arg USEDATABASE} is {lisp NIL} (or {arg FN} is not a litatom), {fn CALLS} will perform a one-time analysis of {arg FN}.  Otherwise (i.e. if {arg USEDATABASE} is non-{lisp NIL} and {arg FN} a function name), {fn CALLS} will use the information in Masterscope's database ({arg FN} will be analyzed first if necessary).
}}



{FnDef {Name CALLSCCODE} {Args FN {anonarg}}
{Text
The sub-function of {fn CALLS} which analyzes compiled code.  {fn CALLSCCODE} returns a list of {it five} elements: a list of all the functions called via "linked" function calls, a list of all functions called regularly, a list of variables bound in {arg FN}, a list of variables used freely, and a list of variables used globally.
}}



{FnDef {Name FREEVARS} {Args FN USEDATABASE}
{Text
Equivalent to {lisp (CADDR (CALLS {arg FN} {arg USEDATABASE}))}.
Returns the list of variables used freely within {arg FN}.
}}



{FnDef {Name MASTERSCOPE} {Args COMMAND {anonarg}}
{Text
Top level entry to Masterscope.  If {arg COMMAND} is {lisp NIL}, will enter into a {fn USEREXEC} in which the user may enter commands.  If {arg COMMAND} is not {lisp NIL}, the command is interpreted and {fn MASTERSCOPE} will return the value that would be printed by the command. Note that only the question commands return meaningful values. 
}}



{FnDef {Name SETSYNONYM} {Args PHRASE MEANING {anonarg}}
{Text
Defines a new synonym for Masterscope's parser.  Both {arg PHRASE} and {arg MEANING} are words or lists of words; anywhere {arg PHRASE} is seen in a command, {arg MEANING} will be substituted.  For example, {lisp (SETSYNONYM 'GLOBALS '(VARS IN GLOBALVARS OR @(GETPROP X 'GLOBALVAR)))} would allow the user to refer with the single word {lisp GLOBALS} to the set of variables which are either in {var GLOBALVARS} or have a {prop GLOBALVAR} property.
}}



The following functions are provided for users who wish to write their own routines using Masterscope's database:


{FnDef {Name PARSERELATION} {Args RELATION}
{Text
{arg RELATION} is a relation phrase; e.g., {lisp (PARSERELATION '(USE FREELY))}.  {fn PARSERELATION} returns an internal representation for {arg RELATION}.  For use in conjunction with {fn GETRELATION}.
}}



{FnDef {Name GETRELATION} {Args ITEM RELATION INVERTED}
{Text
{arg RELATION} is an internal representation as returned by {fn PARSERELATION} (if not, {fn GETRELATION} will first perform {lisp (PARSERELATION {arg RELATION})}); {arg ITEM} is an atom.  {fn GETRELATION} returns the list of all atoms which have the given relation to {arg ITEM}.  For example, {lisp (GETRELATION 'X '(USE FREELY))} returns the list of variables that {lisp X} uses freely.  If {arg INVERTED} is {lisp T}, the inverse relation is used; e.g. {lisp (GETRELATION 'X '(USE FREELY) T)} returns the list of functions which use {lisp X} freely.

If {arg ITEM} is {lisp NIL}, {fn GETRELATION} will return the list of atoms which have {arg RELATION} with {it any} other item; i.e., answers the question {lisp WHO {arg RELATION}S ANY}.  Note that {fn GETRELATION} does {it not} check to see if {arg ITEM} has been analyzed, or that other functions that have been changed have been re-analyzed.
}}



{FnDef {Name TESTRELATION} {Args ITEM RELATION ITEM2 INVERTED}
{Text
equivalent to {lisp (MEMB {arg ITEM2} (GETRELATION {arg ITEM} {arg RELATION} {arg INVERTED}))}, that is, tests if {arg ITEM} and {arg ITEM2} are related via {arg RELATION}.  If {arg ITEM2} is {lisp NIL}, the call is equivalent to {lisp (NOT (NULL (GETRELATION {arg ITEM} {arg RELATION} {arg INVERTED})))}, i.e., {fn TESTRELATION} tests if {arg ITEM} has the given {arg RELATION} with {it any} other item.
}}



{FnDef {Name MAPRELATION} {Args RELATION MAPFN}
{Text
Calls the function {arg MAPFN} on every pair of items related via {arg RELATION}.  If {lisp (NARGS {arg MAPFN})} is 1, then {arg MAPFN} is called on every item which has the given {arg RELATION} to {it any} other item.
}}



{FnDef {Name MSNEEDUNSAVE} {Args FNS MSG MARKCHANGEFLG}
{Text
Used to mark functions which depend on a changed record declaration (or macro, etc.), and which must be LOADed or UNSAVEd (see below).  {arg FNS} is a list of functions to be marked, and {arg MSG} is a string describing the records, macros, etc. on which they depend.  If {arg MARKCHANGEFLG} is non-NIL, each function in the list is marked as needing re-analysis.
}}



{FnDef {Name UPDATEFN} {Args FN EVENIFVALID {anonarg}}
{Text
Equivalent to the command {lisp ANALYZE '{arg FN}}; that is, {fn UPDATEFN} will analyze {arg FN} if {arg FN} has not been analyzed before or if it has been changed since the time it was analyzed.  If {arg EVENIFVALID} is set{note =non-NIL???}, {fn UPDATEFN} will re-analyze {arg FN} even if Masterscope thinks it has a valid analysis in the database.
}}



{FnDef {Name UPDATECHANGED} {Args }
{Text
Performs {lisp (UPDATEFN {arg FN})} on every function which has been marked as changed.
}}



{FnDef {Name MSMARKCHANGED} {Args FN TYPE REASON}
{Text
Mark that {arg FN} has been changed and needs to be reanalyzed.  See {fn MARKASCHANGED}, {PageRef Fn MARKASCHANGED}.
}}



{FnDef {Name DUMPDATABASE} {Args FNLST}
{Text
Dumps the current Masterscope database on the current output file in a {fn LOAD}able form.  If {arg FNLST} is not {lisp NIL}, {fn DUMPDATABASE} will only dump the information for the list of functions in {arg FNLST}.  The variable {var DATABASECOMS}{index DATABASECOMS Var} is initialized to {lisp ((E (DUMPDATABASE)))}; thus, the user may merely perform {lisp (MAKEFILE 'DATABASE.{arg EXTENSION})} to save the current Masterscope database.  If a Masterscope database already exists when a {lisp DATABASE} file is loaded, the database on the file will be merged with the one in core.  Note that functions whose definitions are different from their definition when the database was made must be {lisp REANALYZE}d if their new definitions are to be noticed.

The Databasefns package ({PageRef Tag Databasefns}) provides a more convenient way of saving data bases along with the source files which they correspond to.
}}


}{End SubSec Masterscope Entries}



{Begin SubSec Noticing Changes that Require Recompiling}
{Title Noticing Changes that Require Recompiling}
{Text

When a record declaration, iterative statement operator or macro is changed, and Masterscope has "noticed" a use of that declaration or macro (i.e. it is used by some function known about in the data base), Masterscope will alert the user about those functions which might need to be re-compiled (e.g. they do not currently have {lisp EXPR} definitions).{foot
Extra functions may be noticed; for example if {lisp FOO} contains {lisp (fetch (REC X) --)}, and some declaration other than {lisp REC} which contains {lisp X} is changed, Masterscope will still think that {lisp FOO} needs to be loaded/unsaved.
}{comment endfootnote}
The functions which need recompiling are added to the list {var MSNEEDUNSAVE}{index MSNEEDUNSAVE Var} and a message is printed out: 

{lispcode
The functions {arg FN1}, {arg FN2},... use macros which have changed.
Call UNSAVEFNS() to load and/or unsave them.}


In this situation, the following function is useful:


{FnDef {Name UNSAVEFNS} {Args {anonarg}}
{Text
Uses {fn LOADFNS} or {fn UNSAVEDEF} to make sure that all functions in the list {fn MSNEEDUNSAVE} have {lisp EXPR} definitions, and then sets {var MSNEEDUNSAVE} to {lisp NIL}.
}}



}{End SubSec Noticing Changes that Require Recompiling}



{Begin SubSec Implementation Notes}
{Title Implementation Notes}
{Text

Masterscope keeps a database of the relations noticed when functions are analyzed. The relations are intersected to form "primitive relationships" such that there is little or no overlap of any of the primitives.  For example, the relation {lisp SET} is stored as the union of {lisp SET LOCAL} and {lisp SET FREE}.  The {lisp BIND} relation is divided into {lisp BIND AS ARG}, {lisp BIND AND NOT USE}, and {lisp SET LOCAL}, {lisp SMASH LOCAL}, etc.  Splitting the relations in this manner reduces the size of the database considerably, to the point where it is reasonable to maintain a Masterscope database for a large system of functions during a normal debugging session.


Each primitive relationship is stored in a pair of hash-tables, one for the "forward" direction and one for the "reverse".  For example, there are two hash tables, {lisp USE AS PROPERTY} and {lisp USED AS PROPERTY}.  To retrieve the information from the database, Masterscope performs unions of the hash-values.  For example, to answer {lisp FOO BINDS WHO} Masterscope will look in all of the tables which make up the {lisp BIND} relation.  The "internal representation" returned by {fn PARSERELATION} is just a list of dotted pairs of hash-tables.  To perform {fn GETRELATION} requires only mapping down that list, doing {fn GETHASH}'s on the appropriate hash-tables and {fn UNION}ing the result.


Hash tables are used for a variety of reasons: storage space is smaller; it is not necessary to maintain separate lists of which functions have been analyzed (a special table, {lisp DOESN'T DO ANYTHING} is maintained for functions which neither call other functions nor bind or use any variables); and accessing is relatively fast. Within any of the tables, if the hash-value would be a list of one atom, then the atom itself, rather than the list, is stored as the hash-value. This also reduces the size of the database significantly.

{note what are the real names of these hash tables (they aren't really called USE AS PROPERTY, USED AS PROPERTY, and DOESN'T DO ANYTHING, are they?)  do we want to document these names? ---mjs}

}{End SubSec Implementation Notes}