[phylum]<3-LISP>course>NOTES>lecture-11.notes!2

Page Numbers: Yes X: 306 Y: 1.0" First Page: 1
Margins: Top: 1.0" Bottom: 1.3"
Heading:

LECTURE NOTES #11LISP: LANGUAGE AND LITERATUREMay 17, 1984
————————————————————————————————————————————

Lecture Notes #11 — Quotation and Structural Abstraction

Filed as:[phylum]<3-lisp>course>notes>Lecture-11.notes
User.cm:[phylum]<BrianSmith>system>user.classic
Last edited:May 17, 1984 1:35 PM

————————————————————————————————————————————

A. Introductory Notes

—Still no problem set #2!

—A draft is done, but the TA’s are working through it first. Will definitely be ready on Monday.

—So have a relaxed weekend; things will start up in earnest at the beginning of next week.

—Be encouraged to play with the machines; the definition of OBJECT, if you want to use it, is on both of:

{turing}<3-lisp.utilities>object.3-lisp
{phylum}<3-lisp>course>utilities>object.3-lisp

—Today: Quotation and processing:

—Have talked so far about various kinds of abstraction:

—The use of higher-order functions and higher-order procedures

—Compound objects to represent various kinds of complex objects

—Control abstractions (last time)

—Have presented enough basic machinery to enable people to go off and program for a year or two.

—There are three more things that we will want to examine before the class is over:

—Meta-linguistic techniques (today)

—Explicit theories of how it all works

—Implementation and other architectures

—Today will focus on the first of these: meta-linguistic and "meta-structural" abstractions: dealing with quotation, macros, so-called "meta-circular processors", etc.

—Will pick up on the "internal account" presented in lectures 4 and 5.

—This first task is a pre-requisite for the other two, since

—what the semantics needs to describe, and

—how this (or any other) implementation works

depend on

—the explicit encoding of information (in procedures, programs, whatever)

which is, in essence, what quotation and meta-structural techniques deal with.

—Things this will enable us to do:

—discharge the debt taken out when we introduced OBJECT.

—provide general macro definitions.

—define things like IF*, which we couldn’t do at the time of the first problem set.

—explain, in much clearer detail, how the 3-LISP processor works.

—Should read Abelson and Sussman Chapter 4.

B. Quotation and Structures

—Quotation: what it is (rigid designation of other linguistic expressions)

—Different kinds:

—notational (designators of characters and character-strings)

—structural (designation of internal structures)

—Deal primarily with structural quotation

—But important to see that there is a little algebra of all this stuff.

—Review the various structure types, and operations defined over them.

—Structures introduced to designate structure and notation:

—Handles: for structural quotation;

—Stringers and charats: for notational quotation

—Note both are normal-form; charats and handles are canonical.

—Notations used to notate the handles etc.

—notation n1 for a structure s1 that designates structure s2

—notation n1 for a structure s1 that designates notation n2

—etc.

—Figs 1, 2, and 3:

—Operations that help mediate Q and F:

—INTERNALIZE and EXTERNALIZE

—ATOM-NOTATED and ATOM-NOTATION

—For example:

1(internalize "2")
1= ’2

1> (internalize "(+ x y)")
1= ’(+ x y)

1> (internalize " (first ; this is a comment
;;;
;;; and some more
;;;

x)")
1= ’(first x)

—F(INTERNALIZE) = Q

—PCONS, RCONS, etc. (ACONS)

—FIRST and REST defined (polymorphic), PPROC and PARGS, etc.

C. Macros

—Quotation: what it is (rigid designation of other linguistic expressions)

—two kinds (as you might expect):

—notational macros

—‘↑exp’, for example, is a notational abbreviation for ‘(UP exp)’

—Also: ‘↑exp’ for ‘(DOWN exp)’

—Also: ‘(e1 e2 ... ek)’ for ‘(e1 . [e2 ... ek])’

—structural macros

—to motivate all of this, plunge in to structural macros:

—a new kind of procedure-defining operator: MLAMBDA.

—defines a procedure that computes a function from structures to structures, that is used in a particular way:

—if F signifies (nb!) a procedure P, then normalizing (F . args) will first compute P((F . args)), and then normalize the result.

—clear as mud; so an example:

1(define INCREMENT
(mlambda [call]
(pcons ’+ (rcons ’1 (pargs call)))))

—don’t despair; will make this much clearer in a moment. But first notice how it works:

—In processing (INCREMENT X), we run the procedure associated with INCREMENT, giving it as argument the full structure (remember, these are structural macros) (INCREMENT X).

—The procedure is just as if it were defined using:

1(lambda [call]
(pcons ’+ (rcons ’1 (pargs call))))

So, we have the equivalent of:

((lambda [call]
(pcons ’+ (rcons ’1 (pargs call))))
’(increment x))

Which, through a series of normalization steps, would yield:

(pcons ’+ (rcons ’1 (pargs ’(increment x))))

(pcons ’+ (rcons ’1 ’[x]))

(pcons ’+ ’[1 x]))

’(+ 1 x)

This is just what we wanted.

—But of course ’(+ 1 x) isn’t what (INCREMENT X) should normalize to; rather, it is the structure that we want normalized in place of (INCREMENT X). So the 3-LISP processor, as soon as it obtains the ’(+ 1 X), then normalizes the result of running the structure←structure procedure.

—I.e., it then normalizes (+ 1 X), which (supposing X were bound to 3), would yield 4.

—What happened to the quote? To be more precise, how did the quote appear on the (INCREMENT X) when it was used as the argument to the procedure associated with INCREMENT?

—Answer: macro processing — structure to structure mappings like this — are inherently meta-structural; that’s what you want.

—I.e., if you declare

‘CSLI’ is an abbreviation for ‘The Center for the Study of Language and Information’.

and then you say

I went over to CSLI the other day.

Then a person, in reacting to your comment, has to shift levels, do the substitution of the longer name for the abbreviation (which requires mentioning the abbreviation, the longer name, and in general the sentence as a whole), and then shifts back down again to "use" the expanded sentence.

—Note, in other words, that there is a shift-up and then a shift-down, all very smoothly integrated into what is going on.

—Similar shift of levels in 3-LISP in dealing with macros.

—Go back to the definition of INCREMENT:

1(define INCREMENT
(mlambda [call]
(pcons ’+ (rcons ’1 (pargs call)))))

—A more interesting example. Want to define an IF* then enables us to use explicit "then" and "else". I.e., would like to be able to write:

1(if* (= 1 2)
then (+ 10 (factorial ... ))
else [( ... )])

—Straighforward enough:

1(define IF*
(mlambda [call]
(pcons ’if
(rcons (first (pargs call))
(third (pargs call))
(fifth (pargs call))))))

—Perhaps clearer:

1(define IF*
(mlambda [call]
(let [[premise (first (pargs call))]
[c1 (third (pargs call))]
[c2 (fifth (pargs call))]]
(pcons ’if (rcons premise c1 c2))))))

—We would then write (using ‘W>’ to mean "expands into"):

1(if* e1 then e2 else e3) W> (if e1 e2 e3)

—For example:

11> (define CAREFUL-DIVISION
(lambda [dividend divisor]
(if* (= divisor 0)
then "can’t divide by zero"
else (/ dividend divisor))))
1= ’careful-division
1> (careful-division 12 3)
1= 4
1> (careful-division 12 0)
1= "can’t divide by zero"

—Perhaps we should do some error checking:

1(define IF*
(mlambda [call]
(let [[premise (first (pargs call))]
[noise-1 (second (pargs call))]
[c1 (third (pargs call))]
[noise-2 (fourth (pargs call))]
[c2 (fifth (pargs call))]]
(if (and (= noise-1 ’then)
(= noise-1 ’else))
(pcons ’if
(rcons premise c1 c2))
(error "Malformed IF*" call)))))

—Note that this extension to the language — using keywords in the midst of expressions — is a real wrench to the basic 3-LISP style; probably not the greatest macro to define, but it is simple.

—Can simplify it even more:

1(define ARG
(lambda [n pair]
(nth n (pargs pair))))

—enables:

1(define IF*
(mlambda [call]
(pcons ’if (rcons (arg 1 call)
(arg 3 call)
(arg 5 call)))))

—But even this is pretty painful. What we would really like are something like Quine’s corner quotes:

1(define IF*
(mlambda [call]
’(if (arg 1 call) (arg 3 call) (arg 5 call))))

—where the italicized structures are not meant to be quoted themselves, but rather are taken as naming structures that are constituents of the overall quoted structure,

—I.e., where the italicized structures are somehow meta-structural comments on the code.

—In fact, can do this:

—extend 3-LISP notation to include a comma. Basic idea is that a comma, before a notation within the scope of a single quote mark (i.e., within a handle-notation), means that the immediately following structure is not part of the structure being quoted, but rather is a structure that designates the part of the structure to be quoted.

—I.e.,:

1(define IF*
(mlambda [call]
’(if ,(arg 1 call)
,(arg 3 call)
,(arg 5 call))))

—This works fine.

—Indeed, macro definition would be pretty horendous if it weren’t for this facility.

—Notational quotation principle (maual page 9):

—A notational expression E1 preceded by a single quote mark will notate a structure S1 that designates a structure S2 that would be notated by E1, with the exception that those fragments of S2 that would have been notated by portions of E1 that are preceded by a comma will be designated by the structures that those portions notate, rather than notated by them directly.

—What structure these extended notations notate is complex, but you by and large want to ignore that. For example, the expression

‘ ’(/ ,x (if (= ,y 0)
1
,y)) ’

notates the structure

(pcons ’/
(rcons x
(pcons ’if
(rcons (pcons ’=
(rcons y ’0))
’1
y))))

which you wouldn’t want to have to type too often.

—Some more examples:

11> (set X ’(= 2 3))
1= ’(= 2 3)

1> ’(if X A B)
1= ’(if X
A
B)

1> ’(if ,X A B)
1= ’(if (+ 2 3)
A
B)

—Note that the time at which the comma’ed portions are processed, to determine the designation (i.e., when they are used) is not at the time that the notation was internalized, but rather at the time that the internalized structure is itself used.

—Note the following definition of OBJECT we were using last time:

(define OBJECT
(letrec [[define-message
(lambda [name]
’(define ,name
(lambda args
(((first args) ,↑name) . (rest args)))))]]
(mlambda [call]
(letseq [[state-vars (arg 1 call)]
[inited-vars (arg 2 call)]
[pairs (tail 2 (pargs call))]
[fun-names (map (lambda [pair] (acons)) pairs)]]
’(begin
,(map (lambda [pair]
(define-message (first pair)))
pairs)
(lambda ,state-vars
(let ,inited-vars
(letrec ,(map (lambda [pair fun-name]
’[,fun-name ,(second pair)])
pairs
fun-names)
(lambda [message]
(cond . ,(map (lambda [pair fun-name]
’[(= message ,↑(first pair))
,fun-name])
pairs
fun-names)))))))))))

which converts structures of the form:

(object [<init-var-1> ... <init-var-k>]
[[<var-1> <init-1>] ... [<var-n> <init-n>]]
[<mess1> <fun1>]
[<mess2> <fun2>]
...
[<messj> <funj>])

into:

(begin [(define-message <mess1>)
(define-message <mess2>)
...
(define-message <messj>)]
(lambda [init-var-1 ... init-var-k]
(let [[<var-1> <init-1>]
...
[<var-n> <init-n>]]
(letrec [<new-name-1> <fun1>]
...
[<new-name-j> <funj>]]
(lambda [message]
(cond [(= message ’<mess1>) <new-name-1>]
[(= message ’<mess2>) <new-name-2>]
...
[(= message ’<messj>) <new-name-j>]))))))

—For example:

1(object [balance] []
[WITHDRAW (lambda [amount]
(if (< balance amount)
(error "Insufficient funds")
(begin (set balance (- balance amount))
balance)))]
[DEPOSIT (lambda [amount]
(begin (set balance (+ balance amount))
balance))])

would be converted into:

1(begin [(define-message WITHDRAW)
(define-message DEPOSIT)]
(lambda [balance]
(let []
(letrec [[{atom 1} (lambda [amount]
(if (< balance amount)
(error "Insufficient funds")
(begin (set balance (- balance amount))
balance)))]
[{atom 2} (lambda [amount]
(begin (set balance (+ balance amount))
balance))]]
(lambda [message]
(cond [(= message ’withdraw) {atom 1}]
[(= message ’deposit) {atom 2}]))))))

D. Meta-Circular Processors

—In lecture 5, we presented the following description of how the 3-LISP processor worked (it was a function of structures and environments):

1 —If s1 is in normal-form, then s1.

—If s1 is an atom, then the binding of s1 in e1.

—If s1 is a rail, then the rail of the same length consisting of the results of normalizing (in e1) each of the elements of s1.

—If s1 is a pair, then:

—Let proc1 be the result of normalizing the PPROC part of s1.

—If proc1 is simple, then:

—let args1 be the result of normalizing the PARGS part of s1.

—If proc1 is primitive, then just go and do the primitive operations to args1, returning the result.

—Otherwise, normalize the body of proc1, in an environment that results from:

—extending the environment represented inside proc1 by:

—binding the atoms in the pattern encoded inside proc1 to args1.

Otherwise (i.e., if it is special):

—If proc1 is IF0, then normalise the first element of the PARGS part of s1 in e1. Then:

—If it returns $TRUE, normalise the second element of the PARGS part of s1 in e1.

—If it returns $FALSE, then normalise the third element of the PARGS part of s1 in e1.

—If proc1 is LAMBDA0, then construct a closure comprising three things:

—a pattern (the first element of the PARGS part of s1);

—a body (the second element of the PARGS part of s1);

—a representation of e1.

—... other special cases ...

—An obvious idea:

—This looks just like a procedure to compute the Y function. And we have been defining procedures for weeks. Couldn’t we define a procedure that implements this algorithm?

—Answer is clearly yes.

—Such procedures are typically called meta-circular interpreters; since we are using the word processor instead of interpreter, we will call them meta-circular processors.

—Would expect such things as:

11> (normalize ’(+ 1 2) global)
1= ’3

1> (normalize ’[2 3 4] global)
1= ’[2 3 4]

1> (set x 3)
1= 3

1> ’[1 2 x]
1= ’[1 2 x]

1> (normalize ’[1 2 x] global)
1= ’[1 2 3]

1> (let [[x ’4]]
(normalize x global))
1= ’4

1> (let [[x ’4]]
(normalize ’[1 2 x] global))
1= ’[1 2 3]

1> (let [[x ’4]]
(normalize ’[1 2 ,x] global))
1= ’[1 2 4]

—Start out:

(define NORMALIZE
(lambda [struc env]
(cond [(normal struc) struc]
[(atom struc) (binding struc env)]
[(rail struc) (normalize-rail struc env)]
[(pair struc) (reduce (pproc struc) (pargs struc) env)])))

(define REDUCE
(lambda [proc args env]
(let [[proc-nf (normalize proc env)]]
(dispatch (closure-type proc-nf)
[simple (let [[args-nf (normalize args env)]]
(if (primitive proc-nf)
(reduce-primitive-simple proc-nf args-nf)
(expand-closure proc-nf args-nf)))]
[special (if (primitive proc-nf)
(reduce-primitive-special (extract-simple-closure proc-nf)
↑args
env)
(expand-closure proc-nf ↑args))]
[macro (normalize ↑(expand-closure (extract-simple-closure proc-nf) ↑args)
env))]))))

(define NORMALIZE-RAIL
(lambda [rail env]
(if (empty rail)
(rcons)
(cons (normalize (1st rail) env)
(normalize-rail (rest rail) env)))))

(define EXPAND-CLOSURE
(lambda [proc-nf args-nf]
(normalize (body proc-nf)
(bind (pattern proc-nf)
args-nf
(environment proc-nf)))))

(define READ-NORMALIZE-PRINT
(lambda [env stream]
(begin (prompt&reply (normalise (prompt&read stream) env)
stream)
(read-normalize-print env stream))))

A Non-Continuation-Passing Meta-circular Processor

———————————————————————————————————————————

—Requires lots of new things:

—Structures that designate environments, plus associated operations: BINDING, BIND, etc.

—Operations on closures: BODY, CLOSURE-ENVIRONMENT, PATTERN, CLOSURE-TYPE

—A closure construction procedure: CCONS

—DISPATCH

—NORMAL — a predicate true of normal-form structures.

—Will look at these next time.