<< JunoLexer.mesa (was Lexer + pieces of ParseTable)>> <<>> <> << OBSOLETE see JunoParseUnparse June 11, 1984 9:15:18 pm PDT.>> DIRECTORY Rope USING [ROPE]; JunoLexer: DEFINITIONS = BEGIN END. << - - - - - JUNK>> LexerRec: TYPE = RECORD [maxAtomLength: INTEGER, -- max length of identifiers. Default is 200 maxStringLength: INTEGER, -- max length of strings. Default is 200 chType: ARRAY CHAR OF CharType, -- described below twoCharOps: LIST OF TwoCharOp -- list of two-character lexemes stream: Stream, -- Source of characters to be broken into lexemes. lex: Lexeme, -- current lexeme, the result of the most recent call to Lex eof: BOOL, -- set by Lex when lexeme was not found. buf: ROPE, -- contains the substring that lexed to lex error: Rope.ROPE -- initially NIL, set to error message on lexical error ]; CharType: TYPE = {letter, digit, blank, quote, op, invalid, other}; << Identifiers must start with letter and continue with letters or digits; blanks are ignored; quotes delimit strings; ops form one- or two-character operators; invalids are invalid in any context; others are valid only inside strings.>> TwoCharOp: TYPE = RECORD [chars: RECORD [CHAR, CHAR], atom: ATOM -- same two characters in atom form ]; Stream: TYPE = RECORD [pos: INT, rope: ROPE, len: INT]; -- fake character stream << - - - - - LEXING>> StartLexer: PROC [lexer: Lexer, rope: ROPE]; << Attaches the specified stream to the lexer, and puts the first lexeme into lexer.lex>> << Lexer.eof and lexer.error have same meaning as for Lex[lexer]. The lexer table and aliases must have been set up before calling StartLexer. Further lexemes are obtained by calling Lex[lexer] (see below)>>