MesaScanner.mesa
Russ Atkinson, August 27, 1982 12:50 pm
Last Edited by: Teitelman, December 20, 1982 4:21 pm
DIRECTORY
Rope USING [ROPE];
CedarScanner: CEDAR DEFINITIONS
TokenKind:
TYPE = {
tokenERROR, -- token.msg describes the scanning error
tokenID, -- an identifier or reserved word
tokenINT, -- an INT literal
tokenREAL, -- a REAL literal
tokenROPE, -- a ROPE literal
tokenCHAR, -- a CHAR literal
tokenATOM, -- an ATOM literal
tokenSINGLE, -- a single-character token
tokenDOUBLE, -- a double-character token
tokenCOMMENT, -- a comment
tokenEOF -- the end-of-file marker
Token:
TYPE =
RECORD [start:
INT, next:
INT, kind: TokenKind, msg:
ROPE];
start gives the 1st char pos of the token
next gives the char pos immediately after the token
(length = next - size)
msg # NIL only for kind = tokenERROR
GetProc:
TYPE =
PROC [data:
REF, index:
INT]
RETURNS [
CHAR];
type of user proc used to get characters
NUL should be returned to indicate fetch beyond end
ability to randomly access characters is assumed
GetClosure: TYPE = RECORD[proc: GetProc, data: REF ← NIL];
GetToken:
PROC [get: GetClosure, index:
INT]
RETURNS [token: Token];
gets a token given a character source and a starting position
the current errors (in token.msg) are:
"invalid escape code"
"invalid octal constant"
"invalid character code"
"end-of-file in string literal"
"invalid atom"
"invalid character"
RealFromToken:
PROC [get: GetClosure, token: Token]
RETURNS [
REAL];
takes a Token into a REAL (parses the literal)
signals WrongKind if token.kind # tokenREAL
other errors may be propagated from Real.ReadReal
IntFromToken:
PROC [get: GetClosure, token: Token]
RETURNS [
INT];
takes a Token into a INT (parses the literal)
error WrongKind if token.kind # tokenINT
error IntegerOverflow if decimal and number > LAST[INT]
error IntegerOverflow if octal and number > LAST[LONG CARDINAL]
CharFromToken:
PROC [get: GetClosure, token: Token]
RETURNS [
CHAR];
takes a Token into a CHAR (parses the literal)
signals WrongKind if token.kind # tokenCHAR
SingleFromToken:
PROC [get: GetClosure, token: Token]
RETURNS [
CHAR];
takes a Token into a CHAR
signals WrongKind if token.kind # tokenSINGLE
RopeFromToken:
PROC [get: GetClosure, token: Token]
RETURNS [
ROPE];
takes a Token into a ROPE (parses the literal)
signals WrongKind if token.kind # tokenROPE
AtomFromToken:
PROC [get: GetClosure, token: Token]
RETURNS [
ATOM];
takes a Token into a ATOM (parses the literal)
signals WrongKind if token.kind # tokenATOM
ContentsFromToken:
PROC [get: GetClosure, token: Token]
RETURNS [
ROPE];
gets the contents of the token as a ROPE
can be used on any token
WrongKind:
ERROR;
raised when attempting to parse the wrong kind of literal
IntegerOverflow:
ERROR;
raised when attempting to parse an excessive integer