<<>> <> <> <> <<>> DIRECTORY IO USING [STREAM], Scheme USING [Any]; SchemeExtras: CEDAR DEFINITIONS ~ BEGIN <> TokenKind: TYPE ~ { error, -- error describes the scanning error endOfFile, -- the end-of-file marker comment, -- a comment, including the initial semicolon identifier, -- an identifier * moduleReference, -- a structured name "foo#bar" * boolean, -- a boolean * number, -- a Scheme number * character, -- a character literal * string, -- a character string literal * openParenthesis, -- a left parenthesis closeParenthesis, -- a right parenthesis openVector, -- the opening of a vector literal "#(" quote, -- a single-close-quote quasiquote, -- a single-open-quote unquote, -- a comma unquoteSplicing, -- a comma-at-sign dot, -- a period primitiveSyntax -- one of the special "#!" tokens * }; <> <<>> ScanningError: TYPE ~ { none, -- no error earlyEOF, -- an end-of-file in the middle of a token unknownCharacterName, -- "#\" followed by an illegal character name unknownPrimitiveSyntax, -- "#!" followed by an illegal primitive syntax name unknownHashDispatch, -- "#" followed by an illegal character unknownStringEscape, -- "\" followed by an illegal character in a string illegalInitialCharacter, -- the first non-whitespace character cannot begin a token badIdentifier, -- illegally-formed identifier badNumber -- illegally-formed number }; <<>> GetToken: PROC [stream: IO.STREAM, buffer: REF TEXT, flushComments: BOOL _ TRUE, buildValue: BOOL] RETURNS [tokenKind: TokenKind, token: REF TEXT, error: ScanningError, value: Scheme.Any]; <> END.