File: Token.mesa - last edited by:
Copyright Ó 1992 by Xerox Corporation. All rights reserved.
PXK :  8-Nov-82 18:00:44
Mark: 18-May-81 17:12:04
LXR : 14-Sep-82 11:02:45
BXM :  7-Oct-81 13:56:12
AXD :  1-Jul-83 16:49:33
Philip James, March 13, 1991 3:34 pm PST
Copyright (C) Xerox Corporation 1982. All rights reserved.
DIRECTORY
Ascii USING [NUL],
Rope USING [ROPE];
Format USING [NetFormat],
System USING [HostNumber, NetworkAddress, NetworkNumber, SocketNumber],
Window USING [Box];
Token: CEDAR DEFINITIONS =
BEGIN
TYPEs
Client GetCharProc can conceal instance data around h­ passed to h.getChar.
Handle: TYPE = REF Object;
Object: TYPE = RECORD [
getChar: GetCharProcType, break: CHARACTER ¬ Ascii.NUL];
GetCharProcType: TYPE = PROCEDURE [h: Handle] RETURNS [c: CHARACTER];
A filter is called only once for each input character.
FilterState­[0..1] are initialized to 0 by routines using filters
Delimited "tokens" may be parsed via state maintained in the FilterState.
FilterState: TYPE = REF StandardFilterState;
StandardFilterState: TYPE = ARRAY [0..2) OF UNSPECIFIED;
FilterProcType: TYPE = PROCEDURE [c: CHARACTER, data: FilterState]
RETURNS [inClass: BOOLEAN];
QuoteProcType: TYPE = PROCEDURE [c: CHARACTER] RETURNS [closing: CHARACTER];
SkipMode: TYPE = {none, whiteSpace, nonToken};
nonQuote: CHARACTER = Ascii.NUL;
PROCEDUREs
All Rope.ROPEs RETURNed are from the system heap.
All routines update h.break before RETURNing.
Skips leading white space!
Boolean: PROCEDURE [h: Handle, signalOnError: BOOLEAN ¬ TRUE]
RETURNS [true: BOOLEAN];
Number: PROCEDURE [h: Handle, radix: CARDINAL, signalOnError: BOOLEAN ¬ TRUE]
RETURNS [u: UNSPECIFIED];
Decimal: PROCEDURE [h: Handle, signalOnError: BOOLEAN ¬ TRUE]
RETURNS [i: INTEGER];
Octal: PROCEDURE [h: Handle, signalOnError: BOOLEAN ¬ TRUE]
RETURNS [c: CARDINAL];
LongNumber: PROCEDURE [
h: Handle, radix: CARDINAL, signalOnError: BOOLEAN ¬ TRUE]
RETURNS [u: LONG UNSPECIFIED];
LongDecimal: PROCEDURE [h: Handle, signalOnError: BOOLEAN ¬ TRUE]
RETURNS [i: LONG INTEGER];
LongOctal: PROCEDURE [h: Handle, signalOnError: BOOLEAN ¬ TRUE]
RETURNS [c: LONG CARDINAL];
WindowBox: PROCEDURE [h: Handle] RETURNS [Window.Box];
NetFormat: TYPE = Format.NetFormat;
HostNumber: PROCEDURE [
h: Handle, format: NetFormat ← octal, signalOnError: BOOLEAN ← TRUE]
RETURNS [host: System.HostNumber];
NetworkAddress: PROCEDURE [
h: Handle, format: NetFormat ← octal, signalOnError: BOOLEAN ← TRUE]
RETURNS [address: System.NetworkAddress];
NetworkNumber: PROCEDURE [
h: Handle, format: NetFormat ← octal, signalOnError: BOOLEAN ← TRUE]
RETURNS [networkNumber: System.NetworkNumber];
SocketNumber: PROCEDURE [
h: Handle, format: NetFormat ← octal, signalOnError: BOOLEAN ← TRUE]
RETURNS [socketNumber: System.SocketNumber];
Item: PROCEDURE [
h: Handle, temporary: BOOLEAN ¬ TRUE] RETURNS [value: Rope.ROPE];
No default skip!
Skip: PROCEDURE [
h: Handle, data: FilterState, filter: FilterProcType,
skipInClass: BOOLEAN ¬ TRUE];
Filtered: PROCEDURE [
h: Handle, data: FilterState, filter: FilterProcType,
skip: SkipMode ¬ whiteSpace, temporary: BOOLEAN ¬ TRUE]
RETURNS [value: Rope.ROPE];
MaybeQuoted: PROCEDURE [
h: Handle, data: FilterState, filter: FilterProcType ¬ NonWhiteSpace,
isQuote: QuoteProcType ¬ Quote,
skip: SkipMode ¬ whiteSpace, temporary: BOOLEAN ¬ TRUE]
RETURNS [value: Rope.ROPE];
Standard filters
Alphabetic: FilterProcType;
AlphaNumeric: FilterProcType;
Delimited: FilterProcType;
skips leading white space. first char IS the delimiter.
must be called with Skip = nonToken
FileName: FilterProcType; -- '[, '], '<, '>, '*, '!, ';, '#, '-, '., '$, '+, AlphaNumeric
Line: FilterProcType;
NetFilter: FilterProcType;
NonWhiteSpace: FilterProcType;
Numeric: FilterProcType;
Switches: FilterProcType; -- '~, '-, AlphaNumeric
WhiteSpace: FilterProcType; -- SP, TAB, CR
Brackets: QuoteProcType; -- () [] {} <>
Quote: QuoteProcType; -- '"
Misc. utility
FreeStringHandle: PROCEDURE [h: Token.Handle] RETURNS [nil: Token.Handle];
StringToHandle: PROCEDURE [s: Rope.ROPE, offset: CARDINAL ¬ 0]
RETURNS [h: Token.Handle];
FreeTokenString: PROCEDURE [s: Rope.ROPE] RETURNS [nil: Rope.ROPE ¬ NIL];
SIGNALs
NilData: SIGNAL;
SyntaxError: SIGNAL [s: Rope.ROPE];
UnterminatedQuote: SIGNAL;
END...