WalnutOps.mesa
Copyright © 1984 by Xerox Corporation. All rights reserved.
Willie-Sue, July 11, 1985 2:59:51 pm PDT
Top-level Walnut Operations Interface
Last edited by: Donahue on February 2, 1985 9:10:48 am PST
(changed New mail operations around)
(Removed Move operation and changed the types of several other procedures; the major change was to redefine the MsgSet type to include a version stamp -- which may be defaulted. MsgSets now have readlocks associated with them!!)
(Changed new mail operations around again)
(Added complete returned value to GetNewMail)
Last Edited by: Willie-sue, January 4, 1985 10:13:59 am PST
Last Edited by: Wert, August 31, 1984 7:24:05 pm PDT
Implemented by: WalnutOpsImpl.mesa
DIRECTORY
BasicTime USING [GMT],
GVBasics USING [RName],
IO USING [STREAM],
Rope USING [ROPE],
ViewerTools USING [TiogaContents],
WalnutDefs USING [dontCareDomainVersion, dontCareMsgSetVersion, MsgSet, ServerInfo],
WalnutParseMsg USING [ParseProc, MsgHeaders, MsgHeaderItem];
WalnutOps:
CEDAR
DEFINITIONS =
BEGIN
Types
ROPE: TYPE = Rope.ROPE;
dontCareDomainVersion: INT = WalnutDefs.dontCareDomainVersion;
dontCareMsgSetVersion: INT = WalnutDefs.dontCareMsgSetVersion;
DomainVersion: TYPE = INT ← dontCareDomainVersion;
MsgSetVersion: TYPE = INT ← WalnutDefs.dontCareMsgSetVersion;
MsgSet: TYPE = WalnutDefs.MsgSet;
MsgSet: TYPE = RECORD[name: ROPE, version: MsgSetVersion];
Message set arguments to WalnutOps procedures take an (optional) version number argument that can be considered a read lock on the message set
ActiveMsgSetName: ROPE;
DeletedMsgSetName: ROPE;
ServerInfo: TYPE = WalnutDefs.ServerInfo;
ServerInfo: TYPE = RECORD[server: ROPE, num: INT];
EnumeratorForMsgs: TYPE = REF EnumeratorForMsgsObject;
EnumeratorForMsgsObject: TYPE;
ParseProc:
TYPE = WalnutParseMsg.ParseProc;
ParseProc: TYPE = PROC[fieldName: ROPE] RETURNS[wantThisOne, continue: BOOL];
MsgHeaders:
TYPE = WalnutParseMsg.MsgHeaders;
MsgHeaderItem: TYPE = RECORD[fieldName, value: ROPE];
MsgHeaderItem:
TYPE = WalnutParseMsg.MsgHeaderItem;
MsgHeaderItem: TYPE = RECORD[fieldName, value: ROPE];
Signals and Errors
Error: SIGNAL [who, code: ATOM, explanation: ROPE ← NIL];
Startup errors
$AlreadyStarted
The WalnutOps routines are already started, and must be shutdown first.
$DatabaseIncomplete
The database is incomplete with respect to its log. Occurs only when WalnutOps is in readOnly mode.
$LogIncomplete
The log is incomplete, that is, it contains some part of a multi-stage operation (Archive reading, expunging, etc.). Occurs only when WalnutOps is in readOnly mode.
Operation errors
$IllegalOperation
The specified operation is illegal for the specified arguments; that is, if the operation was performed, it would violate the Walnut database invariants.
$InvalidOperation
The operation is invalid for the parameters supplied. That is, the operands are valid operands, but the operation cannot be applied to them.
Running errors
These errors may be reported as part of WalnutOps operation, and in most cases represent serious system failures.
$DatabaseInaccessible
The Walnut database has become inaccessible. This may be a transient problem.
$LogInaccessible
The Walnut log has become inaccessible. This may be a transient problem.
$NotStarted
A WalnutOps routine was called, but WalnutOps is shutdown; that is, not started. This is easily fixed, that is, just call StartupWalnut[].
$ReadOnly
An attempt to modify the Walnut log or database is refused, because the log or the database is read only. It may be read only because you asked that it be, or it may be because you cannot write the files.
$Bug
Informational message, mostly used during debugging
$LogIncomplete
usually means the log is shorter than the database expects it to be
$ServerDied
Procedures
These procedures are all of the primitive atomic actions out of which Walnut is built. More complex operations can be found in WalnutClientOps. WalnutOps actions include:
Starting and stopping Walnut
Startup:
PROC[rootFile:
ROPE, wantReadOnly:
BOOL ←
FALSE]
RETURNS[isReadOnly, newMailExists:
BOOL, mailFor: GVBasics.RName, key:
ROPE];
Start up Walnut with a reference to a Walnut root file. This may involve replaying any actions specified in the "tail" of the log (that portion beyond the lastCommit) so that the database and log agree (in fact, if the database does not exist, it may involve reconstruction of the database in its entirety). If wantReadOnly is TRUE, the log and the database are opened for reading only. If the log or the database cannot be written, readOnly is TRUE upon return.
The startup code may also return information about new mail if the last operation being performed prior to a crash was a GetNewMail (see below).
already started ($AlreadyStarted).
database does not match log ($DatabaseIncomplete).
database has become inaccessible ($DatabaseInaccessible)
cannot write the log or the database (return with isReadOnly TRUE).
Shutdown:
PROC;
Save the Walnut state and shutdown Walnut.
Scavenge:
PROC[rootFile:
ROPE]
RETURNS[newMailExists: BOOL, mailFor: GVBasics.RName, key: ROPE];
Erases the database referenced in rootFile and initializes the schemaVersion date; then the log is read into the database; need NOT call Shutdown before doing an Erase; generates an error if the database is ReadOnly
Information about the Walnut database
ReadOnly:
PROC
RETURNS[readonly:
BOOL];
Is the database readonly?
GetRootInfo:
PROC
RETURNS[createDate: BasicTime.
GMT, rootFile, mailFor:
ROPE];
CreateDate and rootFile for this Walnut; who can retrieve mail
FileName:
PROC
RETURNS[dbFileName:
ROPE];
file used to store the actual database
SizeOfDatabase: PROC RETURNS[messages, msgSets: INT];
LogLength: PROC RETURNS[length: INT];
MsgSetsInfo:
PROC
RETURNS[version: DomainVersion, num:
INT];
version changes whenever a msgSet is created or destroyed. Num is the total number
RegisterReporter:
PROC[reportStream:
IO.
STREAM];
used to report progress of long running operations like scavenge or expunge;
UnregisterReporter:
PROC[reportStream:
IO.
STREAM];
unregisters reportStream
Adding new messages to the database
StartNewMail:
PROC[]
RETURNS[newMailStream:
IO.
STREAM];
Get the right to read new mail from Grapevine to the New Mail log. This operation gives access to the newMailLog. If newMailStream is NIL, either it could not be opened or it was already in use.
GetNewMail:
PROC[activeVersion: MsgSetVersion, proc:
PROC[msg, TOCentry:
ROPE, startOfSubject:
INT] ←
NIL ]
RETURNS[responses:
LIST
OF ServerInfo, complete:
BOOL];
GetNewMail takes a version of the active message set and enumerates the messages that are currently "unaccepted" This is solely a database query and does not change the state of the Walnut database (subsequent GetNewMail operations will return the same result until an AcceptNewMail is done). It may fail either because the of a version stamp mismatch or because of database unavailability. If complete is true, then all of the new messages available were retrieved; if false, there is still new mail in the system.
AcceptNewMail:
PROC[activeVersion: MsgSetVersion];
Accept accepts all currently unaccepted messages and resets the server response entries in the database -- it also increments the version number of the active message set.
RecordNewMailInfo:
PROC[logLen:
INT, server:
ROPE, num:
INT];
The length given must be the length of a legal tail of a newMailLog
EndNewMail:
PROC[];
Release the new mail log (close the stream)
CreateMsg:
PROC [msgName:
ROPE, body: ViewerTools.TiogaContents];
Log a message from some agent other than Grapevine and add it to the database. Body is expected to conform to the syntax for messages. The first message on the log with a given name wins. Any subsequent messages with the same name are dropped on the floor.
Space conservation (removing unreferenced messages)
ExpungeMsgs:
PROC[deletedVersion: MsgSetVersion]
RETURNS[bytesInDestroyedMsgs:
INT];
Removes all messages from the Deleted message set.
GetExpungeInfo:
PROC
RETURNS[firstDestroyedMsgPos, bytesInDestroyedMsgs:
INT];
returns the the log position of the first destroyed msg and the number of bytes in destroyed msgs. The actual amount of space reclaimed during an expunge will probably be more (entry overhead, other entries referring to destroyed msgs)
CopyToExpungeLog:
PROC[];
The log is re-written, starting from the beginning
Primitive message set operations
CreateMsgSet:
PROC [name:
ROPE, msDomainVersion: DomainVersion];
Create a message set with name msgSet. If the named message set exists, this is a noop.
MsgSetExists: PROC [name: ROPE, msDomainVersion: DomainVersion] RETURNS [exists: BOOL, version: MsgSetVersion];
SizeOfMsgSet: PROC [name: ROPE] RETURNS [messages: INT, version: MsgSetVersion];
EmptyMsgSet:
PROC [msgSet: MsgSet]
RETURNS [someInDeleted:
BOOL];
Remove all messages from msgSet. Messages that are no longer in any message set will be added to "Deleted".
msgSet is "Deleted" (do nothing).
messages added to "Deleted" (return with someInDeleted TRUE).
DestroyMsgSet:
PROC [msgSet: MsgSet, msDomainVersion: DomainVersion]
RETURNS [someInDeleted:
BOOL];
Remove the specified message set from the database.
msgSet must exist (ignore otherwise).
msgSet is "Active" (same as EmptyMsgSet).
msgSet is "Deleted" (do nothing).
EnumerateMsgsInMsgSet:
PROC[name:
ROPE]
RETURNS[enum: EnumeratorForMsgs];
a lazy enumerator for the messages in the msgSet name, at the time of the call. The enumerator is valid until the next time the msgSet's version changes (by adding or deleting msgs); its use after that will cause a WalnutDefs.Error with code $InvalidEnumerator. See NextMsg for details.
EnumerateMsgs:
PROC
RETURNS[enum: EnumeratorForMsgs];
a lazy enumerator for the messages in the database, at the time of the call. The enumerator is valid until the next expunge; its use after that will cause a WalnutDefs.Error with code $InvalidEnumerator. See NextMsg for details.
NextMsg:
PROC[enum: EnumeratorForMsgs]
RETURNS
[msgID:
ROPE, msList:
LIST
OF
ROPE, headers:
REF
TEXT];
a lazy enumerator for the messages in the database or in a msgSet, at the time of the call. When the end of the messages is reached, msgID will be NIL. msList is the list of message sets the msg belongs to, at the time of the call. headers contains just the headers information from the message. Be warned - subsequent calls to NextMsg may reuse the REF TEXT.
MsgsEnumeration: PROC[alphaOrder: BOOL ← TRUE] RETURNS[mL: LIST OF ROPE];
MsgSetNames: PROC[alphaOrder: BOOL ← TRUE] RETURNS[mL: LIST OF ROPE, msDomainVersion: DomainVersion];
MsgsInSetEnumeration:
PROC [name:
ROPE, fromStart:
BOOL←
TRUE]
RETURNS [mL:
LIST
OF
ROPE, msVersion: MsgSetVersion];
EnumerateMsgSets:
PROC [alphaOrder:
BOOL ←
TRUE, proc:
PROC[msgSet: MsgSet] ←
NIL]
RETURNS [msDomainVersion: DomainVersion];
NOTE: proc may not do ANY database calls
EnumerateMsgsInSet:
PROC [
name:
ROPE,
fromStart:
BOOL←
TRUE,
proc:
PROC[msg, TOCentry:
ROPE, hasBeenRead:
BOOL, startOfSubject:
INT] ←
NIL]
RETURNS [msVersion: MsgSetVersion];
for each msg in msgSet, will call proc with the indicated information about the msg. NOTE: proc may not do ANY database calls;
Parsing a message
ParseHeaders:
PROC [headers:
ROPE, proc: ParseProc]
RETURNS[msgHeaders: MsgHeaders];
proc is called for each fieldName encountered in the headers; if proc is NIL then all headers are returned. It is ok to take the REF TEXT returned by NextMsg or GetMsgText and use it, wrapped in a RefText.TrustTextAsRope.
Primitive message operations
MsgExists: PROC [msg: ROPE] RETURNS[exists: BOOL];
GetCategories: PROC [msg: ROPE] RETURNS[LIST OF ROPE];
GetDisplayProps: PROC [msg: ROPE] RETURNS[hasBeenRead: BOOL, TOCentry: ROPE, startOfSubject: INT];
GetMsgDate:
PROC [msg:
ROPE]
RETURNS[date: BasicTime.
GMT];
returns the date used to index the message
GetMsg: PROC [msg: ROPE] RETURNS[contents: ViewerTools.TiogaContents, herald, shortName: ROPE];
GetMsgText:
PROC [msg:
ROPE, text:
REF
TEXT]
RETURNS[contents:
REF
TEXT];
returns just the contents part of the msg (not the formatting information); note that the contents of comment nodes will NOT be included.
contents will be a new REF TEXT if text was not big enough
GetMsgShortName: PROC[msg: ROPE] RETURNS[shortName: ROPE];
GetHasBeenRead: PROC [msg: ROPE] RETURNS[hadBeenRead: BOOL];
SetHasBeenRead: PROC [msg: ROPE];
More complex message set / message operations
AddMsg:
PROC [msg:
ROPE, from, to: MsgSet]
RETURNS [exists:
BOOL];
Add the message (msg) from the message set (from) to the message set (to).
msg must exist and be in from (ignore).
to must exist (ignore) and the versions must agree.
from must exist (ignore) and the versions agree
to must not be "Deleted" (ignore).
msg is already in to (return with exists TRUE)
MoveMsg:
PROC [msg:
ROPE, from, to: MsgSet]
RETURNS [exists:
BOOL];
Move the message (msg) from the message set (from) to the message set (to).
msg must exist and be in from (ignore).
to must exist (ignore) and the versions must agree.
from must exist (ignore) and the versions agree
to must not be "Deleted" (ignore).
msg is already in to (return with exists TRUE)
RemoveMsg:
PROC [msg:
ROPE, from: MsgSet, deletedVersion: MsgSetVersion]
RETURNS [deleted:
BOOL];
Remove the message (msg) from the message set (from). If this action results in the message not belonging to any message set, it will be placed in the Deleted message set.
msg must exist (ignore).
from must exist (ignore).
from must not be "Deleted" (ignore).
msg must be in from (ignore).
msg was added to "Deleted" (the deleted version stamp agrees with the database and nowInDeleted is TRUE).
Producing and reading archive files
Reading an archive file is a long operation; it is resilient, so that it will complete when restarted. The file is first written to a readArchiveLog (if the archive file exists), then this log is copied to the current log and then the tail of the log is "replayed", which may involve creating new messages and shuffling them around in the database.
ReadArchiveFile:
PROC [file:
ROPE, msgSet: MsgSet]
RETURNS[numNew:
INT];
Write a "readArchiveFile" log entry; if the archiveFile exists, parses it and writes a separate readArchiveLog with appropriate entries (new msgs and moves if msgSet is not "Active" (NIL defaults to categories specified in the file). The archiveLog is then copied to the tail of the currentLog and the log is replayed. IF the file couldn't be read, numNew = -1; if there were problems parsing file, messages are printed
WriteArchiveFile:
PROC [file:
ROPE, msgSetList:
LIST
OF MsgSet, append:
BOOL ←
FALSE];
Write an archive file that contains the messages from the given message sets. No log entry is written and no updates are made to the database (we just hold the monitor to guarantee that no changes to the message sets occur).
END.