Most clients of FS simply want to create a file stream with which to read or write a file. If that's all you want to do then this section contains all you need to know about FS. Suppose that r is a ROPE containing a file name. If your program needs to read characters from the file named by r (local or global file name, probably with no version part) from a STREAM "s", it should call

If your program will provide all new contents for the file named r (it ignores any existing versions of the file and creates a new version), it should call

If your program is logging output to the file named by r (it updates the file only by adding new characters to the end, and creates a new file of that name if none exists), it should call

Finally, if your program will both read and write the file named by r (it treats the file as an extendible, random-access sequence of bytes that it updates "in place"), it should call

It is an unusual program that requires $write mode, while $create mode is used frequently. The last three forms only work for local files.

If you do not wish to land in the debugger when the file name r is misspelled or otherwise garbled, your calls on FS.StreamOpen should be protected with a catch phrase something like

The moral of this tale: doing simple things is simple. You don't need to understand the multitude of parameters to StreamOpen and StreamFromOpenFile, since they default correctly for most purposes. You don't need to use any procedure from this interface except FS.StreamOpen unless you are doing something special. There is only one signal to catch.

Having created a file stream, you may have questions about the semantics of the generic IO operations when applied to this stream: will SetIndex extend the file, and what does Reset do? To answer these questions, read the "File Streams" section below.

It is also possible that you aren't interested in streams at all, but want to do something else like enumerate a directory. Read on, the answer is here somewhere.

FS is a file system for use on a Cedar workstation. It provides access both to remote file servers and to the local disk. It includes procedures for creating IO.STREAM's on files. In addition to the normal file system facilities for manipulating named files, FS also contains facilities for cacheing remote files, for binding local names to remote files, for finding the version of a file that was created at a particular time, for clients to provide the implementation of open files, and for limiting the number of extant versions of local files.

FS is a successor to the Cedar Interim File System (CIFS) that was developed by Dave Gifford with help from Larry Stewart. The design of FS benefited from advice offered by Andrew Birrell, Mark Brown, Butler Lampson, Roy Levin, Roger Needham, Eric Schmidt, Larry Stewart, Paul Rovner, and Ed Taft.

FSBackdoor.mesa is an additional interface to the Cedar File System that contains facilities intended for use by experts.

FS provides access to files on any remote file server that supports the FTP protocol, in particular IFS's and Alpine server's. These file servers may be acccessed from many Cedar instances on different workstations at the same time.

The local disk is accessed through an abstraction called the local server. A Cedar instance is associated with a set of logical volumes on the local disk, one of which may be designated the system volume. FS provides a directory for each volume. It also provides on the system volume a cache for remote files.

FS allows clients to provide the implementation of open files (see FSBackdoor). Open files obtained from client packages may be presented to FS procedures that operate on open files. Client-provided open files may be local or remote, or many not be disk files at all.

Clients of FS should only have to catch the single ERROR FS.Error. In particular, errors from the File package and the STP package are mapped into FS.Error's. Other packages that hand out FS.OpenFile's (see FSBackdoor.CreateProcsOpenFile) are expected to raise FS.Error.

ErrorGroup: TYPE = {
ok, -- initial group for a new FS.ErrorDesc
bug, -- caused by an internal bug
environment, -- something's wrong in the environment; human intervention required
lock, -- conflict over locks
client, -- illegal operation, probably due to bug in client program
user -- illegal operation, probably due to user action
};

ErrorDesc: TYPE = RECORD [group: ErrorGroup, code: ATOM, explanation: ROPE]
← [ok, NIL, NIL];

Error: ERROR [error: ErrorDesc];

Names for FS files are called "FNames". An FName consists of (in order) a server, a root directory, zero or more subdirectories, a simple name, and a version. An FName may not exceed 120 characters excluding the version part. The version part will always fit in 6 characters, so the maximum space an FName can occupy is:

maxFNameLength: CARDINAL = 126;

An FName that does not start with a server part is called a partial name and will be interpreted relative to a working directory. The working directory may be specified as the argument named "wDir" to FS procedures, or may be defaulted.

To expand the arguments "name" and "wDir" of an FS procedure into a full FName, FS checks the first character of "name". If it is "[" ("/") then "name" is the full FName. If not then FS looks for a working directory name in three places in order:

The first ROPE found whose length is not 0 is preprended to "name" to form the full FName. A working directory obtained from 1 or 2 can be local or global. If it does not end with "]" or ">" ("/") then a ">" ("/") is appended. The defualt working directory must be local.

SetDefaultWDir: PROC [dir: ROPE ← NIL];

GetDefaultWDir: PROC RETURNS [ROPE];

Position: TYPE = RECORD [start, length: CARDINAL];

ComponentPositions: TYPE = RECORD [server, dir, subDirs, base, ext, ver: Position];

ExpandName: PROC[name: ROPE, wDir: ROPE ← NIL]
RETURNS [fullFName: ROPE, cp: ComponentPositions, dirOmitted: BOOL];

ComponentRopes: TYPE = RECORD [server, dir, subDirs, base, ext, ver: ROPE ← NIL];

ConstructFName: PROC [cr: ComponentRopes, omitDir: BOOL ← FALSE] RETURNS [ROPE];

File properties provided by FS are page count, byte count, created-time, and keep. The page count is the number of pages allocated to the file. The byte count is the number of meaningful bytes in a file; the byte count is kept up-to-date by explicit client calls on FS procedures. The created-time is the date and time when the contents of the file were created (or changed). The keep specifies how many versions of a file to keep around (see the next section).

WordsForPages: PROC[pages: INT] RETURNS [words: INT];

BytesForPages: PROC[pages: INT] RETURNS [bytes: INT];

PagesForWords: PROC[words: INT] RETURNS [pages: INT];

PagesForBytes: PROC[bytes: INT] RETURNS [pages: INT];

Most servers include version numbers in file names. Version numbers allow for multiple instances of a file to exist at one time. FS enforces the policy that the version on any newly created file is one larger than the largest existing version, the so-called "next" or !N version. It is not possible to create any other version with FS. When accessing an existing file the version part of an FName may be a number, may be a variable, or may be omitted. The variables allowed are !L, meaning the lowest existing version number, and !H, meaning the highest existing version number. When omitted, the version part defaults to !L or !H, depending on the operation being performed.

A property for each LName is the "keep", a CARDINAL that specifies the number of versions of an LName that should be kept around. Whenever an LName is created, its "keep" is inherited from the existing !H version or set from an argument to the procedure doing the creation. The keep of the existing !H version can be changed with FS.SetKeep.

Keep processing occurs when creating a new version of an LName. In this case, FS will enumerate existing version in decreasing order from !H. After "keep - 1" versions are encountered in this enumeration, additional versions will be deleted if not open. Thus, the keep is the steady state count of versions to be retained. The local volume file of a deleted version will be reused as the disk file for the new version being created. For example, if the only existing version of a file is named Example.bcd!4, it has a keep of 1, and no client has it open, then FS.Create[Example.bcd] will cause Example.bcd!4 to be deleted and its disk file to be reused for Example.bcd!5. Calling FS.SetKeep will have the side effect of deleting files that are obsolete relative to the new keep, even when the keep is not changed.

Note that setting a keep to LAST[CARDINAL] will result in all versions being kept, since there can never be that many versions. A keep of 0 is illegal.

Most remote servers do not implement keeps.

FS manages a cache on the system volume of a Cedar instance. The cache contains complete copies of global files. When the File package underlying FS decides that the volume is getting too full it calls out to FS. FS responds by removing an unopen global file from the cache, using an approximate LRU algorithm to order removals. FSBackdoor contains procedures for manually managing the cache.

Cacheing files introduces the potential for different Cedar instances to see different name to content mappings for global files. When one Cedar instance changes a name to content mapping, the cacheing could delay or prevent propagation of the change to the server or to the caches in other Cedar instances. (Such problems do not arise for local files, of course, since they are not cached.) These problems are mitigated by restrictions on the operations that can be performed. Global files cannot be opened for writing with FS.Open. New global files cannot be created via FS.Create. FS.Copy and FS.Rename can be used only to create the !N version of a global file. Thus, cached global files never need to be written back to the server. (FS.Copy and FS.Rename can overwrite the existing file in the case of a server that does not inplement version numbers.) The next two sections outline other measures for dealing with inconsistencies between caches and servers.

For certain FS procedures the identification of which global file to operate on can include the version variables !L or !H, or can be defaulted to the variables !L or !H (depending on the operation). Binding a version variable to a particular version would be straightforward if the remote server were always accessible and if the client were always willing to incurr the cost of accessing it. In some circumstances, however, a client may wish to complete the binding based on the possibly incomplete information available from the cache. The "remoteCheck" argument to FS.FileInfo, FS.Open and FS.Copy allows the client to control the scope of the version search. (FS.Rename and FS.Delete do not include a "remoteCheck" argument because renaming and deletion occur synchronously on the remote server, so there is no option.) The "remoteCheck" argument is evaluated only when a GName is specified. When "remoteCheck" is TRUE then FS binds a version variable by searching the versions on the remote server. The operation fails if the remote server is inaccessible. When "remoteCheck" is FALSE then FS binds the version variable by searching the versions in the cache. Only if no cached version is found is the remote server interrogated.

All of the FS procedures that accept an FName argument also include a "wantedCreatedTime" argument. This time provides an alternative way to name a file. If "wantedCreatedTime" is not equal to BasicTime.nullGMT, then the file to be operated upon is determined by a search for the version with the specified created-time. Any version part included in the name is treated as a hint. The created-time of a file is treated as a unique identifier of the content. If a version of a global file with the wanted created-time is found in the cache, then FS will not bother to check with the remote server (even if "remoteCheck" is TRUE) to see if the same created-time to version binding exists there.

FS includes facilities for attaching an LName to a GName and created-time. An attachment is a name binding. Attachment is a way to give an LName to a global file, and is useful as a cheap way to copy global files into local files. "BringOver" and "SModel" are the primary clients of the attachment mechanism.

Attachments are created using certain modes of FS.Copy. For most FS procedures, an attached LName behaves as though the global file were in fact copied into the local file when FS.Copy was invoked. There are three exceptions. First, the various procedures for obtaining information about files inform the client when an LName is attached and provide the GName of the attachment. Second, opening an attached LName is like opening the global file; if the global file has been deleted and is not in the cache, then the open will fail; it is the global file that is read locked. Third, when a local file that is attached to a global file is opened for writing, the attachment is broken and the contents of the global file are copied into a local file; the local file assumes the LName and the GName is forgotten.

There are two types of locks that can be set on files: "read" and "write". These locks are local to a Cedar instance.

Lock: TYPE = {read, write};

For access to remote servers FS uses the credentials obtained from the UserCredentials interface. If these do not work then FS.Error[environment, $badCredentials] occurs.

FileInfo: PROC [
name: ROPE,
wantedCreatedTime: BasicTime.GMT ← BasicTime.nullGMT,
remoteCheck: BOOL ← TRUE,
wDir: ROPE ← NIL
] RETURNS [
fullFName, attachedTo: ROPE,
keep: CARDINAL,
bytes: INT,
created: BasicTime.GMT
];

Enumerations are specified with a "pattern" argument, which is an FName in which the character "*", meaning "match anything", may appear zero or more times. "*" may not appear in the server part, or in the root directory of an LName. The presence or absence of a server part in "pattern" controls the use of working directories as usual. The version part of a "pattern" may be !H, !L, !*, or ! followed by digits; or may be omitted (defaults to !*). It cannot contain a mixture of "*" and digits. The order of the enumeration may be different for different sorts of servers. For the local server and IFS's, the enumeration is in lexical order of LNames expressed in bracket syntax (lower cases letters are mapped to upper case) but without the version part, and then in numeric order by version number.

Monitor locks are not held during an enumeration. Thus, the client may call FS.Delete from an FS.InfoProc or an FS.NameProc. Fine point: deleting or storing (by FS.Copy) ahead of the current position in a remote enumeration can produce unexpected results; the change may not be reflected in the enumeration.

InfoProc: TYPE = PROC [
fullFName, attachedTo: ROPE,
created: BasicTime.GMT,
bytes: INT,
keep: CARDINAL
] RETURNS [continue: BOOL];

EnumerateForInfo: PROC [pattern: ROPE, proc: InfoProc, wDir: ROPE ← NIL];

NameProc: TYPE = PROC [fullFName: ROPE] RETURNS [continue: BOOL];

EnumerateForNames: PROC [pattern: ROPE, proc: NameProc, wDir: ROPE ← NIL];

An FS.OpenFile may be presented to other procedures to get access to the file's contents and properties. All global files opened using the FS procedures defined below are accessed via the FTP protocol and the cache.

Other packages can include procedures that return FS.OpenFile's. In such cases, the arguments to the opening procedures and the detailed semantics of operations on the client-provided FS.OpenFile's are defined by that package. (The operations are defined by providing procedures to FSBackdoor.OpenFileFromProcs). For example, the FS.OpenFile's obtained by calling the Alpine package directly do permit use of Alpine's special facilities for transaction, locking, and page access. Access to such FS.OpenFile's is via Alpine procedures and completely bypasses FS's locking and cache.

OpenFile: TYPE = RECORD[REF];

nullOpenFile: OpenFile = [NIL];

Open: PROC [name: ROPE,
lock: Lock ← $read,
wantedCreatedTime: BasicTime.GMT ← BasicTime.nullGMT,
remoteCheck: BOOL ← TRUE,
wDir: ROPE ← NIL
] RETURNS [OpenFile];

Create: PROC [name: ROPE,
setPages: BOOL ← TRUE, pages: INT ← 0,
setKeep: BOOL ← FALSE, keep: CARDINAL ← 1,
wDir: ROPE ← NIL
] RETURNS [OpenFile];

OpenOrCreate: PROC [name: ROPE,
keep: CARDINAL ← 1, pages: INT ← 5, wDir: ROPE ← NIL
] RETURNS [OpenFile];

GetClass: PROC [file: OpenFile] RETURNS [ATOM];

SameFile: PROC [file1, file2: OpenFile] RETURNS [BOOL];

GetName: PROC [file: OpenFile] RETURNS [fullFName, attachedTo: ROPE];

GetInfo: PROC [file: OpenFile]
RETURNS [keep: CARDINAL, pages, bytes: INT, created: BasicTime.GMT, lock: Lock];

SetPageCount: PROC [file: OpenFile, pages: INT];

SetByteCountAndCreatedTime: PROC [file: OpenFile,
bytes: INT ← -1, created: BasicTime.GMT ← BasicTime.nullGMT];

Read: UNSAFE PROC [file: OpenFile, from, nPages: INT, to: LONG POINTER];

Write: PROC [file: OpenFile, to: INT, nPages: INT, from: LONG POINTER];

Close: PROC [file: OpenFile];

File Stream Model

Concurrency

Pragmatics

StreamOptions: TYPE = PACKED ARRAY StreamOption OF FalseBool;

FalseBool: TYPE = BOOL ← FALSE;

StreamOption: TYPE = {

defaultStreamOptions: StreamOptions = ALL[TRUE];

StreamBufferParms: TYPE = RECORD [vmPagesPerBuffer: INT [1 .. 128], nBuffers: INT [1 .. 4]];

defaultStreamBufferParms: StreamBufferParms = [vmPagesPerBuffer: 8, nBuffers: 2];

minimumStreamBufferParms: StreamBufferParms = [vmPagesPerBuffer: 1, nBuffers: 1];

ByteCount: TYPE = INT;

ExtendFileProc: TYPE = PROC [--current--ByteCount] RETURNS [--new--ByteCount];

AccessOptions: TYPE = {read, create, append, write};

StreamOpen: PROC [fileName: ROPE,
accessOptions: AccessOptions ← $read,
streamOptions: StreamOptions ← defaultStreamOptions,
keep: CARDINAL ← 1,
createByteCount: ByteCount ← 2560,
streamBufferParms: StreamBufferParms ← defaultStreamBufferParms,
extendFileProc: ExtendFileProc ← NIL,
wantedCreatedTime: BasicTime.GMT ← BasicTime.nullGMT,
remoteCheck: BOOL ← TRUE,
wDir: ROPE ← NIL
] RETURNS [STREAM];

StreamFromOpenStream: PROC [self: STREAM] RETURNS [STREAM];

InitialPosition: TYPE = {start, end};

StreamFromOpenFile: PROC [openFile: OpenFile,
accessRights: Lock ← $read,
initialPosition: InitialPosition ← $start,
streamOptions: StreamOptions ← defaultStreamOptions,
streamBufferParms: StreamBufferParms ← defaultStreamBufferParms,
extendFileProc: ExtendFileProc ← NIL
] RETURNS [STREAM];

OpenFileFromStream: PROC [self: STREAM] RETURNS [OpenFile];

ErrorFromStream: PROC [self: STREAM] RETURNS [ErrorDesc];

Copy: PROC [from, to: ROPE,
setKeep: BOOL ← FALSE, keep: CARDINAL ← 1,
wantedCreatedTime: BasicTime.GMT ← BasicTime.nullGMT,
remoteCheck: BOOL ← TRUE,
attach: BOOL ← FALSE,
wDir: ROPE ← NIL
] RETURNS [toFName: ROPE];

Delete: PROC [name: ROPE,
wantedCreatedTime: BasicTime.GMT ← BasicTime.nullGMT,
wDir: ROPE ← NIL
];

Rename: PROC [from, to: ROPE,
setKeep: BOOL ← FALSE, keep: CARDINAL ← 1,
wantedCreatedTime: BasicTime.GMT ← BasicTime.nullGMT,
wDir: ROPE ← NIL
];

SetKeep: PROC [name: ROPE, keep: CARDINAL ← 1, wDir: ROPE ← NIL];

Error codes are ATOM's so that packages implementing FS.OpenFile's via FSBackdoor.CreateProcsOpenFile can extend the set of codes. Following is a complete list of the codes generated by the FS implementation. The list is partitioned by the FS.ErrorGroup associated for each code.

Bugs

Environment Errors

Lock Errors

Client Errors

User Errors