[Indigo]<Cedar>IO>FileIO.mesa!4

FileIO.mesa

This interface contains procs to create file streams. It also specifies a model for the behavior of file streams.

Last edited by:

MBrown on January 6, 1983 8:21 pm

DIRECTORY

CIFS USING [OpenFile],

Environment USING [bytesPerPage],

File USING [Capability],

IO USING [STREAM],

Juniper USING [LFH, Transaction],

Rope USING [ROPE],

Transaction USING [Handle, nullHandle];

FileIO: CEDAR DEFINITIONS

IMPORTS

Transaction =

BEGIN

ROPE: TYPE = Rope.ROPE;

STREAM: TYPE = IO.STREAM;

bytesPerPage: CARDINAL = Environment.bytesPerPage;

File streams

Basic usage

Most programs that create file streams can do so with a very simple call to FileIO.Open. Suppose that r is a Rope.ROPE containing a file name, and your program needs to read characters from the file named by r (local file name or a full path name) using an IO.STREAM s. The call

s ← FileIO.Open[r];

in your program will accomplish this. If your program will completely rewrite the file named by r (ignoring the old contents and creating a file of that name if none exists), the call is

s ← FileIO.Open[r, overwrite];

If your program is simply logging output to the file named by r (it does not read the file, but simply adds new characters to the end, and creates a new file of that name if none exists), it calls

s ← FileIO.Open[r, append];

Finally, if your program will both read and write the file named by r (treats the file as an extendible, random-access sequence of bytes that it updates "in place"), it uses

s ← FileIO.Open[r, write];

(The last three forms only work for local files unless you have the Pine package loaded and started, in which case they also work for files on the Juniper server. Note that it is an unusual program that requires write mode, while overwrite mode is used frequently.) If you do not wish to land in the debugger if the file name r is misspelled or otherwise garbled, your program should catch FileIO.OpenFailed with why = fileNotFound or why = illegalFileName.

The moral of this tale: doing simple things is simple. You don't need to understand the multitude of parameters to the stream creation procs, since they default correctly for most purposes. You don't need to use any proc from this interface except Open unless you are doing something special. There is only one signal to catch.

At the same time, it is possible to do more ambitious things. If you want to know more, read on.

File stream model

A file is a sequence [0..fileLen) of mutable bytes; the length (fileLen) of a file can be changed.

The state of a file stream is a file file (perhaps opened under a transaction) with length fileLen, an index streamIndex IN [0..fileLen], plus a number of readonly flags (accessOptions, closeOptions) and a boolean variable closed which jointly determine the effect of stream procs. In a section below, "What STREAM operations do when applied to a file STREAM", we describe each stream proc defined in the IO interface (GetChar, PutChar, etc.) in terms of these quantities.

Concurrency

File streams provide no interlocks to control concurrent access to files. Instead, they rely on the underlying file system for concurrency control. Juniper uses locking to provide concurrency control. The Pilot file system used on the local disk provides no concurrency control, though if all of its clients used CIFS and played strictly by the rules there would be no problems. Until that happy day, clients of file streams for local files should take care that if a file stream is open for writing a file, no other readers or writers of that file exist. It is particularly dangerous to have two file streams open for writing the same file.

Individual file streams are independent objects, so there is never any need to synchronize calls to their procedures. If two processes are to share a single file stream, they must synchronize their accesses to that stream at a level above the file stream calls; individual file stream calls (PutChar, PutBlock, etc.) are not guaranteed to be atomic.

Pragmatics

A Pilot file is represented as a sequence of disk pages containing bytesPerPage bytes. There is a single leader page (overhead) that is used to hold file properties such as the length and create date. There is no functional relationship between the length of a file (in bytes) and the size of that file (in pages).

Allocating disk pages to a file is an expensive operation, especially when done incrementally (one page at a time). If a client is creating a file whose eventual length it can estimate, it should use the createLength parameter to Open to allocate enough pages to hold the entire length all at once; if the file already exists the client should create a stream and then extend the file using SetLength. If the file stream implementation must extend a file it will always extend it by several pages; if asked to shorten a file it will simply adjust the length field in the leader page without freeing any pages. To free extra pages when a stream is closed, use the truncatePagesOnClose CloseOption in the call that creates the stream.

Creating a file STREAM

FileSystem: TYPE = { pilot, juniper };

Trans: TYPE = RECORD [

body: SELECT type: FileSystem FROM

pilot => [trans: Transaction.Handle],

juniper => [trans: Juniper.Transaction ← NIL],

ENDCASE];

Used to unify the two types of transaction, for Open only.

OpenFailed: SIGNAL [why: OpenFailure, fileName: ROPE] RETURNS [retryFileName: ROPE];

OpenFailure: TYPE = { fileNotFound, illegalFileName, fileAlreadyExists, cantUpdateTiogaFile, wrongTransactionType, unknownFileCapability, notImplementedYet };

This signal is raised by the various file stream creation procedures.

CreateOptions: TYPE = {none, newOnly, oldOnly};

See description of Open below.

AccessOptions: TYPE = {read, append, write, overwrite};

Used at the time a stream is created to specify the set of operations allowed on a stream. Disallowed operations raise IO.Error[NotImplementedForThisStream] when called. The characteristics of each option:

read: PutChar, PutBlock, SetLength are disallowed, and the initial streamIndex is 0.

overwrite: all operations are allowed, the file is truncated to zero length at stream creation time, and the initial streamIndex is 0.

append: GetChar, GetBlock, SetLength, SetIndex are disallowed, and the initial streamIndex is fileLen.

write: all operations are allowed, and the initial streamIndex is 0.

CloseOptions: TYPE = CARDINAL;

CloseOptions determine optional processing during Flush and Close calls. If commitAndReopenTransOnFlush, then Flush "checkpoints" the transaction being used by the stream. If truncatePagesOnClose, then Close causes extra pages of the file to be freed. If finishTransOnClose, then Close causes the transaction to be committed or aborted, according to the abort flag to Close.

noCloseOptions: CloseOptions = 0;

commitAndReopenTransOnFlush: CloseOptions = 1;

truncatePagesOnClose: CloseOptions = 2;

finishTransOnClose: CloseOptions = 4;

defaultCloseOptions: CloseOptions = truncatePagesOnClose + finishTransOnClose;

RawOption: TYPE = BOOL;

This parameter determines the mode of access to Tioga format files. If raw = FALSE and file is in Tioga format, then if accessOptions = read, read only the plain text portion of the file (ignore "looks" and nodes with the comment property); if accessOptions # read or overwrite, raise OpenFailure [cantUpdateTiogaFile]. If raw = TRUE or file is not in Tioga format, then operate on the entire file.

StreamBufferParms: TYPE = RECORD [

bufferSize: INT [2 .. 127],

Specifies the number of pages of VM used by the stream for buffering. Juniper streams always use 1 page of buffering.

bufferSwapUnitSize: INT [1 .. 32]];

Specifies the number of pages in each uniform swap unit of the stream buffer.

defaultStreamBufferParms: StreamBufferParms = [bufferSize: 25, bufferSwapUnitSize: 5];

Good for most purposes.

minimumStreamBufferParms: StreamBufferParms = [bufferSize: 2, bufferSwapUnitSize: 1];

Good when opening a stream just to create a file or set its length.

Open: PROC [

fileName: ROPE,

accessOptions: AccessOptions ← read,

createOptions: CreateOptions ← none,

closeOptions: CloseOptions ← defaultCloseOptions,

transaction: Trans ← [juniper[]],

raw: RawOption ← FALSE,

createLength: INT ← 5 * bytesPerPage,

streamBufferParms: StreamBufferParms ← defaultStreamBufferParms]

RETURNS [STREAM];

! OpenFailed with why =

fileNotFound: createOptions = oldOnly and file does not exist (including server not found when fileName is a full path name).

illegalFileName: syntax or other error caused directory lookup to fail.

fileAlreadyExists: createOptions = newOnly and file already exists.

cantUpdateTiogaFile: raw = FALSE, accessOptions # read, and file is in Tioga format.

wrongTransactionType: transaction is a non-null Pilot transaction but fileName is a Juniper file, or vice-versa.

notImplementedYet: CIFS access is implied but accessOptions # read.

! Juniper.Error with why = transactionReset, notDone: don't know how to handle these.

! CIFS.Error with code # illegalFileName, noSuchFile, noSuchHost: don't know how to handle these.

! Volume.InsufficientSpace: couldn't create file on local volume as requested.

Create a new stream on the file specified by fileName (a full path name).

If accessOptions = read then createOptions = oldOnly are assumed.

If createOptions = none or newOnly and the file specified by fileName does not exist, then create it, with initial size (in pages excluding leader page) of createLength/bytesPerPage (rounded up), and initial length (in bytes) zero.

If fileName specifies the server "Juniper" and transaction either has pilot variant and contains Transaction.nullHandle or has juniper variant but contains NIL ([juniper[]] produces this), then call Juniper.UserInit[], and create a new transaction by calling Juniper.BeginTransaction.

StreamFromOpenFile: PROC [

openFile: CIFS.OpenFile,

accessOptions: AccessOptions ← read,

closeOptions: CloseOptions ← defaultCloseOptions,

transaction: Transaction.Handle ← Transaction.nullHandle,

raw: RawOption ← FALSE,

streamBufferParms: StreamBufferParms ← defaultStreamBufferParms]

RETURNS [STREAM];

! OpenFailed with why =

notImplementedYet: accessOptions # read.

Create a new file stream on the open file.

StreamFromCapability: PROC [

capability: File.Capability,

accessOptions: AccessOptions ← read,

closeOptions: CloseOptions ← defaultCloseOptions,

fileName: ROPE ← NIL,

transaction: Transaction.Handle ← Transaction.nullHandle,

raw: RawOption ← FALSE,

streamBufferParms: StreamBufferParms ← defaultStreamBufferParms]

RETURNS [STREAM];

! OpenFailed with why =

cantUpdateTiogaFile: raw = FALSE, accessOptions # read, and file is in Tioga format.

unknownFileCapability: no file identified by capability is present on the local disk volume.

Create a new stream on the file identified by capability (note that StreamFromCapability adds permissions to capability as required to perform the accesses as specified in accessOptions). fileName is stored in the stream, for diagnostic purposes when a stream error occurs.

CapabilityFromStream: PROC [

self: STREAM]

RETURNS [File.Capability];

! IO.Error[NotImplementedForThisStream]: self is not a stream on a Pilot file

Return the file capability for the file underlying self. This capability has the permissions that are necessary to perform stream operations.

StreamFromLFH: PROC [

lfh: Juniper.LFH,

accessOptions: AccessOptions ← read,

closeOptions: CloseOptions ← defaultCloseOptions,

fileName: ROPE ← NIL,

transaction: Juniper.Transaction,

raw: RawOption ← FALSE]

RETURNS [STREAM];

! OpenFailed with why =

cantUpdateTiogaFile: raw = FALSE, accessOptions # read, and file is in Tioga format.

unknownFileCapability: no file with the given LFH is present on Juniper.

Create a new stream on the Juniper file identified by lfh. fileName is a debugging aid, for diagnostic purposes when a stream error occurs.

END.

What STREAM operations do when applied to a file STREAM

Basic STREAM procs

IO.GetChar: PROC [self: STREAM] RETURNS [CHAR]

If streamIndex = fileLen then ERROR IO.EndOfStream. Else return file[streamIndex], and set streamIndex ← streamIndex + 1.

IO.PutChar: PROC [self: STREAM, char: CHAR]

If streamIndex = fileLen then fileLen ← fileLen + 1. Then set file[streamIndex] ← char, streamIndex ← streamIndex + 1.

IO.GetBlock: PROC [self: STREAM, block: REF TEXT, startIndex: NAT ← 0, stopIndexPlusOne: NAT ← LAST[NAT]] RETURNS [nBytesRead: NAT]

Equivalent to (but faster than)

stopIndexPlusOne ← MIN [block.maxLength, stopIndexPlusOne];

nBytesRead: NAT ← MIN[fileLen-streamIndex, stopIndexPlusOne-startIndex];

FOR i: NAT IN [0..nBytesRead) DO block[startIndex+i] ← GetChar[self] ENDLOOP;

IF nBytesRead # 0 THEN block.length ← startIndex + nBytesRead;

RETURN[nBytesRead]

IO.PutBlock: PROC [self: STREAM, block: REF READONLY TEXT, startIndex: NAT ← 0, stopIndexPlusOne: NAT ← LAST[NAT]]

Equivalent to (but faster than)

IF stopIndexPlusOne > block.maxLength THEN stopIndexPlusOne ← block.length;

FOR i: NAT IN [startIndex..stopIndexPlusOne) DO PutChar[self, block[i]] ENDLOOP;

IO.UnsafeGetBlock: UNSAFE PROC [self: STREAM, block: IO.UnsafeBlock] RETURNS [nBytesRead: INT]

Equivalent to (but faster than)

IF block.startIndex < 0 OR block.stopIndexPlusOne < 0 THEN

ERROR IO.Error[BadIndex];

nBytesRead: INT ← MIN[fileLen-streamIndex, block.stopIndexPlusOne-block.startIndex];

FOR i: INT IN [0..nBytesRead) DO

block.base^[block.startIndex+i] ← GetChar[self] ENDLOOP;

RETURN[nBytesRead]

IO.UnsafePutBlock: PROC [self: STREAM, block: IO.UnsafeBlock]

Equivalent to (but faster than)

IF block.startIndex < 0 OR block.stopIndexPlusOne < 0 THEN

ERROR IO.Error[BadIndex];

FOR i: INT IN [block.startIndex..block.stopIndexPlusOne) DO

PutChar[self, block.base^[i]] ENDLOOP;

IO.CharsAvail: PROC [self: STREAM] RETURNS [BOOL]

Return TRUE.

IO.EndOf: PROC [self: STREAM] RETURNS [BOOL]

Return streamIndex = fileLen.

IO.Flush: PROC [self: STREAM]

Force all stream writes since the time of stream creation or the preceding Flush to be written to disk. If commitAndReopenTransOnFlush then first commit trans, then begin a new transaction as trans.

IO.Reset: PROC [self: STREAM]

If accessOptions # append then set streamIndex ← 0.

IO.Close: PROC [self: STREAM, abort: BOOL ← FALSE]

If NOT abort, then force all stream actions since stream creation or the preceding Flush to be written to disk; otherwise discard them. If truncatePagesOnClose, then discard unused pages from end of file. If finishTransOnClose, then commit or abort trans depending on the state of abort. Invalidate self (all operations on self other than Flush, Reset, and Close will raise ERROR IO.Error[StreamClosed]; these three do nothing).

Less basic STREAM procs

IO.PutBack: PROC [self: STREAM, char: CHAR]

If streamIndex = 0 then ERROR IO.Error[IllegalPutBack]. Otherwise, set streamIndex ← streamIndex - 1, and if file[streamIndex] # char then ERROR IO.Error[IllegalPutBack].

IO.PeekChar: PROC [self: STREAM] RETURNS [char: CHAR];

Equivalent to:

c: CHAR ← self.GetChar[]; self.PutBack[c]; RETURN [c];

File-specific STREAM procs

IO.GetIndex: PROC [self: STREAM] RETURNS [INT]

Return streamIndex.

IO.SetIndex: PROC [self: STREAM, index: INT]

ERROR IO.EndOfStream if index > fileLen. Set streamIndex ← index.

IO.GetLength: PROC [self: STREAM] RETURNS [INT]

Return fileLen.

IO.SetLength: PROC [self: STREAM, length: INT]

Set fileLen ← l, then set streamIndex ← MIN[streamIndex, fileLen]. The contents of file[oldFileLen .. fileLen) are undefined.

Change Log

Created by MBrown on 7-Dec-81 10:33:20

By editing FileByteStream.

Changed by MBrown on March 26, 1982 4:41 pm

Added "raw" parm to stream create operations.

Changed by MBrown on August 23, 1982 10:25 pm

Handle -> STREAM (-> Stream in proc names), introduce FileIO.OpenFailed, CIFS access in readonly mode, initial file size in Open, buffer pages and swap units. Format this file using Tioga nodes.

Changed by MBrown on October 23, 1982 9:59 pm

Default transaction to Open is now [juniper[]], since this does not require the average user to import Transaction (to use nullHandle). Format to use current Cedar style.

Changed by MBrown on January 6, 1983 8:15 pm

Attempted to reduce the confusion over what is comment and what is not comment in the interface, and to improve the explanation of AccessOptions.