=
BEGIN
DoFinalization:
BOOL =
TRUE;
DoFinalization should always be TRUE for released software.
It is set to FALSE for testing.
We model the behavior of a File IOStream with variables "file", "fileLen", and "streamIndex", where file is an ARRAY [0..fileLen) OF CHAR streamIndex is an INT [0..fileLen]
Key in this design is the assumption that a file stream will only be used by one process at a time. Processes are free to pass the stream around, but they must synchronize their access to the stream at a higher level than the stream. Two streams on the same file, the second one created by StreamFromOpenStream, do not need synchronization above the stream level unless the application requires it.
The real stream contains a lot of redundant state to make the frequently-used operations GetChar and PutChar fast. We shall list some invariant properties of this state below. (Redundant properties, included for their intuitive value, are given in parentheses.)
We also define a piece of fictional state: trueDataBytesInBuffer. trueDataBytesInBuffer ::= MIN[bufferBytes, fileLen - firstByteInBuffer]. trueDataBytesInBuffer is too expensive for PutByte to maintain at all times, but it is useful for other operations.
ProcHandle: TYPE = REF IO.StreamProcs;
Exported by FileStreamImpl.mesa
ProcHandleFromAccessRights:
PROC [accessRights:
FS.Lock]
RETURNS [ProcHandle];
SetupBuffer:
PROC [fileData: FileDataHandle, fsData: FSDataHandle, fileByte:
INT]
RETURNS [currentNode: BufferNodeHandle] ;
ProcessNode: PROC [ fileData: FileDataHandle, node: BufferNodeHandle ] ;
Exported by FileStreamProcessCacheImpl.mesa
StartRequest: PROC [ fileData: FileDataHandle, node: BufferNodeHandle ] ;
FSDataHandle: TYPE = REF Data;
BufferNodeHandle: TYPE = REF BufferNode ;
FileDataHandle: TYPE = REF FileData ;
data kept on a per stream basis
Data:
TYPE =
RECORD [
index:
CARDINAL ← 0,
index is IN [0..currentNode.bufferBytes].
currentNode.didPut AND index >= dataBytesInBuffer => trueDataBytesInBuffer = index.
This field has the property that is can be freely updated by the stream,
and it can be read by the other streams under the monitor.
It is always less than 64K, so that updates are always atomic (All D-machines
write at least 16 bits at a time).
currentNode: BufferNodeHandle ←
NIL,
All streams have a buffer of data in memory associated with the stream.
This buffer may be empty
This field is always non-NIL between calls the the package unless the stream has been closed.
The control information for this buffer is a BufferNode.
The node currently is use by this stream (and possibly other streams)
is referred to via this field.
Both currentNode and readAheadNode may only be changed under the monitor, and then only by this stream.
readAheadNode: BufferNodeHandle ←
NIL,
If non-NIL, refers to buffer for read ahead data.
lastFirstByteInBuffer:
INT ← -1,
This is the previous value in currentNode.firstByteInBuffer.
It is used to detect sequentiality in reads in the preread logic.
If this stream accesses buffer n, then buffer n+1, then if accesses buffer n+2 then a pre-read will be attempted given that there is more data in the file, there is a free buffer to use, and no other pre-read is in progress.
isWriteStream:
BOOL ←
FALSE,
If TRUE, then this record is referred to in fileData.writeStreamData and this
stream was opened by StreamFromOpenFile with (logical) write access.
If FALSE, this stream is referred to in fileData.firstReadStream, and was opened by StreamFromOpenFile with (logical) read access, or it was opened by StreamFromOpenStream.
streamIsClosed:
BOOL ←
FALSE,
stream.Close[] was called.
(all operations should now raise StreamError[streamClosed]).
fileData: FileDataHandle,
Refers to the data that is related to the file and not to the stream (there can be multiple streams refering to the same file)
FSErrorDesc:
FS.ErrorDesc,
last FS error on this stream. Initially set to [ok,NIL,NIL].
StreamClassData:
REF
ANY ←
NIL,
data for use by FSStreamImpl or AlpineStreamImpl or other file system specific code.
FinalizationProc: FileStream.FinalizationProc ←
NIL,
Client supplied finalization procedure, if any.
ConvertFStoIOErrors:
BOOL ←
FALSE
When a FS error occurs, convert it to an IO.Error if this is true.
This is set to TRUE once the stream is opened.
];
NodeStatus:
TYPE = {valid, invalid,
needsParallelRead, parallelReadActive, needsSequentialRead, sequentialReadActive,
needsParallelWrite, parallelWriteActive, needsSequentialWrite, sequentialWriteActive};
valid - data in buffer is valid
invalid - buffer not active, and data is not valid
otherwise, need => this node has to have this action performed.
Some process finds this, and does the action setting the node active.
control information for a buffer
BufferNode:
TYPE =
RECORD [
buffer:
LONG
POINTER
TO
PACKED
ARRAY [0..0)
OF
CHAR ←
NIL,
buffer is really PACKED ARRAY [0..bufferBytes) OF CHAR.
bufferInterval:
VM.Interval ←
TRASH,
A space of pages for the stream buffer above.
Although all buffers for a file stream are of the same length, this is kept on a per node basis so that a finalization routine can release the buffer space.
bufferBytes:
CARDINAL ← 0,
Size of buffer in bytes
dataBytesInBuffer:
CARDINAL ← 0,
dataBytesInBuffer is IN [0..trueDataBytesInBuffer] => IN [0..bufferBytes].
NOT didPut OR (didPut AND index <= dataBytesInBuffer) => trueDataBytesInBuffer = dataBytesInBuffer.
The update of this field is protected by the monitor, but either the reader or the writer may do the update.
firstFileByteInBuffer:
INT ← -1,
streamIndex = firstFileByteInBuffer + index.
firstFileByteInBuffer < fileLen OR firstFileByteInBuffer = fileLen = 0.
normally, firstFileByteInBuffer MOD bufferBytes = 0, but it is set to -1 initally to indicate that this buffer does not match any file page.
(buffer[0] is indentified with file[firstFileByteInBuffer]. There is always one file byte in the buffer unless file is empty.)
didPut:
BOOL ←
FALSE,
(somebody has done a Put and not updated dataBytesInBuffer, FileLength, and bufferDirty).
bufferDirty:
BOOL ←
FALSE,
didPut OR bufferDirty => there exists some i IN [0..trueDataBytesInBuffer) such that buffer[i] # file[i] as last read from file system.
You must hold the monitor to update this entry.
useCount:
INT ← 0,
Number of streams using this node. This reflects the fact that some Data record has currentNode or readAheadNode referring to this node. This can only be changed under the monitor, and outside the monitor is always the count of REFs pointing at this buffer.
LRUCount:
INT ← 0,
This counter is used to try to do a LRU use of the buffers. To update, you need the monitor.
status: NodeStatus ← invalid,
Changing the status requires owning the monitor
nextBufferNode: BufferNodeHandle ←
NIL
The nodes are kept as a linked list, starting with FileData.firstBufferNode.
Modification requires the monitor.
];
data kept on a per file basis
FileData:
TYPE =
RECORD [
lockRecord:
MONITORED
RECORD [],
Although the monitor is here, it logically covers many fields in BufferNodeand Data records associated with this file.
fileName: Rope.ROPE,
firstBufferNode: BufferNodeHandle,
first of a list of nodes for buffers.
Update requires the monitor.
firstReadStream: FSDataHandle ←
NIL,
First read stream or NIL if no readers
Currently, there is only one reader.
Update requires the monitor.
accessRights: FS.Lock,
tiogaReader:
BOOL ←
FALSE,
TRUE iff file is being read as a Tioga file (reading the plain text only).
This implies that the file length shown in the stream is not the true file length, but instead is the Tioga plain text length.
fileLength:
INT ← 0,
NOT didPut => fileLen = fileLength.
didPut AND index > dataBytesInBuffer => fileLen = firstFileByteInBuffer + index (index is from the writeStreamData).
writeCount:
INT ← 0,
Count of writes pending. Changed only under the monitor.
Currently this is not allowed to exceed 1.
somethingHappened:
CONDITION,
General wait condition. It is BROADCAST when various interesting events occur such as the finish of a preread, and it is WAITed for by code that needs buffers or results from parallel processes.
fileHandle: FS.OpenFile,
streamBufferParms: FS.StreamBufferParms,
extendFileProc: FS.ExtendFileProc,
streamOptions: FS.StreamOptions,
numberOfStreams:
INT ← 1,
The number of streams, read or write, that are open on this file.
Do not close the file until this is 0.
Except for initialization, this is modified under the monitor.
byteLength: INT, --current value of fileHandle.OpenInfo[].bytes
byteSize: INT, --current value of fileHandle.OpenInfo[].pages*bytesPerPage
validBytesOnDisk:
INT,
The count of bytes on disk in the beginning of the file that are valid.
Writing bytes past this count should not cause a read to fill the buffer.
writeStreamData: FSDataHandle ←
NIL
refers to the write stream data (if any) for this file.
];
END.
CHANGE LOG
Created by MBrown on December 12, 1980 12:04 PM
Changed by MBrown on January 6, 1981 8:53 PM
Added Invalidate, added trans field to pilot data.
Changed by MBrown on January 16, 1981 10:00 AM
DataObject is now defined without using a Stream.Handle (copied structure from
FileStreamImpl).
Changed by MBrown on January 18, 1981 1:00 AM
LeaderPage now defined here (but it uses something called FileStream.Subtype for its
"dataType" field).
Changed by MBrown on 18-Jan-81 16:49:31
Added bufferDirty (for juniper stream use). and streamIsClosed (for diagnosis from
the debugger). Improved the description of invariants.
Changed by MBrown on January 21, 1981 3:45 PM
Complete re-think of Pilot FileStream state, to make Put faster and generally reduce
the number of expensive conversions betweeen pages and bytes. Not yet implemented.
Changed by MBrown on January 22, 1981 3:12 PM
Improved comments on invariants.
Changed by MBrown on January 25, 1981 8:19 PM
Added Juniper variant of Data.
Changed by MBrown on 27-Jan-81 15:10:07
FileByteStreamJuniper -> Juniper.
Changed by MBrown on 29-Jan-81 20:43:05
Added JOpen, POpen.
Changed by Russ Atkinson on 26-May-81 14:07:10
CedarString -> Rope, LONG CARDINAL -> LONG INTEGER
Changed by MBrown on 7-Dec-81 10:30:56
Conversion to IO, INT, ROPE, etc.
Changed by MBrown on March 26, 1982 4:40 pm
Added tiogaReader field (this is redundant, but may be useful for debugging), and the
IsThisThingATiogaFile proc.
Changed by Teitelman on May 26, 1982 4:52 pm
changed to safe language, name change IOStream -> IO.
Changed by MBrown on August 23, 1982 2:18 pm
Eliminated TEXTDataOffset, added bufferParms, renamed JOpen -> JuniperOpen,
POpen -> ComSoftOpen, added CIFSOpen, added CIFS open file to pilot variant of Data.
Changed by MBrown on May 16, 1983 2:03 pm
Eliminated Juniper stuff, added Alpine stuff.
Changed by MBrown on June 22, 1983 11:29 am
Eliminated Pilot stuff, added FS stuff.
Changed by MBrown on September 17, 1983 8:42 pm
Eliminated all procs, since file stream impl is one module (FS).
Changed by Hagmann on November 22, 1983 4:28 pm
added modifications for multiple buffering and multiple streams on the same file
added procs from split of FSFileIOImpl (since compiler blows up in pass 3)
Renamed to FileStreamPrivate from FSFileIOPrivate from FILEIOPrivate
Changed by Hagmann on November 28, 1983 11:57 am
added DoFinalization for help in testing
Changed by Hagmann on December 5, 1983
added ProcessNode and StartRequest for process cache
deleted process counts in FileData.
Changed by Hagmann on January 3, 1984 11:14 am
Added ConvertFStoIOErrors field.
Changed by Hagmann on December 19, 1984 9:04:53 am PST
Reformatted.