DIRECTORY FileStream USING [FinalizationProc], FS USING [ErrorDesc, ExtendFileProc, Lock, OpenFile, StreamBufferParms, StreamOptions], IO USING [StreamProcs], Rope USING [ROPE], VM USING [Interval]; FileStreamPrivate: CEDAR DEFINITIONS = BEGIN DoFinalization: BOOL = TRUE; ProcHandle: TYPE = REF IO.StreamProcs; ProcHandleFromAccessRights: PROC [accessRights: FS.Lock] RETURNS [ProcHandle]; SetupBuffer: PROC [fileData: FileDataHandle, fsData: FSDataHandle, fileByte: INT] RETURNS [currentNode: BufferNodeHandle] ; ProcessNode: PROC [ fileData: FileDataHandle, node: BufferNodeHandle ] ; StartRequest: PROC [ fileData: FileDataHandle, node: BufferNodeHandle ] ; FSDataHandle: TYPE = REF Data; BufferNodeHandle: TYPE = REF BufferNode ; FileDataHandle: TYPE = REF FileData ; Data: TYPE = RECORD [ index: CARDINAL _ 0, currentNode: BufferNodeHandle _ NIL, readAheadNode: BufferNodeHandle _ NIL, lastFirstByteInBuffer: INT _ -1, isWriteStream: BOOL _ FALSE, streamIsClosed: BOOL _ FALSE, fileData: FileDataHandle, FSErrorDesc: FS.ErrorDesc, StreamClassData: REF ANY _ NIL, FinalizationProc: FileStream.FinalizationProc _ NIL, ConvertFStoIOErrors: BOOL _ FALSE ]; NodeStatus: TYPE = {valid, invalid, needsParallelRead, parallelReadActive, needsSequentialRead, sequentialReadActive, needsParallelWrite, parallelWriteActive, needsSequentialWrite, sequentialWriteActive}; BufferNode: TYPE = RECORD [ buffer: LONG POINTER TO PACKED ARRAY [0..0) OF CHAR _ NIL, bufferInterval: VM.Interval _ TRASH, bufferBytes: CARDINAL _ 0, dataBytesInBuffer: CARDINAL _ 0, firstFileByteInBuffer: INT _ -1, didPut: BOOL _ FALSE, bufferDirty: BOOL _ FALSE, useCount: INT _ 0, LRUCount: INT _ 0, status: NodeStatus _ invalid, nextBufferNode: BufferNodeHandle _ NIL ]; FileData: TYPE = RECORD [ lockRecord: MONITORED RECORD [], fileName: Rope.ROPE, firstBufferNode: BufferNodeHandle, firstReadStream: FSDataHandle _ NIL, accessRights: FS.Lock, tiogaReader: BOOL _ FALSE, fileLength: INT _ 0, writeCount: INT _ 0, somethingHappened: CONDITION, fileHandle: FS.OpenFile, streamBufferParms: FS.StreamBufferParms, extendFileProc: FS.ExtendFileProc, streamOptions: FS.StreamOptions, numberOfStreams: INT _ 1, byteLength: INT, --current value of fileHandle.OpenInfo[].bytes byteSize: INT, --current value of fileHandle.OpenInfo[].pages*bytesPerPage validBytesOnDisk: INT, writeStreamData: FSDataHandle _ NIL ]; END. CHANGE LOG Created by MBrown on December 12, 1980 12:04 PM Changed by MBrown on January 6, 1981 8:53 PM Changed by MBrown on January 16, 1981 10:00 AM Changed by MBrown on January 18, 1981 1:00 AM Changed by MBrown on 18-Jan-81 16:49:31 Changed by MBrown on January 21, 1981 3:45 PM Changed by MBrown on January 22, 1981 3:12 PM Changed by MBrown on January 25, 1981 8:19 PM Changed by MBrown on 27-Jan-81 15:10:07 Changed by MBrown on 29-Jan-81 20:43:05 Changed by Russ Atkinson on 26-May-81 14:07:10 Changed by MBrown on 7-Dec-81 10:30:56 Changed by MBrown on March 26, 1982 4:40 pm Changed by Teitelman on May 26, 1982 4:52 pm Changed by MBrown on August 23, 1982 2:18 pm Changed by MBrown on May 16, 1983 2:03 pm Changed by MBrown on June 22, 1983 11:29 am Changed by MBrown on September 17, 1983 8:42 pm Changed by Hagmann on November 22, 1983 4:28 pm Changed by Hagmann on November 28, 1983 11:57 am Changed by Hagmann on December 5, 1983 Changed by Hagmann on January 3, 1984 11:14 am Changed by Hagmann on December 19, 1984 9:04:53 am PST !âFileStreamPrivate.mesa Copyright c 1985 by Xerox Corporation. All rights reserved. MBrown on September 17, 1983 8:42 pm Teitelman on May 26, 1982 4:52 pm Levin, September 22, 1983 2:48 pm Hagmann January 3, 1984 11:14 am Russ Atkinson (RRA) February 4, 1985 3:04:15 pm PST Doug Wyatt, February 27, 1985 11:50:20 am PST DoFinalization should always be TRUE for released software. It is set to FALSE for testing. We model the behavior of a File IOStream with variables "file", "fileLen", and "streamIndex", where file is an ARRAY [0..fileLen) OF CHAR streamIndex is an INT [0..fileLen] Key in this design is the assumption that a file stream will only be used by one process at a time. Processes are free to pass the stream around, but they must synchronize their access to the stream at a higher level than the stream. Two streams on the same file, the second one created by StreamFromOpenStream, do not need synchronization above the stream level unless the application requires it. The real stream contains a lot of redundant state to make the frequently-used operations GetChar and PutChar fast. We shall list some invariant properties of this state below. (Redundant properties, included for their intuitive value, are given in parentheses.) We also define a piece of fictional state: trueDataBytesInBuffer. trueDataBytesInBuffer ::= MIN[bufferBytes, fileLen - firstByteInBuffer]. trueDataBytesInBuffer is too expensive for PutByte to maintain at all times, but it is useful for other operations. Exported by FileStreamImpl.mesa Exported by FileStreamProcessCacheImpl.mesa data kept on a per stream basis index is IN [0..currentNode.bufferBytes]. currentNode.didPut AND index >= dataBytesInBuffer => trueDataBytesInBuffer = index. This field has the property that is can be freely updated by the stream, and it can be read by the other streams under the monitor. It is always less than 64K, so that updates are always atomic (All D-machines write at least 16 bits at a time). All streams have a buffer of data in memory associated with the stream. This buffer may be empty This field is always non-NIL between calls the the package unless the stream has been closed. The control information for this buffer is a BufferNode. The node currently is use by this stream (and possibly other streams) is referred to via this field. Both currentNode and readAheadNode may only be changed under the monitor, and then only by this stream. If non-NIL, refers to buffer for read ahead data. This is the previous value in currentNode.firstByteInBuffer. It is used to detect sequentiality in reads in the preread logic. If this stream accesses buffer n, then buffer n+1, then if accesses buffer n+2 then a pre-read will be attempted given that there is more data in the file, there is a free buffer to use, and no other pre-read is in progress. If TRUE, then this record is referred to in fileData.writeStreamData and this stream was opened by StreamFromOpenFile with (logical) write access. If FALSE, this stream is referred to in fileData.firstReadStream, and was opened by StreamFromOpenFile with (logical) read access, or it was opened by StreamFromOpenStream. stream.Close[] was called. (all operations should now raise StreamError[streamClosed]). Refers to the data that is related to the file and not to the stream (there can be multiple streams refering to the same file) last FS error on this stream. Initially set to [ok,NIL,NIL]. data for use by FSStreamImpl or AlpineStreamImpl or other file system specific code. Client supplied finalization procedure, if any. When a FS error occurs, convert it to an IO.Error if this is true. This is set to TRUE once the stream is opened. valid - data in buffer is valid invalid - buffer not active, and data is not valid otherwise, need => this node has to have this action performed. Some process finds this, and does the action setting the node active. control information for a buffer buffer is really PACKED ARRAY [0..bufferBytes) OF CHAR. A space of pages for the stream buffer above. Although all buffers for a file stream are of the same length, this is kept on a per node basis so that a finalization routine can release the buffer space. Size of buffer in bytes dataBytesInBuffer is IN [0..trueDataBytesInBuffer] => IN [0..bufferBytes]. NOT didPut OR (didPut AND index <= dataBytesInBuffer) => trueDataBytesInBuffer = dataBytesInBuffer. The update of this field is protected by the monitor, but either the reader or the writer may do the update. streamIndex = firstFileByteInBuffer + index. firstFileByteInBuffer < fileLen OR firstFileByteInBuffer = fileLen = 0. normally, firstFileByteInBuffer MOD bufferBytes = 0, but it is set to -1 initally to indicate that this buffer does not match any file page. (buffer[0] is indentified with file[firstFileByteInBuffer]. There is always one file byte in the buffer unless file is empty.) (somebody has done a Put and not updated dataBytesInBuffer, FileLength, and bufferDirty). didPut OR bufferDirty => there exists some i IN [0..trueDataBytesInBuffer) such that buffer[i] # file[i] as last read from file system. You must hold the monitor to update this entry. Number of streams using this node. This reflects the fact that some Data record has currentNode or readAheadNode referring to this node. This can only be changed under the monitor, and outside the monitor is always the count of REFs pointing at this buffer. This counter is used to try to do a LRU use of the buffers. To update, you need the monitor. Changing the status requires owning the monitor The nodes are kept as a linked list, starting with FileData.firstBufferNode. Modification requires the monitor. data kept on a per file basis Although the monitor is here, it logically covers many fields in BufferNodeand Data records associated with this file. first of a list of nodes for buffers. Update requires the monitor. First read stream or NIL if no readers Currently, there is only one reader. Update requires the monitor. TRUE iff file is being read as a Tioga file (reading the plain text only). This implies that the file length shown in the stream is not the true file length, but instead is the Tioga plain text length. NOT didPut => fileLen = fileLength. didPut AND index > dataBytesInBuffer => fileLen = firstFileByteInBuffer + index (index is from the writeStreamData). Count of writes pending. Changed only under the monitor. Currently this is not allowed to exceed 1. General wait condition. It is BROADCAST when various interesting events occur such as the finish of a preread, and it is WAITed for by code that needs buffers or results from parallel processes. The number of streams, read or write, that are open on this file. Do not close the file until this is 0. Except for initialization, this is modified under the monitor. The count of bytes on disk in the beginning of the file that are valid. Writing bytes past this count should not cause a read to fill the buffer. refers to the write stream data (if any) for this file. Added Invalidate, added trans field to pilot data. DataObject is now defined without using a Stream.Handle (copied structure from FileStreamImpl). LeaderPage now defined here (but it uses something called FileStream.Subtype for its "dataType" field). Added bufferDirty (for juniper stream use). and streamIsClosed (for diagnosis from the debugger). Improved the description of invariants. Complete re-think of Pilot FileStream state, to make Put faster and generally reduce the number of expensive conversions betweeen pages and bytes. Not yet implemented. Improved comments on invariants. Added Juniper variant of Data. FileByteStreamJuniper -> Juniper. Added JOpen, POpen. CedarString -> Rope, LONG CARDINAL -> LONG INTEGER Conversion to IO, INT, ROPE, etc. Added tiogaReader field (this is redundant, but may be useful for debugging), and the IsThisThingATiogaFile proc. changed to safe language, name change IOStream -> IO. Eliminated TEXTDataOffset, added bufferParms, renamed JOpen -> JuniperOpen, POpen -> ComSoftOpen, added CIFSOpen, added CIFS open file to pilot variant of Data. Eliminated Juniper stuff, added Alpine stuff. Eliminated Pilot stuff, added FS stuff. Eliminated all procs, since file stream impl is one module (FS). added modifications for multiple buffering and multiple streams on the same file added procs from split of FSFileIOImpl (since compiler blows up in pass 3) Renamed to FileStreamPrivate from FSFileIOPrivate from FILEIOPrivate added DoFinalization for help in testing added ProcessNode and StartRequest for process cache deleted process counts in FileData. Added ConvertFStoIOErrors field. Reformatted. Êî˜codešœ™Kšœ Ïmœ1™™>—Kšœ žœÏc.˜?Kšœ žœ¡;˜Jšœžœ˜KšœG™GKšœI™I—šœ ž˜#Kšœ7™7—K˜—K˜—šžœ˜K˜Kšžœž˜ K˜Kšœ.ž˜0K˜Kšœ+ž˜-Kšœ2™2K˜Kšœ-ž˜/KšœN™NKšœ™K˜Kšœ,ž˜.KšœT™TKšœ™K˜K˜'KšœR™RKšœ8™8K˜Kšœ,ž˜.KšœT™TKšœS™SK˜Kšœ,ž˜.Kšœ ™ K˜Kšœ,ž˜.Kšœ™K˜K˜'Kšœ!™!K˜K˜'Kšœ™K˜K˜.Kšœ2™2K˜K˜'Kšœ!™!K˜K˜+KšœU™UKšœ™K˜K˜,Kšœ5™5K˜K˜,KšœK™KKšœT™TK˜K˜)Kšœ-™-K˜K˜+Kšœ'™'K˜K˜/Kšœ@™@K˜K˜/KšœP™PKšœJ™JKšœD™DK˜K˜0Kšœ(™(K˜˜&Kšœ4™4Kšœ#™#K˜—˜.Kšœ ™ —K™˜6Kšœ ™ K˜K˜K˜———…—  7p