1. Introduction
This document is an auxiliary to the documentation of IO (IODoc.tioga) and FS (FSDoc.tioga). Please read the relevant portions of these documents before reading this one.
This implementation of FileStream is intended to be "high performance" in that it will do pre-read when it detects sequentiality (three buffer-fulls in a row in ascending order), and write behind (dirty buffers will be written to disk in parallel, with the client usually not waiting for the write to complete). This package maintains a set of processes that endure throughout the lifetime of the stream. These processes are used to do the parallel operations (pre-read and write behind).
3. Multiple Streams on one File
Multiple streams on one file is supported in a restricted manner. The "write" stream must be opened first. This is a stream where writing is possible (i.e., opened with "write", "append" or "create"). Using this stream (while it is open), it is then possible to open a second "read" stream on the same file. This stream can only read the file. There can be at most one "read" stream open on a file.
A looser synchronization is possible between streams on the same file than is required for multiple processes that use a single stream. All state changing operations on the file by the write stream (e.g., PutChar, SetLength, PutBlock, and UnsafePutBlock) are visible to the read stream once the operation completes. Operations are not atomic, so intermediate states may be visible to the read stream. An example of this is that all the data written from a write stream is available to the read stream after the write request returns. No flush or other call is needed, and clients do not need to concern themselves with the details of the buffer management in the file stream package. During the time between the call is made and the call returns, none, some or all of the data can be visible to the read stream. That is, while a PutBlock call is active, the read stream may be able to read none, some or all the data; exactly how much of the data is available during the time of the PutBlock call is undefined. After the call completes, all of the data is available to the read stream. The read stream cannot damage the write stream by a concurrent call, but unreproducible results, such as end-of-file in the read stream, can occur. If the client "knows" that the data to be read is already in the stream, and is not being modified by a concurrent write request, then no synchronization is needed. This could occur when one process is reading inside a log file, while another process is appending to the end of the log.
4. Guidelines for setting streamBufferParms during calls to FS.StreamOpen
These guidelines were established using FS on the local disk on a Dorado. The use of other processors, remote file systems, or different local disks will change the effectiveness of these settings. It is almost impossible to cover all the possibilities of characteristics of programs, but these hints should help you to get the idea of what's important.
There are two parameters in streamBufferParms: vmPagesPerBuffer and nBuffers. vmPagesPerBuffer is the number of VM pages that will be in each buffer inside the file stream package. These buffers are read and written in their entirety to disk except when partial reads and writes are done when the streams are near end-of-file. The default setting is for 16 pages, or 8192 byte buffers. This is about half a track on the Dorado's local disk. The minimum is one page, and the maximum buffer size is 128 pages (65536 bytes). Making this maximum larger would require a major change to the implementation.
nBuffers is the number of buffers to allocate per stream (1-4). The buffers are managed in a (nearly) LRU manner, and are pooled for all streams on a file.
For most sequential readers (i.e., those that can't "keep up with the disk"), this buffer should be large enough to keep the client busy for about 2 or 3 disk revolutions (33.3-50 microseconds). A very fast reader may use a buffer size as small as 3 or 4 pages and still keep up with the disk using two buffers (modulo file segmentation and interference).
For most sequential writers, use two buffers where each is large enough to hold the output data produced in a disk revolution (16.7 microseconds). However, do not use total buffer space that is larger than a track unless the output is expected to be bursty.
For readers expecting small files, a nBuffers of 1 can be sufficient. This, however, disables all pre-read and write behind logic. Normally, nBuffers of 2 (the default) is the proper choice. For clients with special knowledge of their locality, use of 3 or 4 buffers can gain some performance. Do not use more than 2 buffers for a read stream that is to be read sequentially; the extra buffers will be needlessly dirtied, consume real memory, and cause extra disk activity by the VM Laundry processing. Write streams that have a high volume of (bursty) sequential output, or that update in multiple places (buffers) in the stream can benefit from nBuffers greater than 2. Buffers are written to disk, in parallel if possible, when the file pointer for the stream moves out of the buffer and the buffer was dirtied by a PutChar, PutBlock or UnsafePutBlock.
Things to consider when setting the parameters:
Total real memory consumed by buffers (can cause thrashing)
Total memory dirtied (must be cleaned by VM Laundry)
Rate of production or consumption of data
Size of a disk track (28 sectors on the Dorados)
Is the process "fast", "moderate" or "slow"?
Reference pattern (sequential?)
Latency (large buffers take longer to fill)
Disk file segmentation (excessive number of segments in a file can be caused by extending the file in small pieces, or by disk fragmentation)