Number: 1146

Date: 18-May-84 12':44':42

Submitter: vanMelle

Source: vanMelle

Subject: VALIDATION of file inadequate for detecting changes over exit

Lisp Version: 14-May-84 20':07':29

Description: '
Situation': user does a SAVEVM while a {DSK} file is open for write, then appends more data to the file, crashes, resumes Lisp from as of the SAVEVM.  Lisp does not notice that the file has changed, and so continues after the SAVEVM thinking that its current internal state of the file (i.e. before the additional append) is correct, thus potentially losing the appended data.'
'
If you instantiate the above with {DSK}FOO.MAIL, you have a scenario where the user can lose mail.  Hence the "Serious" rating.'
'
The reason Lisp doesn''t notice the file changed is because the stream''s VALIDATION (which is what Lisp uses to detect file changes over exit) is typically a function of the file''s creation date, which is not changed as more data is added to the file.  The devices for which this is a problem are those where Lisp can incrementally change the file external to its own vmem; specifically DSK (both Dandelion and Dolphin) and Leaf.  The existing VALIDATION scheme was designed to catch file changes external to Lisp (e.g. by the Alto exec, or another user on the file server), and it is adequate for that (assuming all file devices obey the creationdate convention).  However, with SAVEVM, you have the possibility of Lisp itself changing the file without altering the creationdate.'
'
What to do?  I''ve though of various things.  At the very least, the VALIDATION should include the file''s length in its computation, or the generic file system should check length in addition to VALIDATION.  That doesn''t cover all cases however; if the change you made was to rewrite some data in the middle of the file without changing its length, this would go undetected.  This suggests the file system should also forget any buffered pages, even if it didn''t think the file changed!  Another possibility is to change the file''s creation date every time you FORCEOUTPUT on a file (since it is only after FORCEOUTPUT that a file is assured to be in a clean state)...'
'
Given what we''re trying to protect against, however, I think it might be sufficient if every device for which this is a problem did the following': in the BEFORExxx event, make a note for each file open for output on the device that the event happened.  Next time you write ANYTHING to the file, change its creationdate and clear the event note.'
'
The only problem I see with that is if you do a SAVEVM in one process while another is doing a COPYFILE, you will lose the proper creationdate on the destination that COPYFILE so carefully copied from the source.  I would hate to have to special case that.

Workaround: 

Test Case: 

Edit-By: 

Edit-Date: 18-May-84 12':44':42

Attn: vanMelle, Kaplan

Assigned To: 

In/By: 

Disposition: 

System: Operating System

Subsystem: Generic File Operations

Machine: 1132

Disk: 

Microcode Version: 5124

Memory Size: 4096

File Server: 

Server Software Version: 

Difficulty: Moderate

Frequency: 

Impact: Serious

Priority: Hopefully

Status: Open

Problem Type: Bug

Source Files: