Number: 179

Date: 18-Mar-84 17':16':05

Submitter: Sannella.PA

Source: Kaplan.pa

Subject: Performance Certification Tools

Lisp Version: 

Description: '
Date':  3 Feb 84 09':42 PST'
From': Kaplan.pa'
Subject': Performance Certification'
To': Lispcore^, Sheil, Raim.pasa'
'
We just came within a hair of releasing a brain-damaged system--and would have it hadn''t been for JonL''s benchmarking observations.'
'
This suggests that we make performance certification an official part of our release process.  We should automate the running of standard benchmarks so that the Release Master can easily compare the relative performance on the new and old systems.  In fact, we should make sure that the results of the various benchmarks are captured in machine-readable form, so that the Release Master can be notified of any changes above some threshhold (e.g. 20% faster or slower, or 20% more or less conses, etc.)'
'
Besides preventing us from making gross mistakes, this information should find its way into the release message':'
'
1.  If we find more than, say, a 20% improvement on something, we should broadcast that to our users, to maintain their rising expectations.'
'
2.  If we find a performance decrement of 20% and we can verify that it is not an error, than users can be told to expect this, and also told what clever trade-offs we made--what else got faster or better.  Even if there is some bad news, it is better that we broadcast it in a controlled fashion indicating that we are on top of the situation rather than have users angrily discover it for themselves.'
'
--Ron'
'
-----'
'
Date': 29 MAR 84 18':58 PST'
From': JONL.PA'
Subject': Re': Lisp': ar#179, Performance Certification Tools'
To':   Masinter, LispSupport'
cc':   JONL'
'
In response to the message sent  27 Mar 84 15':38 PST from Masinter.pa'
'
I remember Ron''s original message.  Yes, I should be the recipient'
of this msg, and it''s a bit low on my stack right now;  could be elevated'
when the DLion local file system is "secure", so that there is a reasonable'
way to re-boot and continue some series of computations.'
'
-----'
'
From': MASINTER.PA'
Date': 29-Mar-84 19':52':55 PST'
Subject': Re': Lisp': ar#179, Performance Certification Tools'
In-reply-to': JONL''s message of 29 MAR 84 18':58 PST'
To': JONL'
cc': Masinter, LispSupport'
'
I don''t disagree that this task could be lower priority than many of the others on your stack, but the problems with the DLion FS are irrelevant': performance certification would help us with Dolphin and Dorado, and the DLion FS problems may still permit the reading of a single INIT.LISP file.'


Workaround: 

Test Case: 

Edit-By: Sannella.PA

Edit-Date: 13-Apr-84 15':15':42

Attn: Lisp, jonl

Assigned To: 

In/By: 

Disposition: 

System: Language Support

Subsystem: 

Machine: 

Disk: 

Microcode Version: 

Memory Size: 

File Server: 

Server Software Version: 

Difficulty: 

Frequency: 

Impact: 

Priority: Hopefully

Status: Wish

Problem Type: Performance

Source Files: