The GVPatch Manual

Steve Temple, October 28, 1982.
File [Indigo]<Grapevine>Tools>GVPatch.tioga


Introduction
GVPatch is a system for remotely patching up Grapevine servers when they encounter an error in the main database file. The errors are commonly bit corruptions of this file and there are a variety of different failure modes which have been observed. GVPatch therefore attempts to provide general facilities for tracing these errors and correcting them. Typically the error will be detected by routine consistency checks made by the programs running on the server or, in the event of a hard disk error, by the disk controller. In the latter case the scavenger program may be run which will "correct" any apparent errors and provide a log of the pages in the file which were corrected.


The Grapevine database.
The database file is named heap.data (henceforth referred to as the heap) and in present servers is just under 5000 pages long. The data structure contained by it can be described as a sequence of objects whose size varies from a few bytes to a few thousand bytes, each object can therefore span several disk pages. Each object consists of an object header and a data field, the header contains a unique object number which identifies the object and a size field containing the size of the object. When an object spans a page boundary it is split into sub-objects each of which contains the size of the sub-object and the associated object number. A compacter is used on the heap to clear out dead objects and thus the file contains some pages which have current data on them and other pages which are free. The unit of storage allocation in the heap is the segment which in current implementations consists of six contiguous pages. There is an ordering of segments within the heap and another file heap.segments contains a list of the segments currently in use and their positions within the heap file.


The GVPatch Strategy.
Grapevine servers are distributed widely around the Xerox Internet and it is generally the case that at a given server there is no-one familiar with the internal workings of the Grapevine system. Thus any repairs must be effected by wizards at PARC using the facilites of the Internet. The method adopted by GVPatch is to retrieve from the server only the structural information of the heap, that is the object headers and size information, and use this to create a local copy of the heap which contains none of the data contents of the original objects. If the entire contents of a page are required then this may be retrieved explicitly. This method has the advantage that the amount of data transferred is about 12 times less than if the whole heap were transferred (to transfer the entire heap over a route contining a 9600 baud line takes around 3 hours). GVPatch also provides for retrieving and storing arbitrary files on the server and thus the debug log or scavenger log can be read at will. The GVPatch system consists of two programs, the first of which runs on the Grapevine server and provides an interface over the Internet to the second which runs in the Cedar environment on a D-Machine. The user then interacts with this second program to repair the heap.


The GVPatch User Interface
GVPatch has three distinct modes of operation, each of which has its own viewer. Each viewer contains a number of buttons and data input facilities along with a typescript on which the output for that mode appears. Only one viewer is visible at any given time and each viewer contains buttons to change to either of the other modes.

Server mode provides facilities for reading, writing and setting the length of arbitrary files on the server, for reading the database and copying it back when corrections have been made, and for restarting the server when it is thought to be in a workable state.

Browser mode allows the user to run around in the heap looking at the structural information
contained in it. There are facilities for scanning for errors and looking for objects with given object numbers or given type. This mode allows the user to locate the position of the error and then to enter editor mode to try to correct it.

Editor mode provides facilities for reading entire pages from the Grapevine server, for displaying them in various ways and for altering the words on them. Alterations are initially applied only to the local copy of the heap and only when the user is satisfied that this is correct are those pages which have been altered written back to the server.

A log is kept of many of the operations (in particular all editing operations) performed by GVPatch and this may be saved for future reference.


Using GVPatch

Program Sources
The sources for GVPatch are presently on the directory [Indigo]<Grapevine>Tools>. The part which runs on the Grapevine server is written in Alto-Mesa and is in GVPServer.bcd while the driver program which runs in the Cedar environment is in GVPatch.bcd or can be got by a public Bringover of GVPatch.df. Using Bringover will also retrieve GVPatchExternal.mesa, a sample external editing program which can easily be edited to do what the user wants, and GVPDefs.bcd which is necessary if external editing is to be performed.


The GVPatch Server
In order to run GVPatch on the Grapevine server the modules VM, GrapevineUser, TinyPup and Sponge are needed in addition to GVPServer. A command line of the form

Grapevine.image TinyPup GrapevineUser VM GVPServer Sponge

must be given to the Alto Executive and the server will then be ready to accept commands from the driving program. Mesa.image may be substituted for Grapevine.image.


The GVPatch Driver
The driving program is started by giving a Run GVPatch command to the Cedar Executive, this starts the program and also registers GVPatch as a command with the Executive. A viewer
will have now appeared on the screen, this is the server mode viewer. The typescript of this viewer also serves as the system log.


A GVPatch Session

Server Mode - File Transfer
Assuming that the server and driver parts of GVPatch have been started as described above, the first thing to do is to click the button marked server-> and type in the name of the server. If the heap and associated files are not present they are brought from the server by clicking the button marked read-heap (and waiting a while). Note that read-heap will overwrite the local files heap.data, heap.segments and heap.map if they exist. Be careful to ensure that if local heap files exist they correspond to the files on the server being patched and are not left over from some previous patching session of another server. Any error messages are flashed onto a small message window below the buttons. If all is well the typescript will print messages saying that the connection has been made, that the file heap.segments has been read from the server and files heap.data (the heap) and heap.map have been created. The purpose of heap.map will become apparent later. When the transfer is complete (the message window shows the progress of the transfer) the user can use the read-file button to fetch any other files which might be needed from the server. The button file-> is used to enter the file name. In order to leave server mode it is necessary that the three files mentioned above exist on the local machine.


Browser Mode - Page Numbering
Having now got the copy of the heap and possibly some indication of where the error is (from scavenger or debug log files), the next step is to get into browser mode with the BROWSER button. When this is clicked the server viewer is replaced by the browser viewer which contains a new set of buttons and a new typescript. Most of the browser buttons are concerned with moving around in and looking at the heap. There is the concept of a current page within the heap to which most operations are referenced. There are two ways of describing the same page within the heap, the page number is the physical page number of that page in the heap file while the page position is the logical position of that page in the logical sequence of pages within the heap. Data is stored in the heap in logical order and thus an object which spans several pages occupies consecutive page positions in the heap. When dealing with the server and reading the scavenger or debug log it is convenient to use page numbers, and when dealing with the ordering of objects to use page positions.


Browser Mode - Setting and Displaying the Current Page
The top row of browser buttons are used to set up a current page in the heap. Use number-> to enter a page or position number and then click position-# or page-# to establish the place. The button reset sets the current page to position 0 and is useful in conjunction with the scanning operations. The buttons next and prev adjust the current page appropriately and work in terms of page position rather than page number. All the previous operations are checked to see that the current page is always within the active part of the heap.
Associated with the next and prev buttons is the REPEAT button. This allows the next or prev operation to be repeated an arbitrary number of times. Succesive clicks of REPEAT arm and disarm the repeat facility, it is armed when the button background is black. The repeat count is entered using the repeat-> button to input an integer. Clicking next or prev now will cause the operation to be repeated the required number of times. The button ! STOP ! may be used to stop a repeated operation. The button marked show is used to display the structural information of objects on the current page. The amount of information presented is controlled by the button FULL. When this is clicked it turns into BRIEF (and vice-versa). The FULL mode of display shows the position of each sub-object on the page, its type and object number and its size. The BRIEF mode shows only type information.
The AUTO button is a switch (like REPEAT) which when armed causes an automatic show to take place whenever the current place changes (ie. by next, prev, reset, position-# or page-#). A useful summary of the heap can be obtained by setting a large repeat count of say 10000 and then setting BRIEF, AUTO on and REPEAT on and then clicking reset and next. The output from this may then be made the current selection and printed.


Browser Mode - Heap Scanning
The other facility provided by browser mode is the ability to scan the heap looking for inconsistencies in the data structure, or for objects with a given type or object number. The buttons scan-error and scan-match initiate searches in the heap, the former will print only error information while the latter also prints any pattern matches which have occurred. The scan begins at the current page and continues to the end of the heap unless the ! STOP ! button is clicked. The current page is then the page at which the scan stopped so that the scan may be stopped temporarily and then restarted.
The button pattern-> is used to enter a search pattern for an object number. There are three search parameters and each may be made a "don't care" by entering the character "/" instead of a data item. The items are entered in order and are page, index number and type of the object number for which it is desired to search. The first two are entered as octal numbers and the third as a string. Thus to search for all objects of type Gap the pattern is "//gap", while to search for all objects with object number (page: 24, index: 16) the pattern is "24 16 /". If all three search parameters are "/" then no matching will take place. Note that the object type string must contain at least three characters unless it is "/".
There are three types of message from the scan routines, ERRORs indicate a fault in the structure, such as an invalid object number or object size. NOTEs indicate that either a duplicated object number has been found or a minor inconsisency in the structure has been found, probably indicative that the compacter was working in that area of the heap. SEARCHes are printed when a pattern match is made (and only if the scan match button was used to start the scan).
Hopefully the facilities provided by the browser will enable an error to be located and the page on which it occurs be made the current page. When this is the case editor mode should be entered to make corrections to the current page. This is done by clicking the EDITOR button at which point the editor viewer will replace the browser.


Editor Mode - Page Table
GVPatch maintains a table with an entry for each page of the heap. Entries in this table can be in one of three states
i) the page is not present in the local heap
ii) the page is present and is the same as the server page
iii) the page is present and has been altered, i.e. is different to the server page

The editor works only on the contents of the current page and the browser must be reentered to change the page on which the editor works.


Editor Mode - Reading Pages
The first thing to do on entering editor mode is to get the entire contents of the current page. This is done by clicking the get-page button. This gets the page from either the local copy of the heap if the entire page is to be found there, or from the server if not. If brought from the server it is also copied into the local heap and in either case the page is placed in the edit buffer which is then displayed on the screen. When edits are made they are applied in the first instance to the edit buffer, then when the button put-page is clicked the edit buffer is copied into the local heap and a note is made that this page has been altered. If it is decided that the edits made should be discarded then the button reset-page will always fetch the page from the server. At any time clicking get-page will read from the local heap into the edit buffer and thus discard any edits have been made since the last put-page. The button show-page will re-display the contents of the edit buffer. The buttons get-next and get-prev are convenient ways to get pages adjacent to the current one. They work in terms of position number. To get at other pages browser mode must be reentered.


Editor Mode - Display Format
The format of display is 256 words displayed as 32 rows of 8 words initially printed in octal. The first two words of the page are an offset field, non-zero when the first sub-object on the page is a continuation of the object on the previous page. Thereafter object headers are printed in bold text and each occupy 3 words. The first word is the size printed in octal while the remaining two words are not shown and their printing positions hold the object type and whether the object number is valid or malformed. The display format of all other words on the page may be selected to be either characters or octal words. The third word on the page must always be an object header and subsequent object positions on the page are determined from the size fields of previous objects.


Editor Mode - Display Modes
Editing and display mode operations are performed by pointing at words in the edit buffer with the current selection. There are four operations for displaying in detail a data structure within an object. These work by using the first word that the cuurent selection touches as the start of the structure. The structure is then displayed in context in the message window and the selection is extended over the entire structure. The button Object shows a sub-object in detail while RName shows a Mesa string. Time shows a time stamp while Size uses the word as a word size and extends the selection by the value of the word.
The buttons Chars and Words change the display style of words in the buffer in the obvious way. Note however that show-page must be clicked after the use of these buttons in order to re-display the buffer in the new formats.


Editor Mode - Making Edits
In order to replace words in the buffer the button Values-> is used to input a sequence of items. These items may be octal words or characters. Characters must be surrounded by double quotes and if the character string contains an odd number of characters the final word will be padded with zero. Be aware that the input "xyz" "abc" will need four words while "xyzabc" needs three. The current selection indicates the words to be updated and the button Replace does the update and re-displays the buffer. The number of words updated is the minimum of the number of words implied by the value string and the number in the current selection. A log of the words replaced and their replacements is kept. There is also a facility whereby a user defined procedure may be called to perform edits. This is invoked with the RUN extern button and described later in this document under the heading External Editing Facility. Be aware that edits are made in the first instance only to the buffer and though they appear in the log only take effect when put page is clicked. Each time put page is used the action is logged and thus it is possible to deduce what has actually happened to a page from the log.


Server Mode - Writing the Heap
When corrections have been made to the local heap and it is thought to be correct then it may be written back to the server. Theis is done by entering server mode once more and clicking write-heap. Only those pages which have been altered since the heap was last brought from the server will be written back. This operation alters the page table so that all pages present in the local heap are deemed to be the same as the corresponding server pages. Thus a second click of write-heap would not cause any pages to be transferred.


Server Mode - Restarting the Server
In order to restart the server the button RESTART must be clicked. When the server gets the restart command it will return to the Alto Executive and if the file Rem.cm exists it will use it as a command file. If the button command-> is clicked a command line can be input which will be placed in the file Rem.cm and sent to the server when the RESTART button is clicked. The default command line is @Server which is the standard way of starting up a server. The command line can accommodate up to 80 characters.

ServerMode - Other Facilities.
The server typescript contains a log of all major operations performed in this session of GVPatch, in order to get a permanent copy of this the write-log button may be clicked which will copy the typescript to the file GVPatch.log. Note that the previous contents of this file will be lost when this is done. It is sometimes the case that files on the server become longer than necessary for normal operation and so the button Set-server allow the length of a file to be altered. The length is entered using the length-> button, two decimal integers are needed, the first is the number of pages in the file the second the number of bytes (ie. for a file of 612 bytes enter "1 100"). The file name is input with the file-> button. The button Set-local performs the same operation for a local file.

Leaving GVPatch
The menu item destroy will stop the GVPatch program at the driving end, the server end is only stopped by an explicit RESTART. When destroy is clicked GVPatch tidies up and writes the page table to the file heap.map. When GVPatch is started again the state of the local heap will be the same as when the last session stopped.


External Editing Facility.
GVPatch exports two procedures which allow the user to gain access to the edit buffer. The procedure GVPDefs.GetEditBuffer returns a REF to a SEQUENCE of words copied from the edit buffer. The SEQUENCE contains only those words covered by the current selection and the REF will be NIL if the edit buffer is empty or the selection is not in the edit buffer. Having gained access to these words the user's program may change them and the call the procedure GVPDefs.PutEditBuffer to copy the SEQUENCE back into the buffer and log the changes. PutEditBuffer takes two arguments, the first is the REF to the SEQUENCE (NIL is acceptable if no edits were made) the second is a rope which will appear in the log and may be used to remind the user what was done. The facility is used by placing a (compiled) program which calls the above procedures in the file GVPatchExternal.bcd and then clicking the RUN extern button. The file GVPDefs.bcd must be present in order to compile the program. A sample program which sets all words in the current selection to zero is presented below . . .


-- GVPatchExternal.mesa
-- Steve Temple, November 2, 1982 11:00 am

DIRECTORY GVPDefs: TYPE USING [GetEditBuffer, PutEditBuffer, WordSeq];

-- WordSeq: TYPE = RECORD[SEQUENCE length: CARDINAL OF CARDINAL];

GVPatchExternal: PROGRAM IMPORTS GVPDefs = BEGIN

buffer: REF GVPDefs.WordSeq ← GVPDefs.GetEditBuffer[];

IF buffer#NIL
THEN BEGIN
FOR i: CARDINAL IN [0..buffer.length) DO buffer[i] ← 0 ENDLOOP;
GVPDefs.PutEditBuffer[buffer, "zeroed"]
END
ELSE GVPDefs.PutEditBuffer[buffer, NIL]

END.