HOW TO USE WALNUT WALNUT 4.2 How To Use Walnut Version 4.2 Release as [Indigo]Documentation>HowToUseWalnut.tioga, .press Came from [Indigo]Manual>HowToUseWalnut.tioga, .press By Willie-Sue Haugeland Last edited By Willie-Sue Haugeland on June 8, 1983 9:36 am Jim Donahue on June 2, 1983 1:17 pm Abstract Walnut is a computer mail system interface that runs in Cedar. It provides facilities to send and retrieve mail (using the Grapevine mail transport system), and to display and classify previously retrieved messages. Walnut is under active development. This document describes how to obtain and use Walnut 4.2, the latest version of Walnut released with Cedar 4.2. How to Use Walnut: Contents 0. Introduction 1. Database structure 2. User interface 3. Becoming a user 4. Coping with releases and crashes 5. User profile options 6. Shortfalls and wishes [If you are reading this document on-line, try using the Tioga Levels and Lines menus (if you can) to initially browse the top few levels of its structure before reading it straight through.] XEROX Xerox Corporation Palo Alto Research Center 3333 Coyote Hill Road Palo Alto, California 94304 DRAFT  For Internal Xerox Use Only  DRAFT 0. Introduction Walnut is a mail system that runs in Cedar. Since most Cedar users are familiar with Laurel, it may be useful to describe Walnut in terms of Laurel. Like Laurel, Walnut provides facilities to send and retrieve mail (using the Grapevine mail transport system), and to display and classify previously retrieved messages. Many of Laurel's other facilities (such as file copying and program running) are not included in Walnut, since they are provided directly by the User Executive. Laurel implements a specialized file structure, the "table-of-contents", that gives random access to stored messages. Walnut uses the Cypress database system to maintain this information. In future releases, Walnut will make more use of the underlying database structure. In particular, the database will help Walnut to integrate smoothly with new applications, such as a calendar system or an online "white pages". Walnut is under active development. As an early user of Walnut you can expect new features to appear frequently, and the user interface to change occasionally. Though the underlying database structures will also change from time to time, these changes will not require any hand-editing; a conversion program will do the work. 1. Database Structure We shall describe a user's model of Walnut's database. This model suppresses many details whose understanding would be required in writing a new application on Walnut's database, but are irrelevant for accessing the database through Walnut's user interface. Walnut's database contains two entity types: message and message set. A message entity corresponds to a message retrieved from the Grapevine mail transport system. Like all database entities it has a name, consisting of the sender's RName concatenated with a unique message ID provided by Grapevine. A message also has several immutable properties: its sender, its subject, and so on. Its unread property is a BOOL whose value is TRUE when the message is first stored in the database, and is set to FALSE when the message is first displayed. A message entity is also a member of one or more message sets. A message set entity is named by a text string containing no embedded blanks. There are two distinguished message sets: Active and Deleted. A newly-retrieved message is made a member of Active. A message that is removed from all other message sets is added to Deleted. Using the Active and Deleted message sets in this way ensures that each message belongs to at least one message set. 2. User Interface Walnut implements four viewer types: the Walnut control viewer, the message set display viewer, the message display viewer, and the message composition viewer. When Walnut is running there is one Walnut control viewer, and any number of instances of the other viewer types. The iconic form of the Walnut control viewer is a mailbox. Anything that can be done with Walnut can be done by starting from the control viewer (sometimes by creating other viewers). Walnut's user interface attempts to be consistent with conventions used elsewhere in Cedar. Clicking LEFT on a button representing a Walnut entity (a message or a message set) "selects" the entity (makes it an implied parameter to other operations); clicking MIDDLE on such a button opens the entity (displays more information somehow). This is analogous to the behavior of icons in Cedar. Unless otherwise specified, hereafter "click" means "click with the LEFT mouse button". 2.1 Message sets The message sets in a Walnut database are represented by buttons in the Walnut control viewer. The Active and Deleted message set buttons always appear first; other message set buttons appear in alphabetical order. There may be several rows of these buttons. To create a new message set (containing no messages) type its name into the MsgSet: field of the control viewer, then click Create. A message set button with this name will appear. Similarly, to delete an existing message set, type its name and click Delete. Walnut requests confirmation if the message set contains any messages. If you delete a message set containing messages, the messages are deleted from the set (as if clicking each message with CTRL-LEFT as described below) before the set is destroyed. To find out how many messages are currently in a message set, type the name of the set and click SizeOf. To create a message set display viewer, click MIDDLE on the corresponding message set button of the control viewer. The iconic form of a message set displayer is a stack of envelopes. The message set displayer contains a one-line button for each message in the set. Each message button looks much like a line in the top window of Laurel: it shows the date of the message, the name of the sender (or To: field if the sender is the current user), and the message subject. At most one of the message buttons in a set is selected (shown with a grey background). To select a message, click LEFT on it; to select and display a message, click MIDDLE on it. To delete a message from the message set, click CTRL-LEFT on it; its button will disappear. (Note that if this message belonged to no other message set, it is added to the Deleted set, and hence is still accessible.) A message set displayer also contains several command buttons that operate on the selected message. Categories lists (in the Walnut control window) the message sets in which the selected message appears (a message can simultaneously be in several message sets). MoveTo adds the message from the message set to all of the message sets selected in the control window, after first deleting it from the message set. Display displays the selected message. Delete deletes the message from the message set; if it thus becomes a member of no other message sets, then it is moved to the Deleted message set. AddTo adds the messages to the selected sets without first doing the deletion. Finally, the Active message set includes a NewMail button that reads new mail and adds the messages read to this message set. For the MoveTo, Delete, and AddTo buttons, LEFT-clicking simply performs the operation described above, while RIGHT- or MIDDLE- clicking performs the operation and displays the next message in the set. 2.2 Messages As described above, a message can be displayed by clicking MIDDLE on a message button of a message set displayer. This creates a message display viewer, whose iconic form is an envelope (clicking SHIFT-MIDDLE on a message button causes the created viewer to fill the entire column). The message within such a viewer is not editable. This viewer is associated with the message set that created it, so clicking MIDDLE on another message of the same message set shows this new message in the same message displayer. This is designed to avoid a proliferation of message displayers. Of course, there are times when you really want to create viewers on several different messages in one set. Clicking a message displayer's Freeze button (which then disappears) permanently binds the message to the message displayer. If all message displayers for a given message set are frozen, then MIDDLE clicking in the message set creates a new displayer. A frozen message displayer cannot be unfrozen, but can be destroyed. A message displayer has several other buttons. The Categories button is the same as that on message set displayers. Answer and Forward create a Walnut Send viewer initialized either with a proper heading for an answer to the message or a copy of the message for forwarding. The Print button uses the TSetter to print the message -- it will complain (in the Walnut control window) if the TSetter is not loaded. And the remaining Split, Places and Levels buttons are from Tioga. 2.3 Sending mail To create a message composition viewer click the NewForm menu item in the Walnut control viewer, or use the Answer or Forward buttons on a message display viewer. In iconic form this viewer is the back of an envelope, with the title "WalnutSend ...6/02/83". The message composition viewer is a Tioga viewer for typing in the message header fields and message body. NewForm, PrevMsg, and GetForm menu items are guarded while the Sender is dirty. When a command requires confirmation, appropriate buttons appear in the viewer menu area; the control window is used for messages. Clicking the Clear button of a message composition viewer causes its message text to be replaced by a standard message form. PrevMsg restores the contents of the last successfully sent message. GetForm loads into the viewer the file whose name is currently selected (if no file extension is given and no file is found with the selected name, the extension ".form" is used); it also sets the input focus at the first placeholder. Confirm and Deny buttons appear for confirmation if the existing message text has been edited. StoreMsg stores the contents of the viewer in the selected file; this gives a way to save partially composed messages. The Split, Places and Levels menu items are the usual Tioga operations; if a message composition viewer is split, then NewForm, PrevMsg and GetForm operations on a split viewer will cause all but the one bugged to be destroyed (the message composition viewer becomes "unsplit"). This will also happen if Answer or Forward reuse a split message composition viewer. Clicking the Send button initiates the sending process. If the message is addressed to a public distribution list or to more than twenty individuals, it probably should contain a Reply-To: field. If it does not, a message is printed in the control window and buttons labeled Self, All, and Cancel appear. These buttons mean "Reply-To: sender", "no Reply-To: field", and "Reply-To: sender but don't send", respectively. If errors occur in sending, a message is printed in the control window. If the transmission is successful, the message is saved and a new form appears; RIGHT-clicking Send causes the message composition viewer to become iconic after syntax checking has been done. While a Send is in progress, the viewer is not editable. An AbortSend button is visible during part of the sending process, should you decide not to send the message. The last successfully sent message can to restored by clicking PrevMsg. Clicking the Destroy button asks for confirmation if there is an unsent message in the viewer. It is a bad idea to destroy a message composition viewer while message transmission is in progress. 2.4 Retrieving mail Walnut polls the mail servers at regular intervals. If there is new mail for the logged-in user, the Walnut control viewer displays a message like You have new mail at 27-Sep-82 14:28:45 PDT Clicking the NewMail button in either the control viewer or the Active message set viewer retrieves all the new mail. You can "button ahead" during message retrieval. The new messages appear as buttons at the bottom of the Active message set viewer, with a "?" in the leftmost column of each new message button to show that the message is unread. If the control window is iconic and there is new mail, the flag on the icon is raised; for convenience, there is a NewMail button in the Active MsgSet displayer, which will retrieve the waiting mail. 2.5 Global operations The Walnut control viewer has a few more buttons that will now be described. Clicking Commit commits all changes that you have made to Walnut's database. The only other operations that automatically commit database changes are Destroy and CloseAll. If you perform a Walnut operation that updates the database and then boot or rollback without doing a Commit, the effects of that operation are in the log and will be performed on the database and then committed when you next run Walnut. The CloseAll button furnishes a quick way of ending a session with Walnut. CloseAll destroys all message displayers, and closes all message set displayers. Then it executes a Commit and closes the Walnut control viewer. The Archive operation provides a way to copy a set of messages into a file that can later be read by either Walnut or Laurel. Type a filename into the OnFile: field and click Archive. All messages in the currently selected message sets are copied to the named file (if the file name has no extension, ".ArchiveLog" is assumed). Clicking the (guarded) Expunge button expunges the Walnut database: all messages in the Deleted set disappear without a trace. (Laurel purges deleted messages every time you Quit or change mail files.) Without the Expunge operation, Walnut's database would grow without bound. The Expunge operation may fail if your disk is nearly full; this is a motivation for regular Expunges. Section 4 contains more information on this topic. Finally, clicking the Destroy button causes Walnut to commit and then close the Walnut database. This is a good thing to do before booting or rolling back; it reduces the possibility of mangling the database. Walnut registers several infrequently-used commands with the Cedar Executive. All of the command names contain the prefix "Walnut", so typing Walnut*? to the Executive enumerates them. The Walnut command creates a Walnut control viewer if you should happen to Destroy yours. The WalnutExpunge command has the same effect as clicking Expunge, but can be used when the control viewer does not exist. The WalnutOldMailReader command is described in Section 3, and the WalnutScavenge command is described in Section 4. WalnutNewMail simulates clicking the NewMail button. 3. Becoming a User 3.1 Standard usage First, bring over the latest Walnut by typing Bringover /a /p Top>Walnut to the Executive; this will also retrieve WalnutSend, Cypress, and AlpineUserImpls. These latter files will be loaded by Walnut (AlpineUserImpls only if needed). Edit your personal profile to contain all of the entries specified in WalnutDefault.profile (public in Walnut.df). The only profile entry that most users will want to experiment with is "InitialActiveRight: TRUE"; making it FALSE causes the Active message set displayer to create itself in the left viewer column, like all other message set displayers. To start Walnut, type Walnut to the Executive. This will spend a long time loading, but finally a Walnut control viewer will appear. You can include Walnut in a checkpoint. Be sure not to click checkpoint until the message "Walnut 4.2" appears in the Walnut control viewer typescript. Message and Message set displayers get updated after each rollback; if a displayed message or message set has since been deleted, the viewer will be destroyed. To read a Laurel or Hardy mail file, or a file created by Walnut's Archive operation, first run Walnut as just described. Then type WalnutOldMailReader {optional message set name} to the Executive. If you fail to specify a message set name, the messages will be placed in Active. If the specified message set does not exist, it will be created. 3.2 Using Alpine for Walnut database storage The Alpine server "Luther.alpine" can be used to store Walnut databases. Using Alpine improves Walnut performance somewhat, especially for operations that write to the Walnut database. It frees up space on the local disk that is otherwise occupied by the Walnut database. It also reduces the cost of using Walnut on a public machine (you move only the log to the new machine, not the log and database.) The drawback of using Alpine is that on occasion it may abort Walnut's transaction; Walnut will recover from this gracefully, but it may take some time to replay your uncommitted actions stored in the log. Transaction aborts are infrequent, so on balance the Alpine server is an improvement over the local disk. Contact Karen Kolling to obtain an Alpine account. Using Alpine changes the procedure for Walnut installation only slightly. Edit your personal profile to contain the entries: Walnut.WalnutSegmentFile: "[Luther.alpine]Walnut.segment" Walnut.WalnutLogFile: "YourName.WalnutDBLog" With your profile in this state, you must run AlpineUserImpls before invoking the Walnut command. One way of ensuring this is to include the items Run AlpineUserImpls; RunAndCall Walnut in your CommandsFrom: profile entry. It is ok to take a checkpoint after running AlpineUserImpls and Walnut. There are two ways to create a Walnut database on Alpine. The first is to follow the procedure above, in which case Walnut will notice the absence of a database and create one by scavenging from the log. The second is to copy the database from the local disk to the Alpine server, using the procedure call _ AlpineCmds.Copy[to: "[Luther.alpine]Walnut.segment", from: "Walnut.segment"] You must run AlpineUserImpls before attempting to call this procedure. For more information on Alpine operations consult [Indigo]AlpineUser>HowToUseAlpine.msg. 4. Coping with Releases and Crashes Walnut sometimes crashes because its database has gotten into a bad state. Also, a new release of Walnut or the Cedar database system will occasionally change the database format that Walnut understands. From Walnut's point of view these circumstances are very similar. Walnut's technique for recovering from database failures is the same as the approach taken by Laurel. In the Laurel world there are .mail files and -dmsTOC files. The .mail files have a very simple format: you can look at a .mail file, and even perform minor surgery on it, using a text editor. The -dmsTOC files have a more complex format that allows efficient random access to information the .mail files. If any inconsistency is detected in a -dmsTOC file, Laurel will reconstruct it by reading the corresponding .mail file. Walnut keeps a log of all retrieved messages and all database updates in a file called Walnut.DBLog (or the file named in your user profile; this discussion will use Walnut.DBLog as its generic name). The format of this file is an extension of the .mail format used by Laurel (and Hardy); you can load Walnut.DBLog into a viewer to see this. Walnut uses Cypress to keep the equivalent of the Laurel -dmsTOC files. A Walnut database can always be reconstructed by replaying a Walnut.DBLog file. If Walnut is not loaded, you can reconstruct the Walnut database by executing the command file WScav.cm, which is included in Walnut.df; when the command file completes, you will be running Walnut just as if you had used Walnut.cm. If Walnut is already loaded, you can scavenge by typing WalnutScavenge to the Executive. The Walnut.DBLog file grows with each retrieved message and database update until an Expunge command is given. This command writes a new Walnut.DBLog that only contains information about the messages that have not been deleted, and updates the database to be consistent with this. Because the cost of a full Expunge is proportional to the size of your Walnut log (which can grow to be quite large, if you are a "pack rat"), there is a "short cut" Expunge that only deletes messages from the point in the log of the previous Expunge; it is enabled by setting the Walnut.EnableTailRewrite user profile option. When enabled, LEFT-clicking Expunge causes the short-cut expunge to be performed, while RIGHT-clicking performs the full expunge. Note that with the tail-rewrite expunge any quite old messages that you delete (which appear before the expunge cut-off point) will not be removed from the log or database; thus, once in a while you will still need to do a full expunge. Jim Morris's advice concerning files on the local disk is to "keep your bags packed". A prudent individual will apply this philosophy to Walnut.DBLog. Include this file in a personal .df file (such as the one that contains your user profile), and make it a habit to SModel it every few days. (Someday soon you'll be able to keep both the log and the database on an Alpine server.) 5. User profile options Below is a complete list of all of the current Walnut user profile options (copied from UserProfile.doc): Walnut.ReplyToSelf: BOOL _ FALSE; if TRUE, causes walnut to automatically supply a Reply-To: field, if appropriate. Walnut.DestroyAfterSend: BOOL _ FALSE; if TRUE, causes sender to be destroyed after a successful delivery, if Send was clicked with RIGHT Walnut.InitialActiveRight: BOOL _ TRUE; true says to bring up the active message set on the right column, false on left. Walnut.InitialActiveOpen: BOOL _ FALSE; true says open a message set viewer on Active. Walnut.InitialActiveIconic: BOOL _ FALSE; if true and InitialActiveOpen = TRUE, then the Active message set viewer is opened as an icon. Walnut.MsgSetButtonBorders: BOOL _ FALSE; if TRUE, puts borders around the MsgSet buttons in the control window. Walnut.EnableTailRewrite: BOOL _ FALSE; if TRUE, performs "tail rewrite" on Expunge. Walnut.WalnutSegmentFile: TOKEN _ "Walnut.Segment"; value is the name of the file to be used for the walnut data base. Walnut.WalnutLogFile: TOKEN _ "Walnut.DBLog"; Name of log file. 6. Shortfalls and Wishes What follows is a listing of known deficiencies and contemplated extensions to Walnut. Nobody guarantees that everything listed below will be implemented. But the list does indicate some directions for future work, and may provide context for your own Walnut wishes. Send both bug reports and wishes to WalnutSupport^. 6.1 Message sets It would be nice to allow selection of more than one message in a message set (perhaps even spanning message sets). When a message set displayer is created, it should display the newest messages first (perhaps by painting from the bottom of the viewer towards the top), since these are most likely to be accessed. 6.2 Retrieving mail It seems desirable to make mail retrieval a continuous background activity. This would tend to insulate users from the response time of Grapevine. Once mail retrieval is implemented in the background, a natural next step is to provide some means for a user procedure to classify incoming mail according to its significance, file it in sets other than Active, let the user know the status of his new mail ("You have important new mail"). The procedure that stores new mail in the database should understand the In-Reply-To relationship. Eventually, users should be able to write queries or other commands that exploit this relationship. 6.3 Sending mail It should be possible to forward or answer multiple messages. This seems to require the ability to select multiple messages. Feedback from mail parse errors can be improved. When displaying a message containing a parse error or bad a recipient, the point of error in the message header should be highlighted. When sending a sequence of messages with the message composition viewer, you tend to click Send, wait for the feedback "sending ...", then make the viewer iconic (to reclaim the screen area) and finally click NewForm to create a new viewer. It would be smoother to reuse the same message composition viewer, but without waiting for the message to be sent (since this can take quite awhile). Since it is quite unusual to have two messages in transit (as contrasted with two messages being composed) at the same time, this can be achieved by passing responsibility for the message from the message composition viewer to the Walnut control viewer when message parsing is complete, and clearing the composition viewer for reuse. Any errors in transmission would be reported in the control viewer rather than the composition viewer. 6.4 Global operations Walnut needs a way to make queries. (This is what databases are all about!) For starters, we'd like to have something analogous to Laurel's SearchMail program for performing text pattern matching in the messages of a message set. The Walnut Expunge operation currently requires more resources (such as disk space) than Walnut requires to perform other operations on the same database. This is unfortunate, since it means that a user can get stuck in a situation where he must hand-edit his Walnut.DBLog in order to recover.