So You Want to Write a Cypress Application? By Jim Donahue and Rick Cattell Last updated February 7, 1983 Warning: we have not yet completed this document as of February 1983; there are known holes in it, and anyone trying to use it in building a database application will certainly find them. Our goal is to make it reasonably easy for a Cedar programmer to build a new database application in a relatively short period of time. The documentation for the Cedar database system, called Cypress, can be found under separate cover (see Rick for latest version). We will assume some familiarity with that documentation and with Cedar itself, particularly with the Viewers screen manager. Our purpose here is to cover general issues in the development of a database application, and to describe some tools we have built that may be helpful, namely the Squirrel system. We we will discuss Cypress, Squirrel, and Viewers, and then talk some about their intended (and unfortunate) interactions. This document consists of four main sections. In Section 1, we describe Cypress and its applications in general. In Section 2, we cover the use of the Squirrel Tool, the portion of Squirrel which the end user sees, and the common conventions seen by the end user of our family of database applications. In Section 3, we cover the Squirrel Manager, the program interface to Squirrel we use to coordinate and simplify our programs. 1. Cypress and Database Applications The most important of the underlying software packages is Cypress. It is described in [Indigo]MLD.press; see Rick for the latest copy. Some simple examples of the use of Cypress are given there. In writing a new application, you need to do two things: 1. Design a database schema for your application. Some hints at how to do this are given in the Cypress documentation; you should also seek out other people who have done applications for hints on ways to do cleanly the things you have in mind. 2. Learn how to use the DB operations. These are described in the Cypress documentation -- the Cedar interface is found in DB.mesa in the latest Cypress.df (on [Indigo]Top>). There are certain idioms that are helpful; talk to Rick Cattell, Willie-Sue Haugeland, John Maxwell or Jim Donahue if you have questions about the best way to do something. The program [Indigo]Test>VLTest1Impl.mesa gives and example of a simple database application, as does the Cypress documentation. These applications do not use Squirrel. Cypress segments and your database application The Cypress system provides a mechanism for segmenting databases into independent physical files so that applications do not step on each other's toes. This independence is a great advantage, although it is not without cost. The main disadvantage is that cross-application data references are now a little trickier than they were in the old Cedar database system; we discuss a component of Squirrel that simplifies cross-segment references in Section 2. Another side-effect of our segment scheme is that applications must be assigned a segment name and number by a CSL "database administrator", as the name space for segments is global. The current database administrator is Rick Cattell. An application may be assigned more than one segment (we can currently have up to 1024 segments), and the segment files may be local or remote. Sometimes more than one application want to share a segment, in which case they must both know the segment and schema item names. When more than one application shares a segment, it is wise for them to do so by sharing a common interface and implementation through which access to the common data is performed. In fact, even in the case of a simple application it is wise to package up the portion of your code which does database access in a single interface; this makes your program more independent of changes in the logical representation of data. Segments allows the schema and data for applications to be separate, therefore you may define your application schema in any way you find convenient for your application without regard for other applications. We discuss the definition of an application data schema in the next section. Your data schema We have adopted in some conventions in our applications to make our code more understandable. Generally, the data schema is defined in an interface, with an Initialize procedure which does the DeclareRelations, DeclareDomains, etc. to define them. For domain and relation names, we use the same names for the Cedar Mesa variables as for the database items they represent. We capitalize domain names, and use lower case for relations and attributes. Attribute names, whose names in the database system are unique only in the context of their relation (just as Mesa record field names are unique only in the context of their record type), are given names that are the concatentation of the relation and attribute names. Examples: Person: Domain_ DeclareDomain["Person", $MySeg]; Document: Domain_ DeclareDomain["Document", $MySeg]; author: Relation_ DeclareRelation["Friend", $MySeg]; authorOf: Attribute_ DeclareAttribute[author, "of", Person]; authorIs: Attribute_ DeclareAttribute[author, "of", Document]; Unfortunately one may not generally provide the Cedar Mesa declaration and the database declarations of schema items on the same line as we have above, because it may be necessary to re-initialize the schema, e.g. when a transaction is closed and re-opened. We discuss this problem in the next subsection. Initialization and re-initalization One tricky aspect of writing a database application is getting the initialization of your global data structures correct. Note that you do not in general know whether this is the first time a user is running your program. The NewOrOld option the DB interface provides in the definition of schema items is useful in this regard, and you should probably use it: if a relation or domain does not exist, it is created, otherwise the existing one is used. The NewOrOld option may also be used in defining the segment in which your application's data resides. The NewOrOld option is not a panacea, however. If you change your schema, for example, you must be very careful. The current Cypress implementation does not always check that your schema is consistent (it cannot in the general case, at least in the sense of checking that you are aware the semantics of schema items you have not declared). You must therefore be prepared in a new release of you database application to: (1) write a program which updates your database from the old schema to the new, or (2) reload the database from a schema-independent log. We discuss the latter alternative in the next subsection, and the latter in Section 2. We have made some progress towards simplifying both of these alternatives, though there is much work to do. Note that your schema (and any other database entities or relationships your application maintains) must be re-initialized in the event of aborting a transaction, or closing and re-opening it. It therefore behooves one to put this initialization in one place, and call it whenever a transaction is re-opened. We provide a mechanism in Squirrel to allow you to register this initialization procedure so that an aborted transaction or segment open/close may automatically be handled by Squirrel if desired. Application logs You may find it useful to keep a log of all database updates made by your application to its segment. We are building a common package in Squirrel to simplify the maintenance of this log for an application. The common package provides facilities for rebuilding your application's segment from its log, and also for keeping pointers into the log from the database, e.g. so that your application need not store large pieces of text as database strings. [not completed] 2. The Squirrel Tool and User Interface Conventions In this section we describe what the user sees. The fundamental philosophy of the Squirrel system is that it should be possible to form a loose confederation of database applications which appear integrated to the user. To make this possible, it must be necessary for the database to be self-describing, so that applications need not build in all of the knowledge of the structure of the database. Moreover, there must be a natural means of invoking other applications, even though the number and structure of such applications is not know in advance and may change over time. There must also be a means of broadcasting the occurrence of database events that require applications to update critical data structures, for instance when the current transaction has been aborted. The facilities of the DB interface of Cypress and the Nut and SquirrelOps interfaces of Squirrel provide these functions. In the next section we discuss each of these interfaces. Here we discuss the tools provided by Squirrel to make it easier to build and debug new applications and the rights and responsibilities of each application in a communicating confederation, and present some of the user-interface decisions made by the developers of the first applications (Walnut, Whiteboards and Hickory) to try to present a consistent interface. Squirrel and our application programs use three primary types of application viewers: displayers, editors, and queryers. A displayer presents information in the database about a database entity and provides a command menu to perform appropriate operations on the entity. For example, for an electronic message displayer, the message's header and body would be displayed, along with commands such as Answer, Forward, and Delete. An editor looks much like a displayer, but allows the user a form with which to modify the information about the entity, and usually has a different set of commands. For example, a message editor is created in response to a NewForm or Answer command in the Walnut system; the user fills in the fields and invokes the message editor's Send or File command. A queryer also looks like a displayer and editor, but the user may fill in the fields with values, boolean expressions of values, or other application-interpreted information; the queryer represents all entities in its domain which satisfy the constraints. When the user invokes the query, the entities satisfying it are displayed and may be browsed or printed. The Squirrel tool provides basic database functions: commiting or aborting a transaction, dumping or loading a database segment, or erasing portions of a database. The Squirrel tool also allows invocation of a displayer, editor, or queryer on a particular domain or entity. If no application has registered itself for the given domain, Squirrel invokes its default displayer, editor, or queryer. These default displayers, editors, or queryers may be used for various special purposes without the need for developing a database application, although we do not yet have the manpower to provide support for Squirrel as a robust end-user application. The default displayer The default displayer shows the domain and name of the entity displayed in the title of the window it allocates; all relationships in the database that reference the entity are shown in the main body of the window. The relationships are diplayed in the form relation attribute1: value1 attribute2: value2 ... attributeN: valueN The attribute that references the displayer's entity is not shown. String, integer, and boolean valued attributes are displayed in the obvious way. For entity values, the entity name is displayed and may be selected with the center (yellow) button on the mouse to invoke a displayer for it. The default editor The default editor also displays the domain and name of the entity being edited at the top of the window. It differs from the default displayer in that a form is provided in the body of the window for the entity, showing not only those relationships which already reference the entity, but "blank" relationships for relations that could reference an entity of its domain; these blank fields of such a relationship can be filled in by the user. (If the editor window is on a new entity, then all the relationships shown will be blank.) The editor window also provides menu items for changing the name of an entity or committing the edits of the current entity. The default queryer The default queryer has not been completed at the time of this writing. When it is completed, it will provide a form much like the default editor, providing blank forms for the kinds of relationships an entity of a given type may participate in. Unlike an editor, the form represent a generic set of entities of the given domain which satisfy the constraints fillied in by the user. When the special ProcessQuery button is pressed, a Viewer appears on the screen listing the entities which satisfy the query. These entities may then be examined by the ordinary browsing capabilities of Squirrel. Displayers for domains and relations In addition to displayers and editors on ordinary data items, Squirrel implements special displayers and editors for domains and relations. The domain displayer shows the attributes in the data schema that can reference an entity from the domain, the sub-types and super-types of that domain, and all of the entities of the domain sorted by name. The editor for a domain shows only the sub-types and super-types. The editor automatically copies entities and renames domains if necessary to achieve the effect of a requested change (the underlying database system does not allow defining sub-types of a domain that already contains entities). The displayer for a relation shows a table whose columns are labelled with the names, types, and uniqueness constraints of the relation's attributes; the rows of the table show all of the relationships in the relation. The editor for a relation shows only the attribute names, types, and uniquenesses. The user may also delete attributes of the relation or create new ones; the relation editor will automatically copy all the relationships into a new relation with the same name, since the underlying database system does not allow changing attributes of a relation after relationships exist. Note that these default displayers and editors cannot be used once your application has registered its own procedures for these operations -- bugging display through the Squirrel window uses the normal Nut.Display procedure. However, the Squirrel window also provides a "Debug" menu item; when Debug is invoked, all currently registered procedures are saved and the default procedures used until Debug is invoked again, when all of the application procedures are restored. So, when debugging your application, you can still rummage around in the database when your code fails. (The variable "debug" in the Nut interface is set in this case; thus, if you don't want to get the Squirrel window up just to set this variable, evaluate the expression "NutImpl.debug _ TRUE" using the executive.) Other operations that can be invoked from the Squirrel window The Squirrel window also provides menu items for other important database operations. Reset/Save You can abort or commit the current transaction by bugging "Reset" or "Save". Open/Close The "OpenDB" button opens the database specified in the Database field of the control window; the database name must be of the form "[Local]DatabaseName" or "[Juniper]DatabaseName" (at present Juniper is the only remote file server on which one can store databases -- Alpine is coming, folks!). Load/Dump The Load button causes the file "DatabaseName.dump" (where DatabaseName is the name of the currently open database) to be read and its contents added to the database. The format of these files is given below: /Domain\name\nametype\ ...for each domain /Relation\name\ ...for each relation /Attribute\name\relation\type\cardinality\length\ ...for each attribute /domain\name-elt1\name-elt2\...\ ...for each entity /relation\attr1:val1\attr2:val2\...\ ...for each relship The operations that perform dumping and loading can be found in the NutDump interface; they include DumpToFile: PROC[fileName: ROPE _ NIL, dl: LIST OF Domain _ NIL, rl: LIST OF Relation _ NIL, complement: BOOLEAN _ FALSE]; -- dumps only the domains and relations given -- if complement = TRUE, dumps everything but the domains and relations given DumpAll: PROC[fileName: ROPE _ NIL] = INLINE {DumpToFile[fileName, NIL, NIL, TRUE]}; LoadFromFile: PROC[fileName: ROPE _ NIL]; -- Loads whatever is in the given file; uses same dump format as DumpToFile. Additionally, the NutDump interface provides operations to perform selective dumping; these are WriteSchema: PROC[s: STREAM, dl: LIST OF Domain, rl: LIST OF Relation]; WriteSystemEntity: PROC[s: STREAM, e: Entity];-- use for Domains, Relations, and Attributes WriteSystemRelship: PROC[s: STREAM, r: Relship]; -- schema for a SubType relship WriteDomain: PROC[s: STREAM, d: Domain]; -- writes all the entities of the domain WriteEntity: PROC[s: STREAM, e: Entity]; -- writes a particular entity WriteRelation: PROC[s: STREAM, r: Relation]; -- writes all the relships of the relation WriteRelship: PROC[s: STREAM, r: Relship]; -- writes a particular relship EraseDomain(s)/EraseRelation(s)/EraseAll If things get badly screwed up, it may be necessary to go in and perform radical surgery on your database. To do this, the Squirrel window provides operations that erase selected domains or relations or wipe the entire database. The EraseDomain(s) and EraseRelation(s) operations read the Name field of the Squirrel window for a list of domain or relation names -- all the entities in the named domains (and all of their associated relationships) or all of the relationships in the named relations will be deleted. EraseAll makes the entire database go away -- it asks for confirmation first. Again, these operations also can be accessed through a program by importing the SquirrelOps interface: the procedures to be used are EraseDomains: PROC[dl: LIST OF DB.Domain]; EraseRelations: PROC[rl: LIST OF DB.Relation]; EraseDataBase: PROC[newTrans: BOOL_ TRUE]; Registered commands from Squirrel Squirrel also registers several commands that are useful, particularly when debugging. They are (with the optional arguments specified in brackets): DBDump [file _ ] [~] [[Domains: ] domain1 ... domainn] [Relations: relation1 relationm], which dumps the named domains and relations to the specified file (or DB.dump if none is given). The "~" before the list of domains and relations means "dump all except those named"; this is particularly useful if you want to dump all of the database that does not belong to one particular application. DBEraseAll, which erases the entire database, DBLoad [file], which loads the database from the given file (or DB.dump if none is given), DBShow domain entityname, which displays the named database entity. Application Conventions To make a set of applications developed by different people at different times look to a user like a "system" it is necessary to adhere to some conventions for use of mouse buttons, menu names, window layout, etc. Below, we discuss the current choices made by the original application developers; note that they are more in the spirit of suggestions than hard and fast rules. Selections and Mouse Buttons Squirrel and application programs support a common interpretation of mouse buttons for the user. This common interpretation is as follows. Button Interpretation Red Select the entity or datum value at the current cursor position Blue Extend a selection made by red to the current position Yellow Open a new displayer on the entity at the current position Ctrl-Red Delete the relationship at the cursor position Ctrl-Blue Extends the action of Ctrl-Red over a number of relationships Shift-Red Paste the entity or datum value at the cursor into the current input focus Shift-Blue Extends selection with Shift-Red Others Currently application-defined The yellow button was dedicated to the "show me" interpretation because this was deemed the most common operation upon a selection. To avoid proliferating windows on the screen, the Squirrel windows follow the following convention for opening new windows when a user yellow-selects an entity: 1. If the user has yellow-selected an entity in this window before and the displayer window thus created is still on the screen, re-use the previous displayer window. Note that the existing applications all use a "Freeze" menu item to allow a user to "hide" a viewer to prevent it from being reused; see the section on menu items below. 2. Otherwise, create a new entity displayer on the screen for the selected entity, in non-iconic form on the left-hand side. The interpretation of the mouse buttons was chosen to be as compatible as possible with existing interpretations in the Cedar Viewers environment. There is in fact some overlap. For example, one can select an entity by selecting the entire window (or icon, if in iconic form) for its displayer. It can then be used by a command, e.g. to add it to a set of entities in another window. Menu Items and their ordering We are trying to make the menu items and their position in the menu consistent among all applications (and be as consistent with Tioga as possible). Thus, the menu for all display or edit windows should begin with the following items (reading from left to right): Clear. When bugged with the left mouse button, it should replace the current viewer with an empty one for the domain; when bugged with the middle mouse button, it should create a new viewer. Reset (guarded). Undo all edits to the viewer. Freeze. Prevent this viewer from being reused. AddSelected. Following the example of whiteboards and Hickory, AddSelected takes the selected viewer and associates the corresponding database entity with the one currently displayed. Save. Save all edits; note that this is necessary for every window (even displayers), since one can always perform an "AddSelected". An interface will be built into Squirrel that provides a menu having all these items (and in the right order). 3. Database Application Manager Your application will interact with the database through the DB interface; it will interact with other applications through the Nut and SquirrelOps interfaces, which are implemented by Squirrel. The basic idea of Squirrel is that each database application is responsible for certain domains of entities; when an entity is to be displayed or edited or when a database query is to be performed on a particular domain, Squirrel passes on the request to the application that has registered itself for the domain. Applications register their displayer, etc. and call other such procedures through operations in the Nut interface. Squirrel also provides some general-purpose database tools; these include: 1. Windows for simple browsing, editing and querying of the database. 2. A facility to dump and reload databases. The dump format is human-readable text, and contains the database schema as well as the data. 3. A facility for examining and modifying the database schema, automatically re-organizing existing data for simple schema changes. These tools are not currently intended for users of Cypress applications -- they are invaluable, however, to the developer of new applications for they allow him to browse in and change a database without having to depend on his (possibly failing) application code. These tools can be invoked through the Squirrel window or through operations in the SquirrelOps interface. Additionally, Squirrel implements facilities to allow database applications to broadcast and to be informed of important database event. These facilities include: 1. Opening the database transaction and notifying application programs when the database or a database file (segment) is closed or re-opened, or when a transaction is aborted. 2. Notifying applications programs when a change is made to the database, allowing them to update data structures or screen display. The operations of opening or closing the database and committing or aborting the current transaction are invoked through the SquirrelOps interface; the operations to broadcast the intent to perform such actions (so other applications can do whatever they must when these things happen) are in the Nut interface. Finally, Squirrel provides a screen management package with convenient procedures for setting up database application windows; these operations are found in the Palm and ViewerNut interfaces. The Nut Interface Each database application program, such as Walnut or Hickory, registers itself with Squirrel by passing procedures to be called when an event of interest to the application occurs. The application may register a displayer, editor, or queryer procedure to be called when the user requests one for a particular domain. Walnut registers such for the message domain, for example. An application may also register itself to be called when a database file / transaction is opened or closed, or when database relations are updated. Registering, displaying, editing and querying is all done through the Nut interface (Nut.mesa in the current [Indigo]Top>Squirrel.df file). Additionally procedures are provided to broadcast to all registered applications that relationships have been added to or deleted from the database or that important database events are about to occur or have just occured. Displaying, Editing and Querying There are three procedures in the interface that invoke displayers, editors and queryers when called: Display: PROC[ e: DB.Entity, parent: Viewer_ NIL, method: Method _ oneOnly, init: REF ANY_NIL] RETURNS [nutRec: NutRec]; Edit: PROC[ d: DB.Domain, eName: ROPE_ NIL, parent: Viewer_ NIL, method: Method _ oneOnly, init: REF ANY_NIL] RETURNS [nutRec: NutRec]; Query: PROC[d: DB.Domain, init: REF ANY_ NIL] RETURNS [nutRec: NutRec]; The Display procedure displays a database entity in a viewer chosen by the following rules: 1. If the method is Nut.Method[replace], replace the parent with a new viewer. If a new viewer is to be created, the Create procedure that is registered for the domain is called to produce one (see below). 2. If the method#Nut.Method[replace] and the parent#NIL, then use the last viewer spawned for this entity. To give the user some control over whether viewers are to be replaced, most applications provide a "Freeze" menu item that "hides" a viewer that was spawned -- more on this is given below in the section on Menu items. 3. If the method is Nut.Method[oneOnly], look to see if there is already a viewer for this entity. The init parameter is for client-defined data; it is passed through to the creation and display procs that have been registered for the domain (see below). The Edit procedure doesn't take an entity as an argument because the entity may not currently exist. Finally, the Query procedure simply is given the domain upon which a query is to be performed. Broadcasting Updates The Nut interface also provides a procedure Nut.Update to broadcast to other interested applications that important changes have been made to the database: Update: PROC[updateType: UpdateType, tuple: Relship, clientData: REF ANY_ NIL] UpdateType: TYPE = {create, destroy} When Update is called, the update procedures that have been registered for each domain will be called (of course, not all applications will find it necessary to register an update procedure for each domain); again the clientData argument is for any other client-defined data. Registering An application calls Nut.Register to become the displayer, editor or queryer for a particular domain and to register update and notify procedures for domains: Register: PROC[ domain: ROPE, display: DisplayProc_ NIL, -- to make displayer for entity in domain edit: EditProc_ NIL, -- to make editor for new or old entity in domain query: QueryProc_ NIL, -- to make queryer on domain create: CreateProc_ NIL, -- to create viewer for above (defaults to container) update: UpdateProc_ NIL, -- to call when database is updated notify: NotifyProc_ NIL -- to call when database closed, commited, aborted ]; DisplayProc: TYPE = PROC[ e: DB.Entity, newV: Viewer, init: REF ANY_NIL] EditProc: TYPE = PROC[ d: DB.Domain, eName: ROPE_ NIL, newV: Viewer, init: REF ANY_NIL] QueryProc: TYPE = PROC[ d: DB.Domain, newV: Viewer, init: REF ANY_NIL] CreateProc: TYPE = PROC[ nutType: NutType, dName, eName, defaultLabel: ROPE, init: REF ANY ] RETURNS [v: Viewer] UpdateProc: TYPE = PROC[updateType: UpdateType, tuple: Relship] NotifyProc: TYPE = PROC[whyNotified: NotifyType] If no application has registered a procedure for a particular action on some domain, a default procedure is provided by the Squirrel implementation; furthermore, new registrations will supercede previous ones unless the new procedures are NIL. To completely deregister an application, call the Deregister procedure: DeRegister: PROC[domain: ROPE] which deregisters all procedures for the domain. The function of the Create procedure was mentioned above -- if a new viewer is needed to display or edit an entity, the Create procedure registered for the domain is called (if none is registered, then a default one is called that creates a container with a default name using the acorn icon). If you want to provide your own Create proc (and most applications do), you must observe the following rules: 1. The viewer should not be filled in or painted yet -- this will be done by the Display or Edit procedure that will be called after your Create proc returns. (Of course, you may set the name of the viewer, whether it is to be iconic or not and its iconic form, or the viewer's property list.) 2. If your Create proc decides not to create a viewer, then it should return NIL. It will sometimes turn out that to decide whether to create a new viewer, you need to see what other viewers are on the screen (both for your application and possibly for others). To allow you to do this, Squirrel provides the following facility. Enumerating the application viewers To find out the state of the display (all of the viewers Squirrel currently know about), call: EnumerateNutList: PROC[proc: PROC[n: NutRec] RETURNS [continue: BOOL]] NutRec: TYPE = REF NutRecObject NutRecObject: TYPE = RECORD[ viewer: Viewer, nutType: NutType, domain: DB.Domain, lastViewerSpawned: Viewer, eName: ROPE ] This has to be done with some caution -- in particular, do not do any operations that might change the viewers on the screen until after EnumerateNutList is finished; you will almost certainly wedge the system if you do so. The approved usage is to save your own list of NutRec's while enumerating the list and then operate on the list you made. If you want to force the screen to clear (perhaps before closing the database), then call DestroyNutList: PROC[forceDestroy: BOOL] RETURNS[allDestroyed: BOOL] which destroys all the nuts on the screen, unless told not to. Now we turn to a discussion of the default facilities Squirrel provides. Startup and event notification When an application starts up, it should register itself and inititialize its schema and other global variables. (See below on how the initialization of the schema should be done.) As part of the registration, the application should register a NotifyProc that behaves as follows. On AfterAbort or AfterOpen, ALL application variables and windows must be refreshed; this means for example, reestablishing any properties that a window has that refers to database entities (this means that it may be better to refer to entities by saving the name of the entity [and the name of its domain] in such instances). On BeforeClose, applications should either take down their windows or somehow disable operations on them. (the easiest thing to do is just to take down the windows) Otherwise, your application must be prepared to catch signals that happen when a DB operation is attempted with the database closed. Handling of errors When an application starts up, it should register itself and inititialize its schema and other global variables. (See below on how the initialization of the schema should be done.) As part of the registration, the application should register a NotifyProc that behaves as follows. Packages to make your job easier Palm MBQueues