[Cyan]<Cedar6.0>Documentation>DFToolDoc.tioga!1

DFToolDoc.tioga

Last Edited by: Russ Atkinson on June 8, 1983 [Previous DFFiles documentation]

Last Edited by: Rick Cattell on May 15, 1984

Last Edited by: Subhana, May 30, 1984 2:39:58 pm PDT

DF TOOL

CEDAR 5.2 — FOR INTERNAL XEROX USE ONLY

Cedar DF Tool Documentation

Release as: [Indigo]<Cedar>Documentation>DFToolDoc.tioga

Abstract: This documentation describes the Cedar DF Tool, that handles DF files.

XEROX Xerox Corporation
Palo Alto Research Center
3333 Coyote Hill Road
Palo Alto, California 94304

For Internal Xerox Use Only

Introduction

The DF Tool is a general package for file management with explicit version control. It uses DF files, which are lists of filenames, one per line, specifying a remote host and directory where the file is normally stored, followed by the file's creation date. (In certain situations some of this information can be omitted.)

The DF files system was initially used by people running Mesa on public Dorados who wanted to guarantee they had the correct version of files they needed and as an easy way to save changed versions of file without unnecessary copying. It is now being used to partially automate the Cedar and Mesa group release process as well. Eric Schmidt was primarily responsible for the programs and the initial versions of this documentation.

The DF Tool provides three basic operations:

BringOver - will insure that the files listed in a DF file are retrieved from their remote file servers, possibly overwriting different versions already on the local disk.

SModel - once BringOver has been run and a few files have been changed (possibly by editing them), SModel will store the newest versions on the remote file servers and produce a new DF file containing references to the newest versions.

Verify is applied to a Cedar .Bcd file and a DF file and verifies that all files needed to re-build that .Bcd are listed in the DF.

This document is intended as the definitive documentation for the commands and procedures described here. Please send messages with comments, questions, and discrepancies to CedarSupport^.PA. Messages about changes to the DF files system are sent to CedarImplementors^.PA and/or CedarUser^.PA.

We also include, at the end of this document, information on use of DF files in the Cedar release process.

Simple DF File Format

A simple DF file is composed of some control lines and lines of the form:

Directory [host]<directory-path> <file name and version> <create-date>

For example,

Directory [Indigo]<Poplar>Cedar>pl.mesa!13 2-Oct-80 15:43:09 PDT

specifies a specific version number for the file server, and also lists the file's create-date. Note that a file cannot have the name Directory, and upper- and lower-case differences are ignored.

Blank lines are OK and lines are treated as comments if they begin with "//" or "--".

The format of DF files is similar to that produced by the List command in the Command Tool. You may make it almost identical by setting the options in the FileTool so that only the file name and file create dates are listed.

Example DF File:

An example file might be named "Poplar.DF" and contain:

-- File Poplar.DF, created 5-Oct-80 19:48:32

Directory [Indigo]<Poplar>Cedar>

poplar.bcd!16 3-Oct-80 13:26:28

checkpoint.bcd!3 24-Sep-80 11:14:26

AFiles.bcd!5 2-Oct-80 17:17:32

overviewdefs.bcd!6 2-Oct-80 17:16:16

coredefs.bcd!7 2-Oct-80 17:16:20

filepageusedefs.bcd!5 2-Oct-80 17:17:28

filepageusedefs.mesa!1 26-Oct-79 12:45:39

filesystemdefs.bcd!5 2-Oct-80 17:17:31

filesystemdefs.mesa!1 22-Oct-79 13:21:59

Cedar File System

We assume the reader is acquainted with the Cedar file system. The normal mode of operation in Cedar is to create attachments to remote files rather than copying them to the local disk. The local disk is just a cache. The attached files may then be referenced by their [short] file names rather than the full path name including the server and directory names. If attached files are subsequently modified, new versions are automatically created on the local disk. The remote versions must later be updated to reflect these "dirty" files in the cache.

The DF Tool Bringover operation is used to create the initial attachments, and the SModel operation to update the remote versions. When we speak of "retrieving" a remote file in this documentation, we mean that an attachment is created to the remote file. (The remote file may be copied to the local disk, in addition, as an option.) When we speak of "storing" a file, we mean that the local copy of the file, if it has been modified, is copied back to the remote directory.

DF Tool

The DF Tool window is divided vertically into five areas:

(1) In the top area, only two of the buttons are implemented: the Do It button, and the Stop! button. The Do It button causes the DF Tool to execute the operations specified by the user via areas (2) and (3) described below, and the Stop! button causes it to abort the action currently in progress.

(2) In the Operations area, the user may select one of the three operations Bringover, SModel, and Verify, and enter a DF file on which the operation is to be performed. The DF file(s) item may be selected with the mouse to automatically cycle through various built-in directories that are commonly used. If a working directory is specified, the operation takes place to or from that directory (if you don't know what a working directory is, ignore it or see Cedar FS documentation).

(3) The Options area changes depending on the operation selected in (2). The options will be described in the following sections. You may cycle through the alternatives for most of the parameters, e.g. the Access option in Bringover mode, by selecting the item label.

(4) The Confirmation area is used to display information about the state of the DF Tool and ask for confirmation. The client may select Auto-Confirm or individually confirm each operation.

(5) The Session Log area provides a transcript of the operations performed.

BringOver

The BringOver operation works as follows: It reads the DF file one line at a time. It takes the remote file name, strips off the directory information, and looks to see if it is on the local disk. One of three things can happen:

(1) If the file is not on the local disk, BringOver will try to retrieve it.

(2) If the file is on the local disk, BringOver looks at the version on the local disk. If the create-date on the local file differs from the the create date listed in the DF file, BringOver will try to retrieve the remote version.

(3) If the create-date is omitted from the DF file, BringOver will ALWAYS try to retrieve the file.

If you omit the IFS version number (e.g. "!3"), BringOver will enumerate all the versions of that particular file looking for one with the correct create time. If there are no versions of the file you list in the DF file on the remote host in the directory you specify, BringOver will give you a warning message. If there are files with the same name and none of the create-dates avaliable match that listed in the DF file, BringOver will give you a warning and offer to retrieve the latest version.

After using BringOver you can be sure the files listed in the DF file are on your local disk, and that their create-dates agree with the create dates listed in the DF file, or BringOver will have printed out error messages.

Normally BringOver will list each file to be retrieved and will ask for confirmation. (You may confirm each retrieval individually or use the Auto-Confirm option.)

BringOver can read a remote DF file as easily as a local one, e.g.

[Indigo]<Poplar>Poplar.DF

Comments may be indicated by lines beginning with // or --, as in the Poplar.DF example above. In addition, BringOver will print comments at the end of the file after the last line listing files when it is run. This way distributors of software can print out useful reminders to their users.

Instead of giving create dates

If the create date entry is a "~=" rather than a normal date, BringOver will retrieve the file only if the version on the remote server is different than the version on the local disk. If the file is not on the local disk, it will be retrieved. For example,

Foo.Mesa ~=

If the create date entry is a ">" rather than a normal date, BringOver will retrieve the file only if the version on the remote server is newer (or more recent) than the version on the local disk. If the file is not on the local disk, it will be retrieved. For example,

Foo.Mesa >

will retrieve from the remote server only if there is a newer version on the remote server.

Indirect Use of Other DF Files

There are two common, distinct uses of one or more DF files by another. The first is syntactic inclusion, similar to macro-expansion, where a DF file references another and all programs treat these DF files as if they were a single DF file with the text of the inner DF file substituted in the outer one at the point of reference. The second use is as an "Importer" of another system, package, or set of files controlled by someone else. By analogy with these concepts in Mesa, there are Exports as well.

The DF file syntax allows both forms of use, which are indicated by different keywords before the name and date of the nested DF file.

Syntactic Inclusion

A line of the form

Include [host]<path>file.DF Of <date>

will cause BringOver to invoke itself on file.DF at the point it encounters the Include statement. If the included file itself has an Include statement, then BringOver will again invoke itself on the inner DF file, and so on, in a recursive fashion. Furthermore, the DF file is retrieved using the usual BringOver rules before the recursive call.

Examples of Include:

Include [Indigo]<Cedar>CompilerSources.DF Of 2-Oct-81 15:43:09 PDT

Include [Indigo]<Cedar>CompilerObjects.DF Of >

(The Of keyword is optional, and the Include statement may be split on multiple lines.)

Exports and Imports

DF files users are encouraged to use a single DF file for packages they maintain. Some, but usually not all, of the files an implementor needs are also needed by users, or clients, of the package. Files that are useful to both users and implementors are said to be exported. These files are indicated by being listed below a statement similar to the Directory statement:

Exports [host]<directory>

Directory [host]<directory>

BringOver normally examines all the files in a DF file (i.e. it assumes you are an implementor), as the Access option in the Options area defaults to all. If you are a user, you can get only those marked Exports by specifying Access: public.

When a DF file needs to refer to other DF files as a user, it may have an Imports statement

Imports [host]<path>file.df Of <date>

which causes BringOver to (1) retrieve file.df to the local disk if necessary and (2) examine all files marked Exports in file.df and retrieve them if necessary. Of course file.df may have Include or Imports statements, so this is a recursive algorithm.

Sometimes the Exports files are too few or too many files and the user of a package would like to list the files he/she needs from the imported DF file. Appending a Using clause to the Imports statement, analogous to the one in the Mesa language, gives the user explicit control over which files he/she needs:

Imports [host]<path>Package.DF Of <date>

Using [list of files, separated by commas]

(The Of <date> clause can be omitted.)

Use of the Using clause is strongly recommended, but is not required.

Examples of Imports:

Imports [Indigo]<Cedar>Pilot>PilotMesaFriends.DF

Imports [Indigo]<Cedar>CoPilot>CoPilot.DF Of 24-Sep-81 11:14:26 PDT

Using [CPSwapDefs.Bcd, CPSwap2.Bcd]

As with the Include clause, the Imports statement (and Using clause) may span multiple lines.

In the CoPilot.DF example above, CPSwapDefs.Bcd and CPSwap2.Bcd can appear anywhere in CoPilot.DF. (It does not matter whether they are Exports or not when a Using clause is used to access them.)

The files referred to by an Imports statement may themselves be exported to the next outer DF file by preceding the word Imports by the word Exports. This is useful when users of a package you maintain need to have files from some other package in order to (e.g.) compile their system. You can import and export these files to assure your users will have them.

Using Clause Semantics

Since, in general, CoPilot.DF (in the example above) may have internal structure (i.e. it references other DF files with Include and Imports lists), CPSwapDefs.Bcd and CPSwap2.Bcd may be defined in some of the DF files referenced by CoPilot.DF. To avoid an explosion in search time, these restrictions apply when using Using lists on a DF file with internal structure:

(1) A using list file (e.g. CPSwapDefs.Bcd) may be retrieved from a DF file included by an imported DF file (e.g. included by CoPilot.DF). (In other words, included DF files are treated as if they were macro-substituted inline in the top DF file.)

(2) A using list file may be retrieved from a DF file imported (without a Using list) by an imported DF file (e.g., imported without a Using list by CoPilot.DF) as long as the file is exported in the inner DF file. (I.e., imports are significant one level below the top DF file, CoPilot.DF.)

(3) A using list file may be retrieved from a DF file imported with a Using list by an imported DF file (e.g. imported with a Using list by CoPilot.DF) as long as the using list file being retrieved also appears on the inner using list. (I.e., multiple, nested Using lists must intersect.)

Useful Modes

If specific file names are entered in the Selected files item, only those files will be retrieved by the DF Tool. (Note: you might have to say Access: all to get them.) If no "." is given, BringOver will append ".Mesa".

The Origin option may be used to reduce the number of files that are retrieved when BringOver is run. Origin: derived causes BringOver to retrieve only those files that are "derived" from source files, i.e. files that end in ".Bcd", ".Signals", ".Boot", and ".Press". Origin: source causes BringOver to retrieve only non-derived files, i.e. all files that do not end in ".Bcd", ".Signals", ".Boot", and ".Press".

Other information about BringOver

When the filename on a line in a DF file is preceded by a tilde ("~"), BringOver will retrieve the indicated version if it is not present on the disk. If there is already a version on the disk, then it will not be overwritten, regardless of its create date.

BringOver will abort itself if the Stop! button is pressed.

BringOver can handle DF files with fewer than 500 files.

BringOver can be given a list of DF files. It will behave as if each file had been given separately.

SModel

The SModel command (SModel stands for "Simple Modeller") can be used to store back new versions of files you've changed since you last ran the BringOver command to fetch files under control of a DF file. For example, if your DF file is called "Poplar.DF" and you've already run the "BringOver" command, the SModel command will execute as follows: The files listed in Poplar.DF are checked on the local disk. If any have different create dates SModel will store them on the remote servers specified in Poplar.DF. Then SModel produces a new Poplar.DF file with the new create dates and remote file system version numbers (e.g. !4).

If a file on the local disk is listed without any create date in the DF file, SModel will fill in the create date from the version on the local disk and then store the file out on the remote directory. If the file is listed in the DF file followed by a ">" or "~=", this file is ignored and will not be transferred.

If the DF file contains a reference to itself (e.g. System.DF lists "System.DF" as one of the files you want), then SModel will also store a new version of the DF file on the remote server. Since SModel has to write out a new DF file before it can store the DF file, SModel cannot put the IFS version number (e.g. !5) on the DF file name in the case of self-referencing DF files. However, BringOver will always get the correct version of the DF file since it will use the create-date of the DF file instead.

SModel will pause before storing each file and ask if it should go ahead. You may respond with the Yes button to go ahead and store it or No to skip the store. The Auto-Confirm button may be used to answer Yes to all confirms. The Stop! button may used to abort the SModel altogether.

ReadOnly files

When you don't want any of the files in a directory to be changed by SModel, regardless of any changes you may make, you can indicate the files in the directory are read-only by preceding the Directory keyword with ReadOnly. For example:

ReadOnly Directory [Indigo]<Mesa>System>

indicates the files listed below should NEVER be stored back on <Mesa>System>.

SModel Options

Two boolean options are provided in the SModel Options area that appears when the SModel operation is selected:

Check existence on server. When Yes is filled into this item, SModel will check that the files actually exist on the remote server. If a file is not on the remote directory, or the version listed in the DF file is not on the remote directory, then SModel will store the file. This is slower, but safe. Otherwise SModel does not enumerate the remote file system, so SModel may not detect that certain files listed in a DF file are not present on remote servers.

Store Changed Files. When Yes is filled into this item, SModel will store files to make the remote files and df consistent with the new local versions. If you simply want to check whether new versions exist, you may fill in No.

Beware potentially confusing interactions between the FS file system and the DF Tool. When Bringover "retrieves" a file, it merely enters it in the local directory as an attachment. You must NOT delete the remote files while they are still referenced by an attachment! To move the files to another directory, change Directory statements in the file to the new destination, and do an SModel with the Check existence on server option. If no one else is using the old DF file, you may delete the old files it references.

Include and Imports

SModel will invoke itself recursively on DF files that are specified in Include statements (and non-ReadOnly DF files preceded by @.) SModel will NOT invoke itself on DF files listed in Imports statements (or on DF files listed in ReadOnly directories preceded by @.)

If the Include or Imports statement is not followed by an Of <date> clause, SModel will insert such a clause in its output DF file with <date> replaced by the create date of the file on the local disk.

Getting Started with SModel

New users are often confused about the relationship of the entries in a DF file to the local and remote directories, and what SModel will do in certain cases. The easiest way to understand it is that SModel assumes 1) that the DF file was an accurate description of the remote directory at some point in the past and 2) files with differing create dates that it finds on the local disk are the "Truth" and that it should transfer any files that are different. However, assumption #1 allows SModel to assume that files with the same create date in the DF file and on the local disk ALSO exist on the remote server. This assumption is often not true when a new DF file is being created. To get started, there are at least these things you can do:

1. Make a DF file of just file names (no create dates) and run SModel on it. The files will be transferred and the create dates will be filled in.

2. Use Verify (see below) to automatically generate a DF file, edit the DF file as you need to, and use SModel with the Check existence on server option to store all the files out onto remote servers.

If you are maintaining a system for others, you may need to have entries in the DF file that are ReadOnly. Filling in all the create dates by hand is painful, so you may use the Store Changed Files: No option to fill in all the create dates in the DF file but not transfer any of the files.

CameFrom handling

SModel changes entries like

Directory [host2]<directory2> CameFrom [host1]<directory1>

into lines like

Directory [host1]<directory1> ReleaseAs [host2]<directory2>

and changes Imports like

Imports [host1]<directory>file.DF Of <date>

CameFrom [host2]<directory2>

into

Imports [host1]<directory>file.DF Of <date>

since the CameFrom is inserted by the Release Tool.

Other Information about SModel

SModel will abort itself if the Stop! button is pressed.

If a file is transferred, the RemoteFileName property (see above) is added to the leader page of the file on the local disk to record where a remote copy can be found.

Verify

The Verify operation attempts to answer the question: Does this DF file have entries for all the files I need to rebuild this program? Verify will analyze a .Bcd file and compare the files recorded in the .Bcd with entries in a DF file. If a file entered in the .Bcd is not in the DF file, Verify will give a warning message about the omitted file(s).

Select the Verify option, and enter your DFFile.DF into the DFFile(s) item, where DFFile.DF is suppposed to be a complete description of all the source files that were compiled and bound to produce "Package.Bcd", and all the object files produced while building "Package.Bcd". Verify will analyze "Package.Bcd" and give warning messages about those files needed to build Package.Bcd not listed in DFFile.DF. If Package.Bcd was produced by the Binder, Verify will complain if the .Config file or if any of the implementation modules listed in the .Config are not also listed in "DFFile.DF". If Package.Bcd was produced by the Compiler, VerifyDF will complain if the .Mesa source file or any of the Definitions Files (.Bcd's) it depends on are not listed. (A list of files omitted is written to the screen and to a file "MissingEntries.DFFile.DF$". Similarly, .Mesa, .Bcd, and .Config files that are not actually required to build "Package.Bcd" are listed.)

Once it has analyzed Package.Bcd, Verify will then analyze any .Bcd's needed by Package.Bcd (described above). This analysis will continue until all .Bcd's that are referenced in the closure of the dependencies among .Bcd's are analyzed.

If the DF file is omitted, Verify will put a list of the top-level dependencies onto "MissingEntries.DF$", which can then be given a more suitable name and Verify can be invoked again. This is a simple way to bootstrap a complete DF file for a package.

Verify will look on remote file servers for the correct version of a Bcd file and will use the Leaf protocol to read it page by page, so the Bcds do not necessarily have to be on the local disk for Verify to do its job.

Other Information about Verify:

The DF file that describes a package may optionally have entries preceded by a "+". If Verify is invoked without a .Bcd file it will look inside the DF file and assume that any file name preceded by a "+" is a top-level Bcd and proceed to analyze it and all its dependencies, as described above. More than one .Bcd may be preceded by a "+" sign. For example, if DFFile.DF had an entry of the form

+Package.Bcd

then Verify will analyze "Package.Bcd" and all its dependencies.

The DF file(s) involved do not need to be on the local disk.

Verify will abort itself if the Stop! button is pressed.

ReleaseTool

The ReleaseTool should be run by the Cedar Release Master. Contact me for more information.

Cedar Release Procedures

[The following section is an edited version of a set of messages written by Roy Levin and Russ Atkinson. Although the procedures described below are required of Cedar implementors, many of the extensions added to DF files will be helpful for non-Cedar users as well. Although it represents a restriction on DF file usage, I recommend non-Cedar users follow this style as much as possible.]

Background

Releasing large systems is a tricky business. SDD has known this for a long time and has developed elaborate release procedures to cope with the matter. We're only just discovering how messed up things can get. However, since we have many fewer human cycles to expend on this problem than SDD does, it is imperative that we devise natural, easy-to-use, and efficient (of people time) procedures for our releases of Cedar.

Rules for DF files

In order to preserve the sanity of the various Cedar users and implementors, the following conventions are used for release components. It is a good idea to remind implementors of these rules from time to time.

In the following rules ® denotes the release version. For example, for the Cedar 5.1 release ® = "5.1", and Cedar® = "Cedar5.1". Further, © denotes the component name. For example, if the component is the Compiler, then © = "Compiler".

General

All submitted components must have passed verification (no errors) by the Verify command of the DFTool. This is the most crucial step in making a timely release.

All submitted components must have their top-level DF files on [Indigo]<Cedar®>Top>, and have those files released to [Indigo]<Cedar®>Top>.

For each component named ©, there should be a command file named Make©.cm (not an export). This file should contain commands that drive the compiler and binder (and whatever else is needed) to reconstruct the component from its sources.

Imports

Imported DF files should always come from some subdirectory of [Indigo]<Cedar®>, and almost always come from [Indigo]<Cedar®>Top>.

Imports should use ~= as their date format.

Exports

The ©.df file itself should be exported, using:

... [Indigo]<Cedar®>Top> ReleaseAs [Indigo]<Cedar®>Top>

The Tioga form of the documentation files should be exported, using

... [Indigo]<Cedar®>Documentation> ReleaseAs [Indigo]<Cedar®>Documentation>

All exported files except top-level DF files and documentation files should use

... [Indigo]<Cedar®>©> ReleaseAs [Indigo]<Cedar®>©>

The major public interface files should be exported, both source and object.

Files required by the implementation (such as *.tip, *.style, *.icons) to be in the local name space should be exported. These file should be commented as such in the DF file.

If the package is not in the boot file, there are two more requirements:

The main package implementation file (i.e. Compiler.bcd) should be exported only if it is not in the full boot file.

For each command that would be registered with Commander by the component's start code, e.g. "Solve", a text file named Solve.load should be exported. This file should contain the commands necessary to get that command registered (and perhaps others too). For example, Compiler.load contains a single command:

Run Compiler.bcd

No other files should be exported.

Documentation

Documentation files need only have the Tioga form.

Release Procedures

Since the DF file is the sole descriptive mechanism presently at our disposal, we will use it in a stylized way as a "poor man's system model". Cedar implementors will observe the following conventions in producing components for release as part of the Cedar system:

(1) Each component will be described by a DF file, which may, in turn refer to other DF files. (Indeed, cross-reference is encouraged, as will be obvious shortly.)

(2) A DF file should mention (directly or indirectly) every file needed to build and use the component it describes. This requirement does not apply to the standard construction programs (PGS, Compiler, Binder, TableCompiler, Packager, MakeBoot) except in the DF files that define them, but such programs may be mentioned if circumstances seem to warrant. Thus, it should be possible to build the component by starting with a "bare disk" (i.e., one with nothing but the standard construction programs and BringOver), and, by applying BringOver to the DF file, acquire all the pieces necessary to build the component.

(3) All files that form an intrinsic part of this component (e.g., the .mesa and .config files and their derived .bcds) should appear in the DF file under Directory headings that are NOT ReadOnly. This applies also to miscellaneous command files useful in building the component as well as the DF file for the component itself. All other files mentioned in the DF file should appear under ReadOnly Directory headings. Thus, a ReadOnly Directory heading says, in effect, "the following files are 'imported' from other components", and a non-ReadOnly Directory heading says, "the following files comprise a part of this component".

(4) It is perfectly permissable to mention a superset of the files actually needed to build the component. For example, instead of painstakingly discovering all of the Pilot public definitions files needed by your program, you could simply include

Imports [Indigo]<Cedar>Pilot>PilotMesaPublic.df Of ~=

in the DF file. There are several convenient DF files of this sort available, e.g.,

[Indigo]<Cedar>Pilot>

PilotMesaPublic.df
PilotMesaFriends.df
ComSoftPublic.df

[Indigo]<Cedar>Tajo>

TajoPublic.df
TajoFriends.df

and others may appear soon.

(5) The "end result" of building a component is usually a single BCD that implements the component. You should use Verify, which takes a BCD and a DF file and determines whether or not the latter mentions all the files necessary to build the former. For the purposes of checking the completeness of a release component, it is useful to have a DF file indicate the BCD(s) named within it that represent(s) the "end result". The DF file syntax will allow the character "+" to appear immediately before a file name. BringOver, SModel, etc. all ignore this character, but the release-checking software uses it to prepare the input for Verify. All "top-level" BCDs in a DF file defining a release component should be marked with a "+" (more than one per DF file is permitted). The author/maintainer of a component should apply Verify to it before submitting it to the release master.

(6) When the component has been built, the (non-ReadOnly) Directory headings in the DF file identify the location of the files intrinsic to the component. However, when the component is released, this location is changed to (an appropriate subdirectory of) the release directory. The DF file syntax has been extended to enable Directory headings to identify both locations, e.g.,

Directory [Indigo]Levin>SpyBuild ReleaseAs [Indigo]<Cedar>Spy>

The release automation software uses this information to move the files under this heading to the indicated place on the release directory and to update DF files (including others that reference the original directory with ReadOnly) accordingly. The ReleaseAs clause is only appropriate for non-ReadOnly Directory headings, and all DF files involved in a release must have ReleaseAs clauses in all of their non-ReadOnly Directory headings.

(7) It is highly desirable to have a single DF file identify which of the files that comprise a release component are of interest to the client of the component and which files are of interest to the implementor. Several syntactic changes to DF files and the BringOver command line together provide the facilities to do this. They are best explained by an example. Consider the following fragment of a DF file for the Spy:

-- Spy.df

Exports [Ivy]<Levin>SpyBuild> ReleaseAs [Indigo]<Cedar>Top>

Spy.bcd <date>

SpyNub.bcd <date>

SpyClient.bcd <date>

Directory [Ivy]<Levin>SpyBuild> ReleaseAs [Indigo]<Cedar>Spy>

Spy.df <date>

SpyImplA.bcd <date>

SpyImplA.mesa <date>

. . .

Imports [Indigo]<Cedar>CoPilot>CoPilot.df Of <date>

If a user of the Spy wants to retrieve the public files of the Spy, he/she may use the Bringover command with the public option, the implementor with the all option. The Bringover with the all option will retrieve CoPilot.df and, since its contents are imported, BringOver will recur on the contents of this DF file. However (the nested instance of) BringOver will retrieve only the files in CoPilot.df that appear under Exports headings. Thus, the implementor of the Spy will get all of the Spy's files, but only the exported portions of CoPilot (which is appropriate since the Spy is a client of CoPilot). To summarize: Exports in place of a Directory heading identifies files that are of interest to the client, i.e., those files that will be retrieved by BringOver /p. Imports causes BringOver to modify its behavior when recurring to retrieve only files identified as Exports. If /p is omitted, all files are retrieved, except that, once BringOver enters "exports only" mode, all nested retrievals of DF files will be exports only, regardless of the subsequent appearance of Include statements in nested DF files. This has the natural, intuitive behavior.

(8) As each implementor completes a component needed for an upcoming release, he/she sends a message to the release master containing the name of the DF file that defines the component. The implementor then must leave the component's constituent files undisturbed until the release master notifies him/her that the files have been safely moved to the release directory.

The following summary of qualifiers that appear in DF files may be helpful:

Qualifier Interpreted by Ignored by

------------ ------------------- ---------------

ReadOnly SModel, Verify BringOver
Exports BringOver SModel, Verify
Imports BringOver SModel, Verify
Include BringOver, Verify, SModel
ReleaseAs ReleaseTool BringOver, Verify, SModel

Guidelines for Date Specifications

Within a DF file that is intended to conform to the guidelines for release:

(a) files that are an intrinsic part of the component defined by the DF file (i.e., those appearing under non-ReadOnly Directory headings) should specify explicit dates,

(b) files which the component imports (i.e., those appearing under ReadOnly Directory headings or those imported DF files) should specify "#" (or equivalently "~=").

However, there is an important exception to (b). It occasionally happens that a single person is concurrently developing both the implementation of an interface and a client of the same interface. (Example: a collection of BCD manipulation facilities is imported by both the compiler and the binder. Ed Satterthwaite is the implementor of the BCD facilities as well as these two illustrious clients. When developing a new version of the BCD stuff, Ed frequently and rapidly switches between his roles as implementor-of-BCD-package and implementor-of-client-of- BCD-package.) In such situations, specifying "#" in the client's DF file will cause the wrong thing to happen; it was to accommodate this situation that the ">" form of date specification was originally added to DF files. Implementors who find themselves in this predicament should use ">" instead of "#" where necessary.

It should also be understood that when the ReleaseTool is constructing the DF files to be stored on the release directory, it will uniformly replace "#" and ">" specifications by explicit dates. This is important for a release, since it is intended to be a snapshot of the state of the files at some instant of time, and the loose binding implied by "#" and ">" is inappropriate. During the development process, however, these "dynamically bound" date specifications are very useful, and it is unreasonable and unnecessary to expect people to perform the manual translation to specific dates when release time arrives.

How, then, does this transformation of dates fit in with the ongoing development process? When an implementor finds it necessary to change a component following a release, he/she has the choice of starting from the DF file on the release directory or from the one used to build the component in preparation for the release. (Of course, the latter may validly disappear after the release; only the DF file on the release directory is guaranteed to be retained.) In the former case, the maintainer will have to edit the released DF file to make it convenient for subsequent development, e.g., by adjusting Directory headings to reference working directories and by replacing specific dates with "#" or ">". In the latter case, the DF file is already (still) suitable for development work. If a particular component is frequently changed and usually by the same person, the latter approach will likely be followed. If the component is infrequently modified or if many different people have occasion to change it, the former approach will be less error-prone. The choice should be made on a package-by-package basis.

Complete Syntax Description of DF Files

Directory [host]<path>

Exports [host]<path>

ReadOnly [host]<path>

Any of the above can be followed by ReleaseAs [host]<path> or CameFrom [host]<path>. Exports, ReadOnly, and Directory can be combined on one line. (Public is a synonym for Exports.)

filename <date>

Multiple lines consisting of filenames and (optionally) create dates may follow Exports, ReadOnly, and Directory headings. filename may be preceded by "+", "@", and/ or {PublicOnly}. The filename may have a directory path (but no host), a short filename, possibly followed by a version number. The <date> can have the usual form, "~=", "#", ">", or be omitted.

Include [host]<path>file.df!version Of <date>

The Include statement may be followed by a ReleaseAs [host]<path> or CameFrom [host]<path> statement. The Of <date> is optional.

Imports [host]<path>file.df!version Of <date>
Using [ list of filenames, separated by commas ]

The Imports can be preceded by Exports, the Of <date> and Using [ ] can be omitted, and the Using clause may be preceded by a CameFrom statement.

Semantics of DF Files

All DF programs in this set use a consistent view of what a line in a DF file means. In summary, here is a list of ways to specify files in a DF file:

filename!vers <date>

Any file on the remote directory with create time <date>. If necessary, Df programs like BringOver will enumerate the remote server looking for such a file. The version number vers (if present) is used as a hint.

filename!vers >

filename!vers #

filename!vers ~=

The highest version (!H). If ~= or # is given, a program like BringOver will retrieve filename from the remote directory if its create time differs from that of the file on the local disk. The version number vers (if present) is used as a hint. > is like ~= except that the file will be retrieved only if its create time is newer.

filename!vers

When no date is present, and a version # is given, then this refers to the file filename on the remote server with version number !vers.

filename

When no date and no version number is present, then it is treated like ~=. (Except by SModel, which fills in the local create time.)

The cases for DF files that are included or imported are exactly analogous.

For example,

Directory [Indigo]<Cedar>Top>

Spy.Bcd!2 1-Jan-82 11:11:11

Refers to a file created on 1 Jan 82. Version # is a hint.

Spy.Bcd!3 ~=

Refers to the highest version on the directory. The version # is ignored.

Spy.Bcd!4

Refers to version 4.

Spy.Bcd

Refers to the highest version on the directory.