BringOverImpl.mesa
Copyright Ó 1984, 1985, 1986, 1991 by Xerox Corporation. All rights reserved.
Levin on December 19, 1983 4:18 pm
Russ Atkinson (RRA) January 20, 1987 6:07:08 pm PST
Bob Hagmann January 9, 1986 4:55:24 pm PST
Hal Murray, January 22, 1986 10:04:05 pm PST
Mike Spreitzer January 8, 1987 6:05:38 pm PST
Michael Plass, January 23, 1992 10:31 am PST
DIRECTORY
BasicTime USING [Period],
DFInternal USING [AbortDF, CheckAbort, Client, ClientDescriptor, DefaultInteractionProc, GetFileInfo, DoAbort, LocalFile, LocalFileInfo, RemoteFileInfo, ReportFSError, ShortName, SimpleInteraction, YesOrNo],
DFOperations USING [AbortInteraction, BringOverAction, BringOverFilter, DFInfoInteraction, FileAction, FileInteraction, InfoInteraction, InteractionProc],
DFUtilities USING [ClassifyFileExtension, Date, DateToRope, DifferenceOfUsingLists, DirectoryItem, FileItem, Filter, ImportsItem, IncludeItem, ParseFromStream, ProcessItemProc, RemoveVersionNumber, SortUsingList, SyntaxError, UsingEntry, UsingList],
FS USING [Close, Copy, Error, ExpandName, Open, StreamOpen],
IO USING [Close, GetIndex, PutF1, PutFLR, PutFR, PutFR1, STREAM],
Rope USING [Equal, Fetch, Flatten, Length, ROPE, Run, Substr];
BringOverImpl: CEDAR PROGRAM
IMPORTS BasicTime, DFInternal, DFUtilities, FS, IO, Rope
EXPORTS DFOperations = BEGIN
ROPE: TYPE = Rope.ROPE;
STREAM: TYPE = IO.STREAM;
BringOver: PUBLIC PROC
[dfFile: ROPE, filter: DFOperations.BringOverFilter, action: DFOperations.BringOverAction, interact: DFOperations.InteractionProc, clientData: REF, log: STREAM, workingDir: ROPE]
RETURNS [errors, warnings, filesActedUpon: INT ¬ 0] = {
client: DFInternal.Client = NEW[DFInternal.ClientDescriptor ¬ [
(interact ¬ IF interact = NIL THEN DFInternal.DefaultInteractionProc ELSE interact),
clientData,
log, log,
workingDir
]];
retrievedFiles: REF DFUtilities.UsingList ¬ NIL;
InRetrievedList: PROC [shortName: ROPE] RETURNS [BOOL] = {
FOR i: NAT IN [0..retrievedFiles.nEntries) DO
IF Rope.Equal[shortName, retrievedFiles[i].name, FALSE] THEN RETURN [TRUE];
ENDLOOP;
RETURN [FALSE];
};
ConvertFilter: PROC [f: DFOperations.BringOverFilter] RETURNS [filter: DFUtilities.Filter] = {
filter ¬ [filterA: VAL[f.filterA.ORD], filterB: VAL[f.filterB.ORD], filterC: VAL[f.filterC.ORD]];
IF f.list ~= NIL THEN {
nEntries: NAT ¬ 0;
i: NAT ¬ 0;
FOR l: LIST OF ROPE ¬ f.list, l.rest UNTIL l = NIL DO
nEntries ¬ nEntries.SUCC;
ENDLOOP;
filter.list ¬ NEW[DFUtilities.UsingList[nEntries]];
FOR l: LIST OF ROPE ¬ f.list, l.rest UNTIL l = NIL DO
file: ROPE = DFUtilities.RemoveVersionNumber[l.first];
IF ~(filter.filterA = $all OR DFUtilities.ClassifyFileExtension[file] = filter.filterA) THEN
NotFoundWarning[file];
filter.list.u[i] ¬ DFUtilities.UsingEntry[name: file];
i ¬ i.SUCC;
ENDLOOP;
filter.list.nEntries ¬ nEntries;
DFUtilities.SortUsingList[usingList: filter.list, nearlySorted: FALSE];
};
};
NotFoundWarning: PROC [file: ROPE] = {
warnings ¬ warnings.SUCC;
DFInternal.SimpleInteraction[
client,
NEW[DFOperations.InfoInteraction ¬ [
class: $warning,
message: IO.PutFR1["'%g' could not be found in any nested DF file.", [rope[file]]]
]]
];
};
BringOverInner: PROC [dfFile: ROPE, date: DFUtilities.Date, filter: DFUtilities.Filter]
RETURNS [retrieved: NAT ¬ 0] = {
Note: 'retrieved' is maintained, and therefore meaningful, only if filter.list ~= NIL.
requested: NAT = IF filter.list ~= NIL THEN filter.list.nEntries ELSE NAT.LAST;
Note: 'iAllocatedRetrievedFiles' is TRUE iff this instance of 'BringOverInner' is the outermost one for which filter.list ~= NIL. Thus, when recursion below this instance is complete, all of filter.list should have been retrieved.
iAllocatedRetrievedFiles: BOOL ¬ FALSE;
directoryPath: ROPE ¬ NIL;
Localize: PROC [remote: ROPE, date: DFUtilities.Date, action: DFOperations.BringOverAction, directory: ROPE ¬ NIL] RETURNS [localInfo: REF DFInternal.LocalFileInfo, dontProcess: BOOL ¬ FALSE] = {
remoteInfo: REF DFInternal.RemoteFileInfo
¬ NEW[DFInternal.RemoteFileInfo ¬ [remote, date]];
AttachNeeded: PROC RETURNS [attachAction: {alreadyLocal, doAttach, userSaidNo}] = {
attach: BOOL ¬ TRUE;
sameDate: BOOL = (localInfo.date.gmt = remoteInfo.date.gmt);
Assert: remoteInfo.date.gmt ~= BasicTime.nullGMT
Assert: (localInfo.date.format = $explicit and localInfo.date.gmt ~= BasicTime.nullGMT) iff local file exists.
IF localInfo.date.format = $explicit THEN {
remoteIsntGlobal: BOOL = DFInternal.LocalFile[remoteInfo.name, client];
localAttachedTo: ROPE = localInfo.attachedTo;
localLen: INT ¬ localAttachedTo.Length[];
remoteName: ROPE = remoteInfo.name;
pos: INT ¬ Rope.Run[localAttachedTo, 0, remoteName, 0, FALSE];
SELECT TRUE FROM
(NOT action.fetch AND action.check AND action.fetch) OR remoteIsntGlobal =>
IF localLen # 0 THEN GO TO needAttach;
pos < localLen AND Rope.Fetch[localAttachedTo, pos] # '! =>
GO TO needAttach;
pos < Rope.Length[remoteName] AND Rope.Fetch[remoteName, pos] # '! =>
GO TO needAttach;
ENDCASE;
Both a local and a remote file exist and (the local file is attached to a file whose name (without version) matches the remote file's name (without version) or the remote file isn't global and thus can't be attached to and the local file has no attachment). Check to see if they have the correct relationship in time.
SELECT TRUE FROM
sameDate => attach ¬ FALSE;
action.confirmEarlier =>
attach ¬ BasicTime.Period[from: localInfo.date.gmt, to: remoteInfo.date.gmt] > 0;
ENDCASE;
EXITS needAttach => {};
};
IF ~attach THEN RETURN [$alreadyLocal];
IF ~sameDate AND ~DFInternal.YesOrNo[
client: client,
message:
IO.PutFLR["%g {%g} to %g%g?", LIST[
[rope[remoteName]],
[rope[DFUtilities.DateToRope[remoteInfo.date]]],
[rope[localInfo.name]],
[rope[
IF localInfo.date.format = $explicit THEN
IO.PutFR[
" (previously: %g, keep: %d) ",
[rope[DFUtilities.DateToRope[localInfo.date]]],
[cardinal[localInfo.keep]]
]
ELSE NIL
]]
]],
default: TRUE
] THEN RETURN [$userSaidNo];
IF localInfo.date.format = $explicit AND
BasicTime.Period[from: remoteInfo.date.gmt, to: localInfo.date.gmt] > 0 AND
~DFInternal.YesOrNo[
client: client,
message:
IO.PutFLR["Are you sure? (%g {%g} is older than %g {%g} and will overwrite it)", LIST[
[rope[remoteInfo.name]],
[rope[DFUtilities.DateToRope[remoteInfo.date]]],
[rope[localInfo.name]],
[rope[DFUtilities.DateToRope[localInfo.date]]]
]],
default: FALSE,
blunder: TRUE
] THEN RETURN [$userSaidNo];
RETURN [$doAttach];
};
localName: ROPE ¬ NIL;
remoteName: ROPE ¬ NIL;
isLocal: BOOL ¬ FALSE;
SELECT Rope.Length[directory] FROM
0 => {
No directory given, so use the local short name
remote ¬ DFInternal.ShortName[file: remote, keepVersion: FALSE];
};
Rope.Run[remote, 0, directory, 0, FALSE] => {
Try to preserve the local sub-directory information
remote ¬ DFUtilities.RemoveVersionNumber[
Rope.Substr[remote, Rope.Length[directory]]];
};
ENDCASE => {
This should never happen for a decent god-fearing DF file
DFInternal.DoAbort[client.log, IO.PutFR1["Bogus file name (%g).", [rope[remote]] ]];
};
localInfo ¬ NEW[DFInternal.LocalFileInfo ¬ [name: remote]];
DFInternal.GetFileInfo[info: localInfo, notFoundOK: TRUE, client: client
! FS.Error => GO TO quit];
localName ¬ localInfo.name ¬ Rope.Flatten[
DFUtilities.RemoveVersionNumber[localInfo.name]];
IF action.check OR date.format # $explicit THEN {
DFInternal.GetFileInfo[info: remoteInfo, client: client ! FS.Error => GO TO quit];
};
remoteName ¬ remoteInfo.name;
SELECT AttachNeeded[] FROM
$alreadyLocal => {
isLocal ¬ TRUE;
};
$doAttach => {
DFInternal.SimpleInteraction[
client,
NEW[DFOperations.FileInteraction ¬ [
remoteFile: remoteName,
localFile: localName,
dateFormat:
SELECT remoteInfo.date.format FROM
$explicit => $explicit,
$greaterThan => $greaterThan,
ENDCASE => $notEqual,
date: remoteInfo.date.gmt,
action: IF action.enter OR action.fetch THEN $fetch ELSE $check
]]
];
IF action.enter OR action.fetch THEN {
IF NOT DFInternal.LocalFile[localName, client] THEN {
DFInternal.DoAbort[client.log,
IO.PutFR["Attempting to store %g on a file server using Bringover while processing %g.", [rope[localName]], [rope[dfFile]] ]];
};
[] ¬ FS.Copy[
to: localName,
from: remoteName, wantedCreatedTime: remoteInfo.date.gmt,
setKeep: FALSE,
keep: IF DFUtilities.ClassifyFileExtension[localName] = $source THEN 2 ELSE 1,
remoteCheck: action.check, attach: action.enter
! FS.Error => {DFInternal.ReportFSError[error, localInfo, client]; GO TO quit}
];
};
filesActedUpon ¬ filesActedUpon.SUCC;
};
$userSaidNo => {
IF log # NIL THEN
IO.PutF1[log,
"%g NOT updated: userSaidNo. (Probably date mixup)\n", [rope[localName]]];
dontProcess ¬ TRUE;
warnings ¬ warnings.SUCC;
};
ENDCASE;
IF ~ dontProcess AND action.fetch AND action.enter THEN {
We need to fetch the file, but we have only assured attachment so far.
ENABLE FS.Error => {DFInternal.ReportFSError[error, remoteInfo, client]; GO TO quit};
FS.Close[FS.Open[localName]];
};
EXITS
quit => {errors ¬ errors.SUCC; dontProcess ¬ TRUE};
};
DoOneItem: DFUtilities.ProcessItemProc = {
DFInternal.CheckAbort[client];
IF retrievedFiles ~= NIL AND retrievedFiles.nEntries = retrievedFiles.length THEN
RETURN[TRUE];
WITH item SELECT FROM
directory: REF DFUtilities.DirectoryItem => {
Get the directory in angle-bracket format
directoryPath ¬ FS.ExpandName[name: directory.path1].fullFName;
};
file: REF DFUtilities.FileItem => {
remoteName: ROPE = FS.ExpandName[file.name, directoryPath].fullFName;
localInfo: REF DFInternal.LocalFileInfo
= Localize[remoteName, file.date, action, directoryPath].localInfo;
IF retrievedFiles ~= NIL THEN {
Some enclosing call of BringOverInner presented a filter with an explicit list, so the number of files to be retrieved at that point was known then ('retrievedFiles.u.length'). We maintain the names of the files actually retrieved to enable intelligent error reporting at the end if not all of the files in the original filter.list are found.
short: ROPE = DFInternal.ShortName[localInfo.name];
retN: NAT = retrievedFiles.nEntries;
IF InRetrievedList[short] THEN GO TO notNew;
retrievedFiles[retN] ¬ [name: short];
retrievedFiles.nEntries ¬ retN+1;
DFUtilities.SortUsingList[retrievedFiles, TRUE];
};
IF filter.list ~= NIL AND (retrieved ¬ retrieved.SUCC) = filter.list.nEntries THEN
RETURN[TRUE];
EXITS notNew => {};
};
imports: REF DFUtilities.ImportsItem => {
newFilter: DFUtilities.Filter ¬ [
comments: filter.comments, -- comments processing is unaffected by imports
filterA: filter.filterA, -- source/derived distinction is unaffected by imports
filterB: IF imports.form = $exports THEN $public ELSE all,
filterC: $all, -- if the top level passes imports, they can come from anywhere
list: IF imports.form = $list THEN imports.list ELSE filter.list
];
SELECT TRUE FROM
newFilter.list = NIL =>
In this case, both filter.list and imports.list are NIL. There are no optimizations available to permit early termination of BringOver, so we need not maintain 'retrieved'.
[] ¬ BringOverInner[imports.path1, imports.date, newFilter];
newFilter.list.nEntries = 0 =>
We should never get here with filter.list.nEntries = 0 (see next case, below), so it must be the case that imports.list.nEntries = 0. That is, the imports item contained an explicit list, but it had an empty intersection with our filter.list. Accordingly, there is no point in recurring.
NULL;
ENDCASE => {
We are passing a non-NIL, non-empty filter.list to the next lower level. It will therefore return a meaningful count of the number of files it retrieves.
retrievedBelow: NAT ¬ 0;
IF retrievedFiles # NIL AND retrievedFiles.nEntries # 0 THEN {
If all of the files in the new filter have already been fetched then we have no need to do the inner bringover. So compute the difference.
newFilter.list
¬ DFUtilities.DifferenceOfUsingLists[newFilter.list, retrievedFiles];
IF newFilter.list = NIL OR newFilter.list.nEntries = 0 THEN GO TO noNeed;
};
retrievedBelow ¬ BringOverInner[imports.path1, imports.date, newFilter];
IF filter.list ~= NIL AND retrievedBelow # 0 THEN {
The following statement and a similar one in the 'file' case, above, are the only ones that alter 'retrieved', and are executed iff filter.list ~= NIL. Note that 'requested' is initialized to a meaningful value under the same precondition. The test succeeds iff we have retrieved everything that was requested by our invoker's filter.list.
retrieved ¬ retrieved + retrievedBelow;
IF requested = retrieved THEN RETURN [TRUE];
IF imports.form = $list THEN {
In this case, we were invoked with an explicit filter.list and the imports item had one as well. DFUtilities.ParseFromStream intersected the two and handed it to us as 'imports.list', which we passed down as newFilter.list in the nested call above. As a result of that call, newFilter.list became sorted, and filter.list was sorted by the recursion above us (i.e., by the level that invoked this instance of BringOverInner). Therefore, the precondition (sorted input) for DFUtilities.DifferenceOfUsingLists is met.
filter.list ¬ DFUtilities.DifferenceOfUsingLists[filter.list, newFilter.list];
If filter.list is now empty, we should abandon this level of BringOver. In the absence of errors, however, the test above (requested = retrieved) will have been TRUE and control wouldn't get here. However, if some file in newFilter.list was not found (e.g., the Imports item mentioned a file that didn't appear in the lower-level DF, or some other part of the filter prevented a match), the test above will fail (`requested' will still exceed `retrieved'), yet filter.list will be empty.
RETURN[filter.list.nEntries = 0]
};
};
EXITS noNeed => {};
};
};
include: REF DFUtilities.IncludeItem =>
retrieved ¬ BringOverInner[include.path1, include.date, filter] + retrieved;
ENDCASE;
};
dfLocalInfo: REF DFInternal.LocalFileInfo;
dontProcess: BOOL;
localAction: DFOperations.BringOverAction ¬ action;
localAction.check ¬ localAction.enter ¬ TRUE;
RRA: There is no need to force a fetch, since we are about to open this file anyway.
[dfLocalInfo, dontProcess] ¬ Localize[dfFile, date, localAction];
IF ~dontProcess THEN {
dfStream: STREAM = FS.StreamOpen[dfLocalInfo.name
! FS.Error => {DFInternal.ReportFSError[error, dfLocalInfo, client]; GO TO skip};
];
DFInternal.SimpleInteraction[
client,
NEW[DFOperations.DFInfoInteraction ¬ [action: $start, dfFile: dfFile]]
];
IF filter.list ~= NIL AND retrievedFiles = NIL THEN {
retrievedFiles ¬ NEW[DFUtilities.UsingList[filter.list.nEntries]];
retrievedFiles.nEntries ¬ 0;
iAllocatedRetrievedFiles ¬ TRUE;
};
DFUtilities.ParseFromStream[dfStream, DoOneItem, filter !
DFUtilities.SyntaxError -- [reason: ROPE]-- => {
errors ¬ errors.SUCC;
DFInternal.SimpleInteraction[
client,
NEW[DFOperations.InfoInteraction ¬ [
class: $error,
message: IO.PutFR[
"Syntax error in '%g'[%d]: %g\NProcessing of this DF file aborted.",
[rope[dfLocalInfo.name]], [cardinal[dfStream.GetIndex[]]], [rope[reason]]
]
]]
];
CONTINUE
};
DFInternal.AbortDF => dfStream.Close[];
];
dfStream.Close[];
IF iAllocatedRetrievedFiles THEN {
IF retrievedFiles.nEntries ~= filter.list.nEntries THEN {
There were some files specified in the explicit filter list that weren't retrieved.
diff: REF DFUtilities.UsingList;
diff ¬ DFUtilities.DifferenceOfUsingLists[filter.list, retrievedFiles];
FOR i: NAT IN [0..diff.nEntries) DO
IF filter.filterA = $all OR DFUtilities.ClassifyFileExtension[diff.u[i].name] = filter.filterA THEN NotFoundWarning[diff.u[i].name];
ENDLOOP;
};
retrievedFiles ¬ NIL;
};
DFInternal.SimpleInteraction[
client,
NEW[DFOperations.DFInfoInteraction ¬ [action: $end, dfFile: dfFile]]
];
EXITS
skip => errors ¬ errors.SUCC;
};
};
The peculiar date specification has the effect of treating a version number on 'dfFile' as truth. See comment in DFInternal.GetFileInfo.
[] ¬ BringOverInner[dfFile, [format: $explicit], ConvertFilter[filter] !
ABORTED => {
DFInternal.SimpleInteraction[client, NEW[DFOperations.AbortInteraction ¬ [TRUE]]];
Let this one propagate.
};
DFInternal.AbortDF => {
errors ¬ errors.SUCC;
DFInternal.SimpleInteraction[client, NEW[DFOperations.DFInfoInteraction ¬ [action: $abort, dfFile: dfFile]]];
CONTINUE
};
];
};
END.
Bob Hagmann January 9, 1986 4:52:59 pm PST
Inhibit BringOver from writing on file servers
changes to: Localize (local of BringOverInner, local of BringOver)
Mike Spreitzer January 8, 1987 6:05:38 pm PST
Split BringOverAction fetch into checkAndEnterAndFetch and checkAndFetch.
Made it correctly compute the filter.filterB for a nested BringOver.
Made BringOver with action=check refrain from doing the copy.
Eliminated early-out of AttachNeeded when remote file "is local".
changes to: AttachNeeded (local of Localize, local of BringOverInner, local of BringOver), Localize (local of BringOverInner, local of BringOver), imports (local of DoOneItem, local of BringOverInner, local of BringOver), BringOverInner (local of BringOver), DIRECTORY
Russ Atkinson (RRA) January 14, 1987 9:06:44 pm PST
Changed to accomodate new definition of BringOverAction as a record of flags. Also changed the DF file fetch (via call to Localize) to avoid an extra open.