8-Mar-90 Mike Spreitzer DISCUSSION: FamousPaths interface proposal
Date: Thu, 08 Mar 90 07:03:25 PST Sender: Mike Spreitzer:PARC:Xerox Subject: DISCUSSION: FamousPaths interface proposal To: PCedarImplementors^:PARC:Xerox Cc: Mike Spreitzer:PARC:Xerox
If we're going to have the current working directory during start code execution of a module be uncertain, there are other files besides .tip and .icons files that need to be looked up. The TerminalEmulator is a concrete example of a package that needs to read data files other than .tip and .icons at start time.
The mechanism for looking up "famous files" should be exported through a generic public interface, not buried inside the TIP and Icons modules. So here is a proposal. Any comments?
[There is an unfortunate collision between the use of the word "path" for both an ordered list of directories and for a single hierarchical file name (eg, PFS.PATH). If you have a better idea for what to do about it than the one used below, speak up.]
[Another question you might consider is whether this interface should strive to support both PFS and FS clients. A related question is whether PFS clients should be offered the ROPE variants. I think we're quite a ways from using only PFS.PATHs for file names; for example, note that the DF interfaces still use ROPEs. Should this interface try to force people into the brave new world, or be easy to use now?]
FamousPaths: CEDAR DEFINITIONS = BEGIN
GetFamousRopePath: PROC RETURNS [LIST OF Rope.ROPE]; --Returns the famous search path as a list of ROPEs that can be used as FS directories.
GetFamousPathPath: PROC RETURNS [LIST OF PFS.PATH]; --Returns the famous search path as a list of absolute directory PFS.PATHS.
LookupFamousRope: PROC [relative: Rope.ROPE] RETURNS [absolute: Rope.ROPE]; --Searches along the famous search path for an existing file matching the given relative FS name.
LookupFamousPath: PROC [relative: PFS.PATH] RETURNS [absolute: PFS.PATH]; --Searches along the famous search path for an existing file matching the given relative PFS name. --= {RETURN [PFS.FileSearch[relative, GetFamousPathPath[]]]};
END.
8-Mar-90 Mike Spreitzer DISCUSSION: Do we really want famous files?
Date: Thu, 08 Mar 90 08:23:29 PST Sender: Mike Spreitzer:PARC:Xerox Subject: DISCUSSION: Do we really want famous files? To: PCedarImplementors^:PARC:Xerox Cc: Spreitzer:PARC:Xerox
Have we opened a bigger can of worms than we want? We may find the need to follow the famous search path slowly propogate to all our packages that use files.
Every procedure that generates relative file names will have to use the famous path (or some other well-defined path) for lookup, because that procedure could be called from start code. This includes procedures that generate relative file names from other non-filename data.
TIP tables and icons are well-known examples of these.
A less well known example is the way the TerminalEmulator finds the emulation code for a given terminal: at start time it enumerates files matching "Em<prog:*>.terms", and reads each such file for a list of the terminals emulated by the program that "Run <prog>" installs.
Imagine a compiler interface that takes the name of a module (not a file) to compile. This interface could not be used (recursively!) from start code, unless use of the famous search path is built into the implementation. Other fixes would be to expand the interface to optionally take the search path to use, or to expand the interface to optionally take the file name to use [this option could get very messy if you're general about allowing directory delimiters in the relative input and/or output file names]. And every client of the compiler that generates modules names from parameters would also have to have its interface expanded to optionally take a search path or file name. This is an example of the creeping mess that the uncertainty of working directory creates.
On the other hand, this may not turn out to be much of a problem. Some programs use paths anyway, because they can't rely on their current working directory for even stronger reasons; the lookup of fonts and Tioga styles are examples. It's fairly natural for a compiler interface to simply take a file name. The TerminalEmulator can be made to look down the famous search path without too much trouble.
Still, there is an undeniable increase in complexity in several programs. For example, the TerminalEmulator has to enumerate files matching its pattern in each directory on the famous search path and implement a precedence relation amongst the directories.
The compiler is complicated even if it takes file name arguments. Consider the Cedar compiler, which in its full generality (as exposed by the ComplexCompile command) takes: a source file name, an output file name, and a binding between referenced module (i.e. interface) names and input .bcd file names (not all the referenced modules need to be given bindings --- default rules are used to generate input .bcd file names for modules whose file names are not given). Those file names could (in principle if not in fact) be relative names, to be interpreted relative to the current working directory. For example, one way to manage machine dependent sources would involve MakeDo issuing ComplexCompile commands of the form:
Now consider invoking one of these compilations through a Cedar interface from start code. Three relative files names must be specified, and one of the famous directories must be chosen to interpret them relative too --- in fact, the SAME dirctory must be used for all the input and output files. And the output file need not even exist before hand, so we can't use a standard search mechanism to determine its name. If the choosing of the file names is left entirely up to the client, the client no longer has the option of not specifying bindings for some of the referenced modules (because that would require the compiler to generate a file name from a module name, which we know can't work [if you try to use the directory of one of the specified file names, could be looking too low --- looking for Rope.mob in /mumble/Mach/ when it's actually in /mumble/]). A better way would be to give the compiler some relative names and a search path, and let the compiler decide (probably according to existence of one or more input files [this might convert one kind of error into another]) which directory to interpret all the relative names relative to. Thus the compiler interface is at least as complex as it would be if it simply took a module name and the uncertainty of working directory forced it to also take a search path; taking file names instead of module names does NOT simplify the compiler interface in the presence of uncertainty about working directory.
Another way to cope with the uncertainty of working directory during start code is to eliminate it by explicitly using the release directory. For examples:
--Proc[arg, {body}] is a notation we should have for passing procedure arguments
Still another way to eliminate the uncertainty would be to change the way packaged worlds work so that each STARTed module is STARTed in its release directory. I don't know how packaged worlds work --- how difficult would this be?
The question I raise is this: will coping with the uncertainty of working directory during the execution of start code be more or less painful than eliminating the uncertainty (either by chaning the way packaged worlds work or by explicitly using release directories)? What do you think?
Mike
8-Mar-90 Mike Spreitzer Re: DISCUSSION: FamousPaths interface proposal
Date: Thu, 08 Mar 90 08:34:15 PST In-addition-to: "My message of Thu, 08 Mar 90 07:03:25 PST" Sender: Mike Spreitzer:PARC:Xerox Subject: Re: DISCUSSION: FamousPaths interface proposal To: PCedarImplementors^:PARC:Xerox Cc: Mike Spreitzer:PARC:Xerox
For reasons mentioned in my Can Of Worms message, and a convenience consideration, I think the FamousPaths interface should be wider, like this:
FamousPaths: CEDAR DEFINITIONS = BEGIN
GetFamousRopePath: PROC RETURNS [LIST OF Rope.ROPE]; --Returns the famous search path as a list of ROPEs that can be used as FS directories.
GetFamousPathPath: PROC RETURNS [LIST OF PFS.PATH]; --Returns the famous search path as a list of absolute directory PFS.PATHS.
LookupFamousRope: PROC [relative: Rope.ROPE] RETURNS [full, directory: Rope.ROPE]; --Searches along the famous search path for an existing file matching the given relative FS name. directory.Cat[relative].Equal[s2: full, case: FALSE] (except that relative may have less specific version information).
LookupFamousPath: PROC [relative: PFS.PATH] RETURNS [full, directory: PFS.PATH]; --Searches along the famous search path for an existing file matching the given relative PFS name. directory.Cat[relative].Equal[n2: full, case: FALSE] (except that relative may have less specific version information). --= {RETURN [PFS.FileSearch[relative, GetFamousPathPath[]]]};
8-Mar-90 Mark Weiser DISCUSSION: FamousePaths interface proposal
Date: 08 Mar 90 10:52:58 PST Sender: Mark Weiser:PARC:Xerox Subject: DISCUSSION: FamousePaths interface proposal To: PCedarImplementors:PARC:Xerox Cc: Reply-To: Mark Weiser:PARC:Xerox
Mike proposed FamousPaths interface lacks anyway of changing the paths. How about adding:
SetFamousRopePath: PROC [LIST OF Rope.ROPE]; - Set the famous search path to the list of ROPE's.
SetFamousRopePath: PROC [LIST OF PFS.PATH]; - Set the famous search path to the list of Paths
-mark
8-Mar-90 Christian P Jacobi Re: DISCUSSION: FamousPaths interface proposal
Date: Thu, 08 Mar 90 12:00:20 PST In-reply-to: "Mike Spreitzer:PARC:Xerox's message of Thu, 08 Mar 90 07:03:25 PST" Sender: Christian P Jacobi:PARC:Xerox Subject: Re: DISCUSSION: FamousPaths interface proposal To: Mike Spreitzer:PARC:Xerox Cc: PCedarImplementors^:PARC:Xerox
This has been copied to a file server so you could get the formating back copy -c /palain-nfs/palain/jacobi/path.tioga ← path.tioga
I think paths are dependent on purpose. Also, paths might be different in package or dynamically loadd worlds.
Example purposes:
For documentation local-working-directory >> /pcedar2.0/documentation
For p-dfs accessing local-working-directory >> /pcedar2.0/top
For d-dfs accessing local-working-directory >> /Cedar7.0/top >> /CedarChest7.0/top >> /DATools7.0/top
For d-dfs bringover /Cedar7.0/top >> /CedarChest7.0/top >> /DATools7.0/top
For d-dfs smodel local-working-directory
For importing a cell into a ChipNDale design (actually implemented this way) local-working-directory >> directory-of-design >> directory-of-technology >> directory-of-ChipNDale
For tip tables in dynamicly loaded world local-working-directory >> directory-where-the-calling-procedures-module-was-loaded-from
For tip tables in packaged world local-working-directory >> directory-where-the-packager-did-put-tip-tables
For Imager fonts global-font-place
For .command files local-working-directory >> global-place-for-.command-files
I also think the mechanism expands to more then just file system paths (but I don't really want to throw this complication in)
For certain X window resources application-dependant-setup-file >> property-of-root-window-in-X-server
Implementation ideas
The tip example is particularly interesting since its path is also depending on where a module was loaded from (Sub-purpose?)
In FooTool paths ← FamousPaths.GetPaths[purpose: $TIP, package: "Foo"]; or directoryPath ← FamousPaths.LookUpPath[purpose: $TIP, package: "Foo", file: "FooTool.tip"];
In TipImpl FamousPaths.RegisterPaths[purpose: $TIP, check: LIST[$wdir, $packagesource]]
In FamousPaths RegisterPaths: PROC [purpose: ATOM, check: LIST OF REF ANY]; each check element denotes a method to find a directoryPath $wdir use current working directory $packagesource use source of package ROPE convert it to PFS.PATH PFS.PATH: us it Atoms have late binding: resolved at GetPaths or LookUpPath time
FindSourceProcType: TYPE = PROC [package: ROPE, purpose: ATOM] RETURNS [from: PFS.PATH]; most implementations might ignore purpose
In PackagingDescriptorImpl version for packaged worlds MyFindSourceProc: FamousPaths.FindSourceProcType = {RETURN ["famousplace"]}; FamousPaths.RegisterFindSourceProc[MyFindSourceProc]
In PackagingDescriptorImpl version for dynamic worlds MyFindSourceProc: FamousPaths.FindSourceProcType = { find out what the require line did say and return corresponding place }; FamousPaths.RegisterFindSourceProc[MyFindSourceProc];
Christian
9-Mar-90 Mike Spreitzer Re: DISCUSSION: FamousPaths interface proposal
Date: Fri, 09 Mar 90 07:39:30 PST In-reply-to: "Christian P Jacobi:PARC:Xerox's message of Thu, 08 Mar 90 12:00:20 PST" Sender: Mike Spreitzer:PARC:Xerox Subject: Re: DISCUSSION: FamousPaths interface proposal To: Christian P Jacobi:PARC:Xerox Cc: Mike Spreitzer:PARC:Xerox, PCedarImplementors^:PARC:Xerox
I agree that paths are dependent on purpose. I think it's even worse: paths depend on instances; that is, even in two instances of doing "the same kind of thing", you might want to use different paths. I haven't yet thought of a good scoping mechanism that lets users bind the desired path to the desired use with a reasonable amount of convenience and correctness.
Since it's inception, Cedar has accomplished approximately what paths do with a different mechanism: copying the desired files into the working directory. This "style" permeates Cedar --- if you look closely, you can see that a number of low-level user and programmer interfaces assume it. For this reason, I think moving Cedar toward using paths is a BIG DEAL. That doesn't necessarily mean it's a bad idea. But I think it deserves a thorough discussion, which I haven't yet heard. That's why I'm nervous about FamousFiles --- introducing paths here is pulling on a thread that may lead to a lot of unravelling.
The FamousFiles search path is for one distinct purpose: finding data files during the execution of start code. [Note that if we ever have a data file, read during start code, that is also read during some other time that puts a conflicting constraint on where it lives, we will have trouble.] I think that designing a path mechanism for FamousFiles is easier than designing a general one --- the scope of the discussion is much more limited. I think *this* "Discussion" should address itself only to designing a path mechanism for locating famouse files. If people are interesting in discussing more general use of paths, I think that's a fine idea, but should be held as a distinct "Discussion".
I think your suggestion to have the FamouseFile path mechanism take the package name as an additional argument is an interesting one. Since TIPUser.InstantiateNewTIPTable and Icons.NewIconFromFile don't take the package name as an argument, the use of FamousPaths would have to be in the clients of these two interfaces, not their implementations. That works, but requires editing all those clients again. Also, I assume that the idea is that the famous path for package foo is . >> /PCedar/Foo/ >> /CedarCommon/Foo/ --- that is, we've eliminated the use of the FamousFiles subdirectory in favor of the package subdirectory. Again, this works, but requires editing all the DFs again. The only benefit I see is that the FamousFiles name space becomes a two-level hierarchy instead of flat. Is it worth all that editing? How long before some other decision changes whatever we do now?
Mike
9-Mar-90 Mike Spreitzer Re: DISCUSSION: FamousePaths interface proposal
Date: Fri, 09 Mar 90 07:48:14 PST In-reply-to: "Your message of 08 Mar 90 10:52:58 PST" Sender: Mike Spreitzer:PARC:Xerox Subject: Re: DISCUSSION: FamousePaths interface proposal To: Mark Weiser:PARC:Xerox Cc: Mike Spreitzer:PARC:Xerox, PCedarImplementors^:PARC:Xerox
Setting the famous path is the responsibility only of the implementation of the FamousPath interface; random clients don't know what the famous path should be. This is not to say that the implementation of the FamousPath interface can't be split among multiple packages that communicate through a less public interface. As an example of the uncertainty about what the famous path should be, note that there is some discussion of having the process of making a packaged world copy all the needed data files somewhere, so as to bind the packaged versions of the code with the versions of the data they expect. This hasn't happened yet --- and you wouldn't want to have to edit and recompile a bunch of random clients of famous files when this does happen. Also, when it happens there'll be a need to change the famous path after the packaged world is started, so that code dynamically loaded into that world sees current data files.
Mike
9-Mar-90 Mike Spreitzer Re: DISCUSSION: FamousPaths interface proposal
Date: Fri, 09 Mar 90 08:16:07 PST In-addition-to: "My message of Thu, 08 Mar 90 07:03:25 PST" Sender: Mike Spreitzer:PARC:Xerox Subject: Re: DISCUSSION: FamousPaths interface proposal To: PCedarImplementors^:PARC:Xerox Cc: Mike Spreitzer:PARC:Xerox
I had some more thoughts on the names to use.
Firstly, there is only one famous path (at a given time in a given world), so the interface should be named FamousPath, not FamousPaths.
Second, thinking of what uses of that interface in client code look like suggest that the main procedures should be called as:
FamousPath.Get[]; and FamousPath.Lookup[something]
although I'm not sure exactly what that something should be: a ROPE, a PFS.PATH, or a PFSNames.Component?
Note that PFSNames should include something like the following:
I think that in the usual case Lookup will be given a name with no directory delimiters. Thus we could say that FamousPath.Lookup takes the same arguments as PFSNames.ConstructComponent, constructs the component, and does the lookup. Alternatively, we could say that FamousPath.Lookup takes a ROPE argument that is passed through PFS.PathFromRope and then looked up --- which is a more expensive, but more general, process. I tend to favor generality when there is a way to get efficiency --- which there is if we leave a variant that takes a Component.
Taking all this into account, I now suggest something like the following (unless we decide to add the package name as an argument). Note that where my comments equate procedures, I mean "behaves like", not "implementation is".
FamousPath: CEDAR DEFINITIONS = BEGIN
Get: PROC RETURNS [LIST OF PFS.PATH]; --Returns the famous search path as a list of absolute directory PFS.PATHS.
GetAsRopes: PROC RETURNS [LIST OF Rope.ROPE]; --Returns the famous search path as a list of ROPEs that can be used as FS directories. Or should we say these are PFS.RopeFromPath of the PFS.PATHs? Will there ever be a difference?
Lookup: PROC [relative: Rope.ROPE] RETURNS [full, directory: PFS.PATH]; --={RETURN PathLookup[PFS.PathFromRope[relative]]}; --Do we like the dissimilarity of argument and result TYPEs?
PathLookup: PROC [relative: PFS.PATH] RETURNS [full, directory: PFS.PATH]; --Searches along the famous search path for an existing file matching the given relative PFS name. directory.Cat[relative].Equal[n2: full, case: FALSE] (except that relative may have less specific version information). --full = PFS.FileSearch[relative, Get[]]
RopeLookup: PROC [relative: Rope.ROPE] RETURNS [full, directory: Rope.ROPE]; --Like Lookup, except that the arguments and results are the FS filename type and have FS filename semantics (is there any difference?)
END.
9-Mar-90 Howard E Sturgis Re: DISCUSSION: FamousPaths interface proposal
Date: Fri, 09 Mar 90 09:53:43 PST Sender: Howard E Sturgis:PARC:Xerox Subject: Re: DISCUSSION: FamousPaths interface proposal To: PCedarImplementors^:PARC:Xerox Cc: Howard E Sturgis:PARC:Xerox Reply-To: Howard E Sturgis:PARC:Xerox
A remark on Mike Spreitzers comment: "Since it's inception, Cedar has accomplished approximately what paths do with a different mechanism: copying the desired files into the working directory."
This effectively results in a name scope which is a collection of the contents of DF files, rather than a collection of contents of file server directories.
Howard
9-Mar-90 Christian P Jacobi Re: DISCUSSION: FamousPaths interface proposal
Date: Fri, 09 Mar 90 12:25:27 PST In-reply-to: "Mike Spreitzer:PARC:Xerox's message of Fri, 09 Mar 90 07:39:30 PST" Sender: Christian P Jacobi:PARC:Xerox Subject: Re: DISCUSSION: FamousPaths interface proposal To: Mike Spreitzer:PARC:Xerox Cc: Christian P Jacobi:PARC:Xerox, PCedarImplementors^:PARC:Xerox
Stored on a server so you can get formating back copy -c /palain-nfs/palain/jacobi/path.tioga ← path.tioga
I still believe in using purpose; I now call it class. You mention a problem with package name causing changes to tip clients. I have extended my proposal so that package name can be left out or not, depending on client. This should allow unmodified interface to TIPUser.InstantiateNewTIPTable and Icons.NewIconFromFile.
This is not yet implemented. Implementation might show further problems. There are features marked "THIS IS NOT REALLY NECESSARY; IT CAN BE IMPLEMENTED AS A CLASS". In a first reading leave them in since they show intended use.
SearchPaths.mesa Copyright Ó 1990 by Xerox Corporation. All rights reserved. Christian Jacobi, March 9, 1990 11:51:06 am PST DIRECTORY PFS USING [PATH], Rope USING [ROPE];
SearchPaths: CEDAR DEFINITIONS = BEGIN Client operations Unknown class atoms results in quitely not finding any paths No guarantee against returning same PATH twice
EnumerateSources: PROC [package: Rope.ROPE, each: Enumerator] RETURNS [quit: BOOL]; Enumerates the places the loader thinks the package came from THIS IS NOT REALLY NECESSARY; IT CAN BE IMPLEMENTED AS A CLASS
Enumerate: PROC [class: ATOM, each: Enumerator, package: Rope.ROPE ← NIL, classData: REF ← NIL] RETURNS [quit: BOOL]; Enumerates the paths the class thinks appropriate
Class implementations Typical classes are $TIP, $icon, $ChipNDaleImport, $Font Other classes are $wdir: just uses current working directory $sourcedir: like EnumerateSources A class might forward enumerations by using EnumerateSources Whether a class requires or enables usage of classData depends on the class. Eg. $TIP or $icon might not have classData, whereas $ChipnDaleImports will allow a CD.Design as classData.
RegisterSimpleClass: PROC [class: ATOM, list: LIST OF REF ANY]; Registers a EnumerateProc which will go through every element of list and if PFS.PATH => use it Rope.ROPE/REF TEXT => convert it to PFS.PATH and use it (early bound) ATOM => recursively use its class EnumerateProc used for: $wdir => uses current workingdirectory $sourcedir => uses EnumerateSources Classes are responsible for avoiding infinite recursion Repeated registration replaces previous one. Loader implementations THIS IS NOT REALLY NECESSARY; IT CAN BE IMPLEMENTED AS A CLASS Typically packagers or loaders can register a way to find package directories
RegisterSourceEnumerations: PROC [enumerateSources: SourceEnumerationProc]; If multiple enumerateSources are registered, they will be called in sequence Above comment changed to whatever loader wants. If loader simply uses classes no such tricks are necessary to handle multiple loaders. [Multiple loaders = package world later loads additional package with dynamic loader]
END.
Remarks: TipImpl is both a class implementor and a client. Since clients of Tip should not be recompiled and therefore can't call Lookup directly nor provide a package rope, TipImpl plays also client. Since it does not know package, package directories are simply not available for the tip class.
Christian
9-Mar-90 Mike Spreitzer Re: DISCUSSION: FamousPaths interface proposal
Date: Fri, 09 Mar 90 13:34:32 PST In-reply-to: "Christian P Jacobi:PARC:Xerox's message of Fri, 09 Mar 90 12:25:27 PST" Sender: Mike Spreitzer:PARC:Xerox Subject: Re: DISCUSSION: FamousPaths interface proposal To: Christian P Jacobi:PARC:Xerox Cc: Mike Spreitzer:PARC:Xerox, PCedarImplementors^:PARC:Xerox
(You didn't change the name of your Tioga file from your previous message, and its contents right now don't match any message from you).
I still agree that the desired path depends on the purpose, whatever you call it. I also still think it sometimes even varies between instances of a purpose.
Most purposes now do not have paths associated with them. I think we should, in this Discussion, be trying to isolate one purpose (namely, locating a data file during the execution of start code) and define an interface that supports *only* that purpose. I agree that it is interesting to design a path mechanism for general purposes, but I think that design effort should be carried out in a separate Discussion. That's because I believe it is a significantly harder problem, and we need a solution right now for the one purpose I'm trying to focus on (the TerminalEmulator *can not* work in both packaged and incrementally loaded worlds without this solution). We don't need solutions right now for other purposes because they're already working (maybe not as well as we'd like, but they're working) without paths. I recommend that we try to get an interface worked out *real soon* to support *only* the famous files path. If another Discussion comes up with a more general path mechanism, I think that would be good and the implementation of the FamousPath interface should then be changed to use the more general path mechanism.
This is not to say I'm entirely confident that the program I've outlined for this Discussion will succeed. As I've said, I have my fears about how this might unravel --- but I think we should try to contain it as much as possible.
Mike
9-Mar-90 Christian P Jacobi Re: DISCUSSION: FamousPaths interface proposal
Date: Fri, 09 Mar 90 16:29:31 PST In-reply-to: "Mike Spreitzer:PARC:Xerox's message of Fri, 09 Mar 90 13:34:32 PST" Sender: Christian P Jacobi:PARC:Xerox Subject: Re: DISCUSSION: FamousPaths interface proposal To: Mike Spreitzer:PARC:Xerox Cc: Christian P Jacobi:PARC:Xerox, PCedarImplementors^:PARC:Xerox
Sorry I didn't store the Tioga file. I have stored it now and put even some untested implementation code into it. It is stored as /palain-nfs/palain/jacobi/path.tioga
I agree that there is use for an instances of a purpose; I think my proposal allows you to do that using "classData".
I don't think you are speaking of a single purpose since I consider tip tables and icons as two different things.
I also don't really think this should be a separate Discussion. I think most of us are concerned about the problem in general. A hack to get the TerminalEmulator working should not be allowed to have such wide system consequences as to influence tip tables and icons.
Christian
10-Mar-90 Michael Plass Re: DISCUSSION: FamousPaths interface proposal
Date: 10 Mar 90 00:28:29 PST In-reply-to: "Mike Spreitzer:PARC:Xerox's message of Fri, 09 Mar 90 13:34:32 PST" Sender: Michael Plass:PARC:Xerox Subject: Re: DISCUSSION: FamousPaths interface proposal To: Mike Spreitzer:PARC:Xerox Cc: Christian P Jacobi:PARC:Xerox, PCedarImplementors^:PARC:Xerox
(N.B. After I composed the following, I re-read Christian's message, and realized that he is trying to address a different set of issues than I had in mind.)
CommanderFileCommandsImpl already has code for maintaining search paths (including the elimination of duplicates). I would recommend massaging this code (and the commands that manipulate it) into something usable for search paths other than the Commands search path. Thus you might say
(note: the current WD ought not to be consulted unless there is a relative pathname in the search rules; this would be a change for the current search rules for commands, which always look in the CWD first, a behavior I consider a bug)
PrintSearchRules with no args would print all known search rules.
There would be a client interface like this:
-------------------------------------------------------------------------------- PATH: TYPE = PFS.PATH; DefineSearchRules: PROC [for: ATOM, initial: LIST OF PATH ← NIL]; GetSearchRules: PROC [for: ATOM] RETURNS [LIST OF PATH]; --------------------------------------------------------------------------------
-------------------------------------------------------------------------------- UndefinedSearchRules: ERROR; ChangeSearchRules: PROC [for: ATOM, from, to: LIST OF PATH] RETURNS [ok: BOOL]; -- A conditional store; try again if you lose -- MergePaths: PROC [a, b: LIST OF PATH] RETURNS [LIST OF PATH]; -- Removes duplicates, preserving order. -- GetSearchRuleNames: PROC RETURNS [LIST OF ATOM]; NotifyOnChange: PROC [for: ATOM, notify: Notify ← NIL]; Notify: TYPE = REF NotifyRep; NotifyRep: TYPE = RECORD [proc: PROC [Notify, ATOM], data: REF]; --------------------------------------------------------------------------------
The first section is enough for most clients; the second section provides some degree of completeness. (The purpose of the Define mechanism is merely to provide checking against user errors.) (I'm not really sure the NotifyOnChange mechanism is a good idea, but it would be handy for things like the revalidation of styles.)
I think we shouldn't worry too much about providing a ROPE variant; calls to PFS.RopeFromPath or PFS.DoInWDir should suffice.
This function logically belongs in or near PFS, I think. It would seem unfortunate to introduce another whole package for this.
12-Mar-90 Mike Spreitzer Re: DISCUSSION: FamousPaths interface proposal
Date: Mon, 12 Mar 90 07:38:05 PST In-reply-to: "Christian P Jacobi:PARC:Xerox's message of Fri, 09 Mar 90 16:29:31 PST" Sender: Mike Spreitzer:PARC:Xerox Subject: Re: DISCUSSION: FamousPaths interface proposal To: Christian P Jacobi:PARC:Xerox Cc: Mike Spreitzer:PARC:Xerox, PCedarImplementors^:PARC:Xerox
I agree also that tip tables and icons are two different things. However, I think the idea behind creation of FamousFiles is that a bunch of different kinds of files could all be found via one single search path, called the famous files search path. At the time of introduction of FamousFiles, that path was expected to have only one element, /PCedar2.0/FamousFiles/ --- or was it two elements, ./ and /PCedar2.0/FamousFiles/? Doesn't matter. The idea is that a bunch of different kinds of files get looked up the same way.
DCedar `works' with only the few, specialized search path mechanisms it has (one for commands, another for Tioga style stuff, perhaps another for fonts --- and that's all I know about). Why are we talking about adding more mechanism to PCedar? I see two reasons:
1. Because of the way packaged worlds work and the way the Commander works (and is used), start code of modules must cope with the working directory sometimes being irrelevent and sometimes being the place to look. This is different from DCedar.
2. Adding a general search path mechanism might be a significant improvement to Cedar.
The problems introduced by (1) have not been completely solved. They have been solved for .tip and .icons files, but not for any other kind of data files --- and there are indeed other kinds of data files. Until we solve these problems for other kinds of data files, PCedar is broken: some packages are forced to have bugs. I'm trying to reach a consensus on how to solve the problems of (1), and I'm trying to reach it soon so that we're not forced to live with buggy packages for long.
The reason I suggested discussing the issues of (2) separately is that I think it is a larger topic, and will thus take longer to discuss. Improving Cedar is nice, but fixing it is more urgent. I also doubted whether there was much interest, since I've never heard much discussion of adding a general path mechanism, even though its been an obvious possibility for several years. Is there anyone besides Christian and me who thinks this is worth discussing?
I'm not sure what the resolution to (1) will be. I even admit that we might end up deciding that (1) should be solved as part of working out (2). I'm not even sure I like the idea of FamousFiles! I take the fact that other people introduced it, and don't complain about it, to indicate that they're happy with searching for all data files read during start code in the same places. If that's a bad idea, the best way to show it is to pursue that direction and see why it doesn't work (does anyone already think that it doesn't work?). If that's a good idea, pursuing that direction will lead us to a working PCedar sooner (I think) than working out (2).
Mike
22-Mar-90 Mike Spreitzer SUMMARY 1: FamousPath discussion
Date: Thu, 22 Mar 90 09:19:57 PST Sender: Mike Spreitzer:PARC:Xerox Subject: SUMMARY 1: FamousPath discussion To: PCedarUsers^:PARC:Xerox Reply-To: PCedarImplementors^:PARC:Xerox
Executive summary:
We've had a long argument about whether and how to publicize the famous files search path in PCedar, and whether or not that should wait for and be one of the problems solved by a general purpose search path mechanism. The loudest voice at yesterday's PFUDGE meeting argued for hard-coding the famous files search path in the packages that need it and having a separate discussion about introducing a general purpose search path mechanism, and by that time nobody wanted to disagree. I think we haven't got a good argument for hard-coding instead of adopting a FamousPath interface, but I get the impression that interest in this question is extremely low.
Files read during the execution of PCedar start code live in either /PCedar2.0/FamousFiles/ or /CedarCommon2.0/FamousFiles/, as appropriate. Procedures wishing to read such files search for a relative name in the following directories, in order: ./, /PCedar2.0/FamousFiles/, and /CedarCommon2.0/FamousFiles/. [Note that although "./" is used in this discussion to denote the current working directory, this is not a promise that your favorite file system interfaces will recognize it as such.] Clients of TIPUser and Icons need not do the search themselves because those packages do the search if given a relative file name; clients of all other packages that take file names are responsible for doing the search themselves. Here is how such a client of other packages (in this case, the MazeWar client of ViewerTools) might look:
famousPath: LIST OF PFS.PATH = CONS[PFS.GetWDir[], LIST[ PFS.PathFromRope["/PCedar2.0/FamousFiles/"], PFS.PathFromRope["/CedarCommon2.0/FamousFiles/"]]];
The reason we have a problem that we didn't have in DCedar is that PCedar follows different rules concerning the setting of working directory during the execution of start code and how users and maintainers bind to packages.
In DCedar, users BringOver CedarChest packages into directories, and the use of .load and .install files arranges that the working directory current during start code is the directory into which the package was BroughtOver. In DCedar, for Cedar (as opposed to CedarChest) packages, the data files read during start code are put in fixed places in the file name space. CedarChest maintainers make and test new versions of a package in a development directory that is current when starting the new version being tested. Thus absolute names or relative names resolved against the current working directory always suffice to find files during start code. [Maintainers of Cedar packages that read files during start code have a harder problem, and either live with it or provide users, via the L boot switch, the opportunity to specify a different file name.]
In PCedar, users always get the latest version of every package they load dynamically, and the use of .command and .require files arranges that the working directory during start code is the release directory of the package being dynamically loaded. PCedar also has packaged worlds, created by putting together packages with UNIX's ld program; during the execution of start code for these worlds the working directory is random (I think it's whatever the UNIX working directory was when the world was invoked). Maintainers of PCedar packages still work in a development directory that's different from the release directory. Thus the correct directory in which to find a file during start code may be either the current working directory or the release directory, which may in turn be under PCedar2.0 or CedarCommon2.0.
The problem is how to arrange packages and their clients so that files are found in the appropriate directory during start code (and, of course, the rest of the time too!).
The chosen solution mostly works, becuase it searches all the directories in which the files may be found. /PCedar2.0/FamousFiles/ comes before /CedarCommon2.0/FamousFiles/ in the search path because it's expected that the transition from source-shared to not will be more common than the reverse, and when that transition happens this order will prevent stale files in CedarCommon from shadowing possibly newer files in PCedar. The current working directory comes before the release directories so that maintainers working in development directories will see their possibly new files in preference to the old, released versions.
Critique of the proposed current rules, and other discussion:
One alternative not chosen is to always look only in the release directory. This has the disadvantage that when the release directory changes (eg, a packages changes between source-shared and not), extra editing, and perhaps extra compiling and binding, are required. This also has the disadvantage that maintainers, when testing, must remember to alter the prefix maps for all files read during start code to divert the lookup from the release directory to the development directory --- and to change them all back again when done testing; this is messy. [Hmm, maybe it wouldn't be too wrong to just remap the prefix(s) for the release directory(s), rather than for each data file read during start code; this would be easier, and more easily automated, but would fool a few things, like OpenR.] This approach has the advantage over the chosen one that exactly one directory is searched in, so lack of files in that directory leads to an appropriate error, rather than strange behavior (from finding the wrong file in a different directory).
The fact that TIPUser and Icons *always* use the famous search path is (IMHO) slightly wrong: they can be called from places other than start code. Of course, we can take the attitude that this searching is a standard part of the TIP and Icon behavior that should be expected at all times; I think this is a little weird when you consider that the problem extends beyond TIP and Icons and is solved differently for other kinds of files read during start code.
By having all other clients hard-code the famous search path in their start code, we duplicate knowledge, fix it in places that maybe shouldn't know it, and preclude architectures where it changes between packaged vs. dynamically loaded modules. So far these won't be serious problems, because there won't be a lot of duplication (there are only a few packages that read non-TIP-or-Icon files during their start code) and we don't currently have an architecture wherein the famous path changes.
One solution to the previous criticism is to have these other packages get the famous path through a standard interface, and have only the implementation of that interface know what the famous path is (and vary it as necessary). This discussion started with me trying to design such an interface. However, it was soon broadened with the suggestion that instead of designing an interface to solve this specific problem, we should design a general-purpose search path mechanism for Cedar.
It was observed that in general one might want search paths for several different purposes in Cedar (in addition to the two or three [one for commands, one for Tioga styles, and maybe one for fonts] already in DCedar). It was further agreed that the desired path may even vary amongst instances of the same purpose. Christian Jacobi suggested a general-purpose path interface wherein the path is determined by (up to) three things: the purpose name, a package name, and a REF ANY; he gave as a new example the purpose of ChipnDaleImports, where the REF ANY is a CD.Design that influences the path. At yesterday's PFUDGE meeting the idea of using the view mechanism to put paths below PFS was discussed (and generally found good, although not the whole solution --- how is the binding of paths to path names specified and scoped?).
I objected that the dependence on working directories and *not* on paths is a fundamental feature of the Cedar "style", and that changing that will require a lot of discussion and quite possibly changes to low-level interfaces. For these reasons I suggested that the general path mechanism be designed as part of a separate discussion, whose results the FamousPath implementation could use upon arrival --- without requiring editing or recompilation of the FamousPath clients. There was some inconclusive argumentation about whether the details fixed by a FamousPath interface designed now would have to change upon the arrival of a general purpose path mechanism. One such detail is whether the package name can have an influence on the famous path. I think that if we say it can, there's a good probability that a FamousPath interface designed now would *not* have to change upon the arrival of a general purpose path mechanism. I think a lack of agreement on this point is one of the major reasons the current proposal does not use a FamousPath interface designed now.
17-Apr-90 Mike Spreitzer FamousFiles
Date: Tue, 17 Apr 90 07:30:37 PDT
From: Mike Spreitzer:PARC:Xerox
Subject: FamousFiles
To: Pier, Plass, Willie-Sue, CHauser, Foote:OSBU North:Xerox
Cc: Spreitzer
I'm trying to write a version of the FamousFiles section of DFSuitesDoc that actually says what we think is true. I also did a bit of design for support of source-chared packages; this is new, and confined to the last paragraph; I'd like your review, since it puts some requirements on low-level things.
FamousFiles
Sometimes a package needs to read a data file that is maintained along with that package. Common examples are TIP tables and icon files. Usually these package data files are read during the start code of the package, but not always. The question arises of how the package should refer to these files, since absolute file names are frowned upon (and not always correct).
The DCedarChest solution starts with making these data files public in the package's DF. Thus, a BringOver of the package puts these data files in the same directory as the program binaries. Because DCedarChest conventions ensure that the directory containing the binaries is current when they are run, a data file can be referred to with a short name (actually any relative name works) --- the same short (relative) name that is used in the package's DF. If the data file is to be read not during the starting of the package but later, the package must remember what directory was current at start time, and then look up the data file by a short (relative) name in that directory.
The solution for DCedar (meaning the stuff in [Cedar7.0], as opposed to all D-machine Cedar stuff) is somewhat different, because DCedar corresponds roughly to the boot file and Basic.loadees, whose components are not started in the same way as DCedarChest components. However, DCedar programs also refer to their data files, if any, by relative names. By convention a certain local directory (unspecified in the convention, but currently []<>7.0>System>) is current during DCedar booting. A special DCedar-wide DF (BootEssentials.df) describes all the data files needed by DCedar programs, and DCedar booting normally includes a BringOver of this DF into that certain local directory. Again, if the data files are to be read later, the directory current during booting must be remembered.
In PCedar, we do not have the Cedar/CedarChest distinction --- but there is a distinction between dynamically loaded packages and those bound into a packaged world via UNIX ld. The PCedar conventions ensure that during the starting of a dynamically loaded package, the release directory of the package (ie, /PCedar/PackageName/) is current. Thus, dynamically loaded packages could also refer to their data files by short names. Alas, it is not so for packages bound into packaged worlds, because there is no guarantee whatsoever about what directory will be current while a packaged world is starting. Note that this is worse than DCedar's problem in two ways: every package can be started in either way, and the current working directory is completely unspecified when starting pcakaged worlds. The solution is that PCedar packages search for their data files down the famous path, which is the following ordered list of directories:
1. the directory current when the package was started,
2. /PCedar/FamousFiles/, and
3. /CedarCommon/FamousFiles/.
The starting directory is first so that when a maintainer is testing a new version, the new data files are found in preference to the old ones. /CedarCommon/FamousFiles/ is in the list so that these data files may be shared between PCedar and DCedar. /PCedar/FamousFiles/ comes before /CedarCommon/FamousFiles/ so that when a maintainer decides to cease source-sharing a data file, the new PCedar-specific version will be found in preference to the old shared version; it is expected that ceasing to source-share a previously shared file will be more common that starting to share a previously unshared-but-present-in-both-DCedar-and-PCedar file.
Because the famous path uses .../FamousFiles/ rather than .../PackageName/, the package DF that owns the data file must put that data file in .../FamousFiles/ rather than .../PackageName/ (where ... is /PCedar or /CedarCommon, depending on whether the data file is shared).
The implementations of Icons.NewIconFromFile and TIPUser.InstantiateNewTIPTable always search along the famous path when given a relative name; for all other kinds of data files this searching is not built in, and the package owning the data file is responsible for explicitly searching the famous path.
There is a feeling underfoot that .../FamousFiles/ might not be a good idea --- a suspicion that we may want to change the famous path to be:
1. the directory current when the package was started,
2. /PCedar/PackageName/, and
3. /CedarCommon/PackageName/.
Note that this makes the famous path depend on the name of the package owning the data file. And note that this may be different from the package that actually reads the data file; for example, while Gargoyle owns its TIP table, that table is read by the TIP package.
If the module that determines where to find a data file is to be source-shared, it must follow both the DCedar(Chest) rules and the PCedar ones. To make this possible, we imagine that there is a famous path in DCedar and DCedarChest, and that it consists of exactly one directory: the one current when the package is started. Furthermore, there is an interface (FamousPath) that supports capturing and searching along this path. This interface can be used by any DCedar(Chest) or PCedar program. To be usable by programs in the DCedar bootfile means it must be a bootfile component, immediately following the FS package. [In an effort to minimize future changes, this interface takes the package name as a parameter, even though it is not currently used.] In PCedar, the implementation of this interface may be different for packaged worlds vs. dynamically loaded worlds; this means that clients cannot write a `Require' command that will always get the correct implementation, and thus the loading of the implementation, right after that of PFS, is the responsibility of each `World.pcr' file and world-packaging process. The interface looks like this:
Returns [NIL, NIL] if not found; otherwise, fullFName is equal to directory.Concat[relativeFName] modulo case, except that relativeFName may have less specific version information than the corresponding part of fullFName.
Returns [NIL, NIL] if not found; otherwise, fullFName is equal to directory.Concat[relativeFName] modulo case, except that relativeFName may have less specific version information than the corresponding part of fullFName.
To: Pier:PARC:Xerox, Plass:PARC:Xerox, Willie-Sue:PARC:Xerox, CHauser:PARC:Xerox, Foote:OSBU North:Xerox
Cc: Mike Spreitzer:PARC:Xerox
It occurs to me that the following two procedures would be good to add to the FamousPath interface (for example, the TIP and Icons impls could just call Resolve).
I've been thinking that maybe it is a mistake for Require to change the working directory. Suppose instead that it created a separate (per-process) RequireDirectory property that was understood by the Run command (and the Install command). This would make the semantic differences between packaged and unpackaged startups mostly disappear.
Application require files could (if desired) set the working directory to the place they expected to get their files, i.e.,
Viewers.require
Require PCedar BasicCedar BasicCedar
Require PCedar PFS FS
Require PCedar UserProfile UserProfile
Require PCedar CedarProcess CedarProcess
Require PCedar Imager Imager
Require PCedar TIP TIP
From /PCedar/FamousFiles/ Run ViewersPackage
and packaged worlds could have an associated directory for application files, which would get refreshed at the time the packaged worlds got made.
This would be a trivial change to the Commander. Thoughts?
23-Apr-90 Mike Spreitzer Re: FamousFiles
Date: Mon, 23 Apr 90 13:37:02 PDT
From: Mike Spreitzer:PARC:Xerox
Subject: Re: FamousFiles
In-reply-to: "Michael Plass:PARC:Xerox's message of 20 Apr 90 10:51:24 PDT"
To: Michael Plass:PARC:Xerox
Cc: Mike Spreitzer:PARC:Xerox, Pier:PARC:Xerox, Willie-Sue:PARC:Xerox, CHauser:PARC:Xerox, Foote:OSBU North:Xerox
I see two separable proposals in your message:
(1) Change Require to set the RequireDirectory instead of the WorkingDirectory.
(2) Let packaged worlds have an associated directory for application files.
I think proposal 2 is clearly a step in the right direction, and I support it. Note that there has been talk about even more ambitious schemes for binding packaged applications to their data files.
[I say packaged applications instead of packaged worlds because packaged worlds can dynamically load further applications. One of my concerns is isolating the knowledge of how the famous path changes (if at all) when this is done.]
I don't see any benefit in proposal 1. Maintainers still want the development directory at the front of the famous path while testing, which means that .require files with names like "/PCedar/FamousFiles/" in them force maintainers to do a lot of tedious and error-prone manipulation of the prefix maps --- and to remember to undo them when done testing!
The current behavior of Require nicely unifies the maintainence and the non-packaged release behaviors; I think it is the packaged behavior that is troublesome and should be attacked. It is only packaged applications that do not know exactly what directory to look in; propogating this unclarity to other situations seems like a bad idea to me.
As long as there is uncertainty of which directory to look in, there will be a need for a FamousPath interface.
Mike
1-May-90 Mike Spreitzer FamousFiles
Date: Tue, 01 May 90 17:05:56 PDT
From: Mike Spreitzer:PARC:Xerox
Subject: FamousFiles
To: Pier, Plass, Willie-Sue, CHauser, Foote:OSBU North:Xerox
Cc: Spreitzer
I'm trying to get some action on my FamousPath proposal. Unless I hear otherwise, I will proceed as proposed. Following is the proposal. It differs from what you saw last time in the recognition that the DCedar famous path includes /CedarCommon2.0/FamousFiles/.
If the module that determines where to find a data file is to be source-shared, it must follow both the DCedar(Chest) rules and the PCedar ones. To make this possible, we imagine that there is a famous path in DCedar and DCedarChest, and that it consists of two directories: the one current when the package is started, and /CedarCommon/FamousFiles/. Furthermore, there is an interface (FamousPath) that supports capturing and searching along this path. This interface can be used by any DCedar(Chest) or PCedar program. To be usable by programs in the DCedar bootfile means it must be a bootfile component, immediately following the FS package; it may be good enough to make the impl one of the Basic.loadees. [In an effort to minimize future changes, this interface takes the package name as a parameter, even though it is not currently used.] In PCedar, the implementation of this interface may be different for packaged worlds vs. dynamically loaded worlds; this means that clients cannot write a `Require' command that will always get the correct implementation, and thus the loading of the implementation, right after that of PFS, is the responsibility of each `World.pcr' file and world-packaging process. The interface looks like this:
Returns [NIL, NIL] if not found; otherwise, fullFName is equal to directory.Concat[relativeFName] modulo case, except that relativeFName may have less specific version information than the corresponding part of fullFName.
Returns [NIL, NIL] if not found; otherwise, fullFName is equal to directory.Concat[relativeFName] modulo case, except that relativeFName may have less specific version information than the corresponding part of fullFName.
Why doesn't SearchList also take a packageName as a parameter?
Why doesn't ListResolve also take a packageName as a parameter?
The comment for SearchList is a copy of the comment for Search instead of a description of the use of the DirectoryList.
Otherwise, I will incorporate your previous documentation for FamousFiles into DFSuitesDoc and wait for a revised DEFINITIONS module is needed.
Thanks.
Ken
4-May-90 Mike Spreitzer Re: FamousFiles
Date: Fri, 04 May 90 09:39:33 PDT
From: Mike Spreitzer:PARC:Xerox
Subject: Re: FamousFiles
In-reply-to: "Kenneth A Pier:PARC:Xerox's message of 01 May 90 17:44:53 PDT"
To: Kenneth A Pier:PARC:Xerox
Cc: Mike Spreitzer:PARC:Xerox, Plass:PARC:Xerox, Willie-Sue:PARC:Xerox, CHauser:PARC:Xerox, Foote:OSBU North:Xerox
SearchList and ListResolve take explicit paths to search, instead of taking a packageName from which to compute the famous path to search.
The comments for SearchList and Search were the same because those two procedures do the same thing, for the issues the comments addressed. I've added a sentence to each comment to address the issues that differentiate those procedures.