RESOURCES: A Resource Management Tool

Interlisp is based on the use of a storage-management system which allocates memory space for new data objects, and automatically reclaims the space when no longer in use.   More generally, Interlisp manages shared "resources", such as files, semaphors for processes, etc.  The protocols for allocating and freeing such resources resemble those of ordinary storage management.

Sometimes users need to explicitly manage the allocation of resources.  They may desire the efficiency of explicit reclamation of certain temporary data; or it may be expensive to initialize a complex data object; or there may be an application that must not allocate new cells during some critical section of code.

The file package type RESOURCES is available to help with the definition and usage of such classes of data; the definition of a RESOURCE specifies prototype code to do the basic management operations.  The filepkg command RESOURCES is used to save such definitions on files, and INITRESOURCES causes the initialization code to be output (similar to INITRECORDS).

The basic needs of resource management are (1) obtaining a data item from the Lisp memory management system and configuring it to be a totally new instance of the resource in question, (2) freeing up an instance which is no longer needed, (3) getting an instance of the resource for temporary usage [whether "fresh" or a formerly freed-up instance], and (4) setting up any prerequisite global data structures and variables.  A RESOURCES definition consists of four "methods":  INIT, NEW, GET, and FREE; each "method" is a form that will specialize the definition for four corresponding user-level macros INITRESOURCE, NEWRESOURCE, GETRESOURCE, and FREERESOURCE.   PUTDEF is used to make a RESOURCES definition, and the four components are specified in a proplist:

		(PUTDEF '<resourceName>
			   'RESOURCES
			   '(NEW  <new-instance-generation code>
				FREE <freeing-up code>
				GET  <get-instance code>
				INIT <initialization code>))

In each case the <.. code> is a form that will appear as if it were the body of a substitution macro definition for the corresponding macro [see the discussion on the macros below].



					A Simple Example

Suppose one has several pieces of code which use a 256-character string as a scratch string.  One could simply generate a new string each time, but that would be inefficient if done repeatedly.  If the user can guarantee that there are no re-entrant uses of the scratch string, then it could simply be stored in a global variable.  However, if the code might be re-entrant on occasion, the program has to take precautions that two programs do not use the same scratch string at the same time.  [Note: this consideration becomes very important in a multi-process environment.  It is hard to guarantee that two processes won't be running the same code at the same time, without using elaborate locks.]  A typical tactic would be to store the scratch string in a global variable, and set the variable to NIL whenever the string is in use (so that re-entrant usages would know to get a "new" instance).  For example, assuming the global variable TEMPSTRINGBUFFER is initialized to NIL:

      [DEFINEQ (WITHSTRING NIL 
	   (PROG ((BUF (OR (PROG1 TEMPSTRINGBUFFER (SETQ TEMPSTRINGBUFFER NIL))
				    (ALLOCSTRING 256))))
		... use the scratch string in the variable BUF ...
		(SETQ TEMPSTRINGBUFFER BUF)
		(RETURN]

Here, the basic elements of a "resource" usage may be seen: (1) a call (ALLOCSTRING 256) allocates fresh instances of "buffer", (2) after usage is completed the instance is returned to the "free" state, by putting it back in the globalvar TEMPSTRINGBUFFER where a subsequent call will find it, (3) the prog-binding of BUF will get an existing instance of a string buffer if there is one -- otherwise it will get a new instance which will later be available for reuse, and (4) some initialization is performed before usage of the resource (in this case, it is the setting of the global variable TEMPSTRINGBUFFER).

Given the following RESOURCES definition:

  (PUTDEF 'STRINGBUFFER 
	     'RESOURCES
	     '(NEW  (ALLOCSTRING 256)
		  FREE (SETQ TEMPSTRINGBUFFER (PROG1 . ARGS))
		  GET  (OR (PROG1 TEMPSTRINGBUFFER (SETQ TEMPSTRINGBUFFER NIL))
				(NEWRESOURCE TEMPSTRINGBUFFER)))
		  INIT (SETQ TEMPSTRINGBUFFER NIL)))

we could then redo the example above as 

  (DEFINEQ (WITHSTRING NIL
	(PROG ((BUF (GETRESOURCE STRINGBUFFER)))
		... use the string in the variable BUF ...
		(FREERESOURCE STRINGBUFFER BUF)
		(RETURN]

The advantage of doing the coding this way is that the resource management part of WITHSTRING is fully contained in the expansions of GETRESOURCE and FREERESOURCE, and thus there is no confusion between what is WITHSTRING code and what is resource management code.  This particuar advantage will be multiplied if there are other functions which need a "temporary" string buffer; and of course, the resultant modularity makes it much easier to contemplate minor variations on, as well as multiple clients of, the STRINGBUFFER resource.

In fact, the scenario just shown above in the WITHSTRING example is so commonly useful that an abbreviation has been added; if a RESOURCES definition is made with *only* a NEW method, then appropriate FREE, GET, and INIT methods will be inferred, along with a coordinated globalvar, to be parallel to the above definition.  So the above definition could be more simply written

  (PUTDEF 'STRINGBUFFER 
	     'RESOURCES
	     '(NEW  (ALLOCSTRING 256)))

and every thing would work the same.

The macro WITH-RESOURCES simplifies the common scoping case, where at the beginning of some piece of code, there are one or more GETRESOURCE calls the results of which are each bound to a lambda variable; and at the ending of that code a FREERESOURCE call is done on each instance.  Since the resources are locally bound to variables with the same name as the resource itself, the definition for WITHSTRING then simplifies to

  (DEFINEQ (WITHSTRING NIL
	(WITH-RESOURCES (STRINGBUFFER) 
	   	 ... use the string in the variable STRINGBUFFER ...)


               Trade-offs in More Complicated Cases

This simple example presumes that the various functions which use the resource are generally not re-entrant.  While an occasional re-entrant use will be handled correctly (another example of the resource will simply be created), if this were to happen too often, then many of the resource requests will create and throw away new objects, which defeats one of the major purposes of using RESOURCES.  A slightly more complex GET and FREE method can be of much benefit in maintaining a pool of available resources;  if the resource were defined to maintain a list of "free" instances, then the GET method could simply take one off the list and the FREE method could just push it back onto the list.  In this simple example, the SETQ in the FREE method defined above would just become a "push", and the first clause of the GET method would just be (pop TEMPSTRINGBUFFER)

A word of caution: if the datatype of the resource is something very small that Interlisp system is "good" at allocating and reclaiming, then explicit user storage management will probably not do much better than the combination of cons/createcell and the garbage collector.  This would especially be so if more complicated GET and FREE methods were to be used, since their overhead would be closer to that of the built-in system facilities.  Finally, it must be considered whether retaining multiple instances of the resource is a net gain; if the re-entrant case is truly rare, it may be more worthwhile to retain at most one instance, and simply let the instances created by the rarely-used case be reclaimed in the normal course of garbage collection.



               User-Level Macros for Accessing RESOURCES

Four user-level macros are defined for accessing RESOURCES:

(NEWRESOURCE <resourceName>  . ARGS)                              [Macro]
(FREERESOURCE <resourceName> . ARGS)                              [Macro]
(GETRESOURCE <resourceName> . ARGS)                               [Macro]
(INITRESOURCE <resourceName> . ARGS)                              [Macro]

Each of these macros behave as if they were defined as a substitution macro of the form

      ((RESOURCENAME . ARGS) ...macrobody...)

where the expression "...macrobody..." is selected by using the "code" supplied by the corresponding method from the <resourceName> definition.

Note that it is possible to pass "arguments" to the user's resource allocation macros.  For example, if the "GET" method for the resource FOO is (GETFOO . ARGS), then (GETRESOURCE FOO X Y) is transformed into (GETFOO X Y).  This form was used in the "FREE" method of the STRINGBUFFER resource described above, to pass the old STRINGBUFFER object to be freed.


(WITH-RESOURCES (<resource1> <resource2> ...)  <form1> <form2> ...)     [Macro] 

The WITH-RESOURCES macro binds lambda variables of the same name as the resources (for each of the resources <resource1>, <resource2>, ...) to the result of the GETRESOURCE macro; then executes the forms <form1>, <form2>, etc., does a FREERESOURCE on each instance, and returns the value of the last form (evaluated and saved before the FREERESOURCEs).

Note:  (WITH-RESOURCES <resourceName> ...) is interpreted the same as (WITH-RESOURCES (<resourceName>) ...).  Also, the singular name WITH-RESOURCE is accepted as a synonym for WITH-RESOURCES.



  				Saving RESOURCES in a File

RESOURCES definitions may be saved on files using the RESOURCES filepkg command.  Typically, one only needs the full definition available when compiling or interpreting the code, so it is appropriate to put the filepkg command in a (DECLARE: EVAL@COMPILE DONTCOPY ...) declaration, just as one might do for a RECORDS declaration.  But just as certain record declarations need *some* initialization in the run-time environment, so do most resources.  This initialization is specified by the resource' INIT method, which is executed automatically when the resource is defined by the PUTDEF output by the RESOURCES command.   However, if the RESOURCES command is in a DONTCOPY expression and thus is not included in the compiled file, then it is necessary to include a separate INITRESOURCES command in the filecoms to insure that the resource is properly initialized.