File [_CDMiwok_01_]<bonsai2.0>pcr>spcr-1.0>gc>README

THIS MATERIAL IS PROVIDED AS IS, WITH ABSOLUTELY NO WARRANTY EXPRESSED
OR IMPLIED. ANY USE IS AT YOUR OWN RISK.

Permission is hereby granted to copy this garbage collector for any purpose,
provided the above notices are retained on all copies.

This is version 2.6. Note that functions were renamed since version 1.9
to make naming consistent with PCR collectors.

HISTORY -

Early versions of this collector were developed as a part of research
projects supported in part by the National Science Foundation
and the Defense Advance Research Projects Agency.
The SPARC specific code was contributed by Mark Weiser
(weiser@parc.xerox.com). The Encore Multimax modifications were supplied by
Kevin Kenny (kenny@m.cs.uiuc.edu). The adaptation to the RT is largely due
to Vernon Lee (scorpion@rice.edu), on machines made available by IBM.
Much of the HP specific code and a number of good suggestions for improving the
generic code are due to Walter Underwood (wunder@hp-ses.sde.hp.com).
Robert Brazile (brazile@diamond.bbn.com) originally supplied the ULTRIX code.
Al Dosser (dosser@src.dec.com) and Regis Cridlig (Regis.Cridlig@cl.cam.ac.uk)
subsequently provided updates and information on variation between ULTRIX
systems. Parag Patel (parag@netcom.com) supplied the A/UX code.
Bill Janssen (janssen@parc.xerox.com) supplied the SunOS dynamic loader
specific code. Manuel Serrano (serrano@cornas.inria.fr) supplied linux and
Sony News specific code.

(Blame for misinstallation of those modifications goes to the first author,
however.) Some of the improvements incorporated in this version were
suggested by David Chase, then at Olivetti Research.

Much of the code was rewritten by Hans-J. Boehm at Xerox PARC.

This is intended to be a general purpose, garbage collecting storage
allocator. The algorithms used are described in:

Boehm, H., and M. Weiser, "Garbage Collection in an Uncooperative Environment",
Software Practice & Experience, September 1988, pp. 807-820.

Some of the ideas underlying the collector have previously been explored
by others. (Doug McIlroy wrote a vaguely similar collector that is part of
version 8 UNIX (tm).) However none of this work appears to have been widely
disseminated.

This collector includes numerous refinements not described in the above paper.

Rudimentary tools for use of the collector as a leak detector are included.

GENERAL DESCRIPTION

This is a garbage colecting storage allocator that is intended to be
used as a plug-in replacement for C's malloc.

Since the collector does not require pointers to be tagged, it does not
attempt to ensure that all inaccessible storage is reclaimed. However,
in our experience, it is typically more successful at reclaiming unused
memory than most C programs using explicit deallocation. Unlike manually
introduced leaks, the amount of unreclaimed memory typically stays
bounded.

In the following, an "object" is defined to be a region of memory allocated
by the routines described below.

Any objects not intended to be collected must be pointed to either
from other such accessible objects, or from the registers,
stack, data, or statically allocated bss segments. Pointers from
the stack or registers may point to anywhere inside an object.
However, it is usually assumed that all pointers originating in the
heap point to the beginning of an object. (This does
not disallow interior pointers; it simply requires that there must be a
pointer to the beginning of every accessible object, in addition to any
interior pointers.) There are two facilities for altering this behavior.
The macro ALL←INTERIOR←POINTERS may be defined in gc←private.h to
cause any pointer into an object to retain the object. A routine
GC←register←displacement is provided to allow for more controlled
interior pointer use in the heap. Defining ALL←INTERIOR←POINTERS
is somewhat dangerous. See gc←private.h for details. The routine
GC←register←displacement is described in gc.h.

Note that pointers inside memory allocated by the standard "malloc" are not
seen by the garbage collector. Thus objects pointed to only from such a
region may be prematurely deallocated. It is thus suggested that the
standard "malloc" be used only for memory regions, such as I/O buffers, that
are guaranteed not to contain pointers. Pointers in C language automatic,
static, or register variables, are correctly recognized.

The collector does not generally know how to find pointers in data
areas that are associated with dynamic libraries. This is easy to
remedy IF you know how to find those data areas on your operating
system (see GC←add←roots). Code for doing this under SunOS4.X only is
included (see dynamic←load.c). (Note that it includes a special version
of dlopen, GC←dlopen, that should be called instead of the standard one.
By default, this is not compiled in, since it requires the -ldl library.)
Note that the garbage collector does not need to be informed of shared
read-only data. However if the shared library mechanism can introduce
discontiguous data areas that may contain pointers, then the collector does
need to be informed.

Signal processing for most signals is normally deferred during collection,
and during uninterruptible parts of the allocation process. Unlike
standard ANSI C mallocs, it is intended to be safe to invoke malloc
from a signal handler while another malloc is in progress, provided
the original malloc is not restarted. (Empirically, many UNIX
applications already asssume this.) The allocator/collector can
also be configured for thread-safe operation. (Full signal safety can
also be acheived, but only at the cost of two system calls per malloc,
which is usually unacceptable.)

INSTALLATION AND PORTABILITY

As distributed, the macro SILENT is defined at the top of gc←private.h.
In the event of problems, this can be removed to obtain a moderate
amount of descriptive output for each collection.
(The given statistics exhibit a few peculiarities.
Things don't appear to add up for a variety of reasons, most notably
fragmentation losses. These are probably much more significant for the
contrived program "test.c" than for your application.)

Note that typing "make test" will automatically build the collector
and then run setjmp←test and gctest. Setjmp←test will give you information
about configuring the collector, which is useful primarily if you have
a machine that's not already supported. Gctest is a somewhat superficial
test of collector functionality. Failure is indicated by a core dump or
a message to the effect that the collector is broken. Gctest takes about
20 seconds to run on a SPARCstation 2. On a slower machine,
expect it to take a while. It may use up to 8 MB of memory. (The
multi-threaded version will use more.)

The Makefile will generate a library gc.a which you should link against.
It is suggested that if you need to replace a piece of the collector
(e.g. GC←mark←roots.c) you simply list your version ahead of gc.a on the
ld command line, rather than replacing the one in gc.a. (This will
generate numerous warnings under some versions of AIX, but it still
works.)

The collector currently is designed to run essentially unmodified on
the following machines:

Sun 3
Sun 4 under SunOS 4.X or Solaris2.X
Vax under 4.3BSD, Ultrix
Intel 386 or 486 under OS/2 (no threads) or linux.
Sequent Symmetry (no concurrency)
Encore Multimax (no concurrency)
MIPS M/120 (and presumably M/2000) (RISC/os 4.0 with BSD libraries)
IBM PC/RT (Berkeley UNIX)
IBM RS/6000
HP9000/300
HP9000/700
DECstations under Ultrix
SGI workstations under IRIX
Sony News
Apple MacIntosh under A/UX

For these machines you should check the beginning of gc.h
to verify that the machine type is correctly defined. On
nonSun machines, you may also need to make changes to the
Makefile, as described by comments there.

Dynamic libraries are completely supported only under SunOS4.X
(and even that support is not functional on the last Sun 3 release).
On other machines we recommend that you do one of the following:

1) Add dynamic library support (and send us the code).
2) Use static versions of the libraries.
3) Arrange for dynamic libraries to use the standard malloc.
This is still dangerous if the library stores a pointer to a
garbage collected object. But nearly all standard interfaces
prohibit this, because they deal correctly with pointers
to stack allocated objects. (Strtok is an exception. Don't
use it.)

In all cases we assume that pointer alignment is consistent with that
enforced by the standard C compilers. If you use a nonstandard compiler
you may have to adjust the alignment parameters defined in gc←private.h.

A port to a machine that is not byte addressed, or does not use 32 bit
addresses will require a major effort. (Parts of the code try to anticipate
64 bit addresses. Others will need to be rewritten, since different data
structures are needed.) A port to MSDOS is hopeless, unless you are willing
to assume an 80386 or better, and that only flat 32 bit pointers will ever be
used.

For machines not already mentioned, or for nonstandard compilers, the
following are likely to require change:

1. The parameters at the top of gc←private.h.
The parameters that will usually require adjustment are
STACKBOTTOM, ALIGNMENT and DATASTART. Setjmp←test
prints its guesses of the first two.
DATASTART should be an expression for computing the
address of the beginning of the data segment. This can often be
&etext. But some memory management units require that there be
some unmapped space between the text and the data segment. Thus
it may be more complicated. On UNIX systems, this is rarely
documented. But the adb "$m" command may be helpful. (Note
that DATASTART will usually be a function of &etext. Thus a
single experiment is usually insufficient.)
STACKBOTTOM is used to initialize GC←stackbottom, which
should be a sufficient approximation to the coldest stack address.
On some machines, it is difficult to obtain such a value that is
valid across a variety of MMUs, OS releases, etc. A number of
alternatives exist for using the collector in spite of this. See the
discussion in gc←private.h immediately preceding the various
definitions of STACKBOTTOM.

2. mach←dep.c.
The most important routine here is one to mark from registers.
The distributed file includes a generic hack (based on setjmp) that
happens to work on many machines, and may work on yours. Try
compiling and running setjmp←test.c to see whether it has a chance of
working. (This is not correct C, so don't blame your compiler if it
doesn't work. Based on limited experience, register window machines
are likely to cause trouble. If your version of setjmp claims that
all accessible variables, including registers, have the value they
had at the time of the longjmp, it also will not work. Vanilla 4.2 BSD
makes such a claim. SunOS does not.)
If your compiler does not allow in-line assembly code, or if you prefer
not to use such a facility, mach←dep.c may be replaced by a .s file
(as we did for the MIPS machine and the PC/RT).

3. mark←roots.c.
These are the top level mark routines that determine which sections
of memory the collector should mark from. This is normally not
architecture specific (aside from the macros defined in gc←private.h and
referenced here), but it can be programming language and compiler
specific. The supplied routine should work for most C compilers
running under UNIX. Calls to GC←add←roots may sometimes be used
for similar effect.

4. The sigsetmask call does not appear to exist under early system V UNIX.
It is used by the collector to block and unblock signals at times at
which an asynchronous allocation inside a signal handler could not
be tolerated. Under system V, it is possible to remove these calls,
provided no storage allocation is done by signal handlers. The
alternative is to issue a sequence of system V system calls, one per
signal that is actually used. This may be a bit slow.

For a different versions of Berkeley UN*X or different machines using the
Motorola 68000, Vax, SPARC, 80386, NS 32000, PC/RT, or MIPS architecture,
it should frequently suffice to change definitions in gc←private.h.

THE C INTERFACE TO THE ALLOCATOR

The following routines are intended to be directly called by the user.
Note that usually only GC←malloc is necessary. GC←clear←roots and GC←add←roots
calls may be required if the collector has to trace from nonstandard places
(e.g. from dynamic library data areas on a machine on which the
collector doesn't already understand them.) On some machines, it may
be desirable to set GC←stacktop to a good approximation of the stack base.
(This enhances code portability on HP PA machines, since there is no
good way for the collector to compute this value.) Client code may include
"gc.h", which defines all of the following, plus a few others.

1) GC←malloc(nbytes)
- allocate an object of size nbytes. Unlike malloc, the object is
cleared before being returned to the user. Gc←malloc will
invoke the garbage collector when it determines this to be appropriate.
GC←malloc may return 0 if it is unable to acquire sufficient
space from the operating system. This is the most probable
consequence of running out of space. Other possible consequences
are that a function call will fail due to lack of stack space,
or that the collector will fail in other ways because it cannot
maintain its internal data structures, or that a crucial system
process will fail and take down the machine. Most of these
possibilities are independent of the malloc implementation.

2) GC←malloc←atomic(nbytes)
- allocate an object of size nbytes that is guaranteed not to contain any
pointers. The returned object is not guaranteed to be cleeared.
(Can always be replaced by GC←malloc, but results in faster collection
times. The collector will probably run faster if large character
arrays, etc. are allocated with GC←malloc←atomic than if they are
statically allocated.)

3) GC←realloc(object, new←size)
- change the size of object to be new←size. Returns a pointer to the
new object, which may, or may not, be the same as the pointer to
the old object. The new object is taken to be atomic iff the old one
was. If the new object is composite and larger than the original object,
then the newly added bytes are cleared (we hope). This is very likely
to allocate a new object, unless MERGE←SIZES is defined in gc←private.h.
Even then, it is likely to recycle the old object only if the object
is grown in small additive increments (which, we claim, is generally bad
coding practice.)

4) GC←free(object)
- explicitly deallocate an object returned by GC←malloc or
GC←malloc←atomic. Not necessary, but can be used to minimize
collections if performance is critical.

5) GC←expand←hp(number←of←4K←blocks)
- Explicitly increase the heap size. (This is normally done automatically
if a garbage collection failed to GC←reclaim enough memory. Explicit
calls to GC←expand←hp may prevent unnecessarily frequent collections at
program startup.)

6) GC←clear←roots()
- Reset the collectors idea of where static variables containing pointers
may be located to the empty set of locations. No statically allocated
variables will be traced from after this call, unless there are
intervening GC←add←roots calls. The collector will still trace from
registers and the program stack.

7) GC←add←roots(low←address, high←address←plus←1)
- Add [low←address, high←address) as an area that may contain root pointers
and should be traced by the collector. The static data and bss segments
are considered by default, and should not be added unless GC←clear←roots
has been called. The number of root areas is currently limited to 50.
This is intended as a way to register data areas for dynamic libraries,
or to replace the entire data ans bss segments by smaller areas that are
known to contain all the roots.

8) Several routines to allow for registration of finalization code.
User supplied finalization code may be invoked when an object becomes
unreachable. To call (*f)(obj, x) when obj becomes inaccessible, use
GC←register←finalizer(obj, f, x, 0, 0);
For more sophisticated uses, and for finalization ordering issues,
see gc.h.

The global variable GC←free←space←divisor may be adjusted up from its
default value of 4 to use less space and more collection time, or down for
the opposite effect. Setting it to 1 or 0 will effectively disable collections
and cause all allocations to simply grow the heap.

The variable GC←non←gc←bytes, which is normally 0, may be changed to reflect
the amount of memory allocated by the above routines that should not be
considered as a candidate for collection. Collections are inhibited
if this exceeds a given fraction (currently 3/4) of the total heap size.
The heap is simply expanded instead. Careless use may, of course, result
in excessive memory consumption.

Some additional tuning is possible through the parameters defined
near the top of gc←private.h.

If only GC←malloc is intended to be used, it might be appropriate to define:

#define malloc(n) GC←malloc(n)
#define calloc(m,n) GC←malloc((m)*(n))

For small pieces of VERY allocation intensive code, gc←inline.h
includes some allocation macros that may be used in place of GC←malloc
and friends.

Somewhat different emulations of the standard C allocation routines are
contained and described in "interface.c" (contributed by David Chase, but
subsequently mangled by Hans Boehm). These are appropriate for mixed
systems, where part of the system uses explicit deallocation, and does not
leak. Exclusive use of interface.c routines can result in needless
fragmentation, since certain kinds of object coalescing are only done
by the collector.

All externally visible names in the garbage collector start with "GC←".
To avoid name conflicts, client code should avoid this prefix, except when
accessing garbage collector routines or variables.

The internals of the collector understand different object "kinds" (sometimes
called "regions"). By default, the only two kinds are ATOMIC and NORMAL.
Its should be possible to add others, e.g. for data types for which layout
information is known. The allocation routine "GC←generic←malloc"
takes an explicit kind argument. (You will probably want to add
faster kind-specific routines as well.) You will need to add another kind
descriptor, including your own mark routine to add a new object kind.
This requires a fairly detailed understanding of at least GC←mark.

USE AS LEAK DETECTOR:

The collector may be used to track down leaks in C programs that are
intended to run with malloc/free (e.g. code with extreme real-time or
portability constraints). To do so define FIND←LEAK somewhere in
gc←private.h. This will cause the collector to invoke the report←leak
routine defined near the top of reclaim.c whenever an inaccessible
object is found that has not been explicitly freed.
Productive use of this facility normally involves redefining report←leak
to do something more intelligent. This typically requires annotating
objects with additional information (e.g. creation time stack trace) that
identifies their origin. Such code is typically not very portable, and is
not included here.
If all objects are allocated with GC←DEBUG←MALLOC (see next section),
then the default version of report←leak will report the source file
and line number at which the leaked object was allocated. This may
sometimes be sufficient.

DEBUGGING FACILITIES:

The routines GC←debug←malloc, GC←debug←malloc←atomic, GC←debug←realloc,
and GC←debug←free provide an alternate interface to the collector, which
provides some help with memory overwrite errors, and the like.
Objects allocated in this way are annotated with additional
information. Some of this information is checked during garbage
collections, and detected inconsistencies are reported to stderr.

Simple cases of writing past the end of an allocated object should
be caught if the object is explicitly deallocated, or if the
collector is invoked while the object is live. The first deallocation
of an object will clear the debugging info associated with an
object, so accidentally repeated calls to GC←debug←free will report the
deallocation of an object without debugging information. Out of
memory errors will be reported to stderr, in addition to returning
NIL.

GC←debug←malloc checking during garbage collection is enabled
with the first call to GC←debug←malloc. This will result in some
slowdown during collections. If frequent heap checks are desired,
this can be acheived by explicitly invoking GC←gcollect, e.g. from
the debugger.

GC←debug←malloc allocated objects should not be passed to GC←realloc
or GC←free, and conversely. It is however acceptable to allocate only
some objects with GC←debug←malloc, and to use GC←malloc for other objects,
provided the two pools are kept distinct. In this case, there is a very
low probablility that GC←malloc allocated objects may be misidentified as
having been overwritten. This should happen with probability at most
one in 2**32. This probability is zero if GC←debug←malloc is never called.

GC←debug←malloc, GC←malloc←atomic, and GC←debug←realloc take two
additional trailing arguments, a string and an integer. These are not
interpreted by the allocator. They are stored in the object (the string is
not copied). If an error involving the object is detected, they are printed.

The macros GC←MALLOC, GC←MALLOC←ATOMIC, GC←REALLOC, GC←FREE, and
GC←REGISTER←FINALIZER are also provided. These require the same arguments
as the corresponding (nondebugging) routines. If gc.h is included
with GC←DEBUG defined, they call the debugging versions of these
functions, passing the current file name and line number as the two
extra arguments, where appropriate. If gc.h is included without GC←DEBUG
defined, then all these macros will instead be defined to their nondebugging
equivalents. (GC←REGISTER←FINALIZER is necessary, since pointers to
objects with debugging information are really pointers to a displacement
of 16 bytes form the object beginning, and some translation is necessary
when finalization routines are invoked. For details, about what's stored
in the header, see the definition of the type oh in debug←malloc.c)

BUGS:

Any memory that does not have a recognizable pointer to it will be
reclaimed. Exclusive-or'ing forward and backward links in a list
doesn't cut it.
Some C optimizers may lose the last undisguised pointer to a memory
object as a consequence of clever optimizations. This has almost
never been observed in practice. Send mail to boehm@parc.xerox.com
for suggestions on how to fix your compiler.
This is not a real-time collector. In the standard configuration,
percentage of time required for collection should be constant across
heap sizes. But collection pauses will increase for larger heaps.
(On SPARCstation 2s collection times will be on the order of 300 msecs
per MB of accessible memory that needs to be scanned. Your mileage
may vary.) Much better real-time behavior would be possible if we
had a portable way to identify sections of memory that were recently
modified. Experience with PCR indicates that 100 msec pause times
are probably possible, almost independent of heap size.

RECENT VERSIONS:

Version 1.3 and immediately preceding versions contained spurious
assembly language assignments to TMP←SP. Only the assignment in the PC/RT
code is necessary. On other machines, with certain compiler options,
the assignments can lead to an unsaved register being overwritten.
Known to cause problems under SunOS 3.5 WITHOUT the -O option. (With
-O the compiler recognizes it as dead code. It probably shouldn't,
but that's another story.)

Version 1.4 and earlier versions used compile time determined values
for the stack base. This no longer works on Sun 3s, since Sun 3/80s use
a different stack base. We now use a straightforward heuristic on all
machines on which it is known to work (incl. Sun 3s) and compile-time
determined values for the rest. There should really be library calls
to determine such values.

Version 1.5 and earlier did not ensure 8 byte alignment for objects
allocated on a sparc based machine.

Please address bug reports to boehm@xerox.com. If you are contemplating
a major addition, you might also send mail to ask whether it's already
been done.

Version 1.8 added ULTRIX support in gc←private.h.

Version 1.9 fixed a major bug in gc←realloc.

Version 2.0 introduced a consistent naming convention for collector
routines and added support for registering dynamic library data segments
in the standard mark←roots.c. Most of the data structures were revamped.
The treatment of interior pointers was completely changed. Finalization
was added. Support for locking was added. Object kinds were added.
We added a black listing facility to avoid allocating at addresses known
to occur as integers somewhere in the address space. Much of this
was accomplished by adapting ideas and code from the PCR collector.
The test program was changed and expanded.

Version 2.1 was the first stable version since 1.9, and added support
for PPCR.

Version 2.2 added debugging allocation, and fixed various bugs. Among them:
- GC←realloc could fail to extend the size of the object for certain large object sizes.
- A blatant subscript range error in GC←printf, which unfortunately
wasn't excercised on machines with sufficient stack alignment constraints.
- GC←register←displacement did the wrong thing if it was called after
any allocation had taken place.
- The leak finding code would eventually break after 2048 byte
byte objects leaked.
- interface.c didn't compile.
- The heap size remained much too small for large stacks.
- The stack clearing code behaved badly for large stacks, and perhaps
on HP/PA machines.

Version 2.3 added ALL←INTERIOR←POINTERS and fixed the following bugs:
- Missing declaration of etext in the A/UX version.
- Some PCR root-finding problems.
- Blacklisting was not 100% effective, because the plausible future
heap bounds were being miscalculated.
- GC←realloc didn't handle out-of-memory correctly.
- GC←base could return a nonzero value for addresses inside free blocks.
- test.c wasn't really thread safe, and could erroneously report failure
in a multithreaded environment. (The locking primitives need to be
replaced for other threads packages.)
- GC←CONS was thoroughly broken.
- On a SPARC with dynamic linking, signals stayed diabled while the
client code was running.
(Thanks to Manuel Serrano at INRIA for reporting the last two.)

Version 2.4 added GC←free←space←divisor as a tuning knob, added
support for OS/2 and linux, and fixed the following bugs:
- On machines with unaligned pointers (e.g. Sun 3), every 128th word could
fail to be considered for marking.
- Dynamic←load.c erroneously added 4 bytes to the length of the data and
bss sections of the dynamic library. This could result in a bad memory
reference if the actual length was a multiple of a page. (Observed on
Sun 3. Can probably also happen on a Sun 4.)
(Thanks to Robert Brazile for pointing out that the Sun 3 version
was broken. Dynamic library handling is still broken on Sun 3s
under 4.1.1U1, but apparently not 4.1.1. If you have such a machine,
use -Bstatic.)

Version 2.5 added Solaris dynamic libary support, Solaris/Intel support,
and fixed the following bugs:
- Removed an explicit call to exit(1)
- Fixed calls to GC←printf and GC←err←printf, so the correct number of
arguments are always supplied. The OS/2 C compiler gets confused if
the number of actuals and the number of formals differ. (ANSI C
doesn't require this to work. The ANSI sanctioned way of doing things
causes too many compatibility problems.)

Version 2.6 fixed a bug diagnosed by Al Dosser at DEC. The marker
could lose some pointers in the event of a mark stack overflow, a case
it was intended to handle correctly. (He also pointed out a performance
bug that was tickled under the same circumstances.)