///Users/Atkinson.pa/ExamineStorageDoc.tioga
Copyright © 1985, 1986 by Xerox Corporation. All rights reserved.
Russ Atkinson (RRA) April 1, 1986 1:57:57 pm PST
Spreitzer, June 9, 1986 10:43:59 pm PDT
EXAMINE STORAGE
CEDAR 6.0 — FOR INTERNAL XEROX USE ONLY
Examine Storage
-- commands to examine Cedar heap storage
Russ Atkinson
© Copyright 1985, 1986 Xerox Corporation. All rights reserved.
Abstract: Many programs can squander storage without presenting any obvious symptoms to the user. ExamineStorage.bcd provides commands and procedures to aid the user in examining Cedar heap storage usage. ReclaimFreePages provides commands to reclaim free pages from the SafeStorage heap.
Created by: Russ Atkinson
Maintained by: Russ Atkinson <Atkinson.pa>
Keywords: heap storage, reference
XEROX   Xerox Corporation
    Palo Alto Research Center
    3333 Coyote Hill Road
    Palo Alto, California 94304

For Internal Xerox Use Only
1. ExamineStorage.bcd
ExamineStorage.bcd is a module that assists in examining the state of the SafeStorage heap. Most users will only be interested in the TakeHeapStats command.
TakeHeapStats Command
TakeHeapStats is a command to take statistics over the heap. It can take either aggregate or delta statistics. The following is sample output using aggregate statistics (italics are comments to explain the fields):
-- Heap storage statistics (August 6, 1985 1:04:44 pm PDT) --
types: 433, objects: 33373, words: 650850 (words/obj: 20)
types: number of different types of objects
objects: number of objects seen
words: number of words in the objects (includes overhead)
RC - 0: 40, 1: 27331, 2: 3632, 3: 1001
40 objects with RC = 0
27331 objects with RC = 1
... and so on
RC - 4: 311, 5: 394, 6: 163, 7: 146
RC - 8: 65, 9: 42, 10: 24, 11: 19
RC - 12: 20, 13: 13, 14: 16, 15: 20
RC IN [16..31]: 75, IN [32..63]: 19, overflow: 42
75 objects with RC IN [16..31]
42 objects with overflow RC
currently with RC > 32
at one time, RC > 63
inZCT: 77, inZCT0; 40
77 objects with references in the Zero Count Table
40 objects with RC = 0 and references in the ZCT
-- Rank by words --
rank 0, words: 142928, objects: 8477, type: 17B
Rope.RopeRep
rank 1, words: 65024, objects: 6, type: 566B
VersionMap.MapRep
rank 2, words: 21752, objects: 9, type: 16B
RefQueueImpl.Object
rank 3, words: 19880, objects: 2485, type: 1234B
[first: TIPTables.TIPTerm, rest: LIST OF TIPTables.TIPTerm]
rank 4, words: 19576, objects: 137, type: 46B
TEXT
rank 5, words: 17596, objects: 727, type: 1127B
ImagerDevice.CharMaskRep
rank 6, words: 15400, objects: 1925, type: 36B
AtomPrivate.AtomRec
rank 7, words: 14960, objects: 374, type: 1254B
RECORD[class: ViewerClasses.ViewerClass ← NIL, wx: INTEGER ← 0, wy: INTEGER ← 0, ww: INTEGER ← 0, ...]
(actually = ViewerClasses.ViewerRec)
rank 8, words: 14912, objects: 313, type: 25B
SafeStorageOpsImpl.RefLitTable
rank 9, words: 14334, objects: 2389, type: 41B
[first: REF ANY, rest: LIST OF REF ANY]
When using delta statistics, the ranking is based on the absolute value of the difference in words (or objects) for each type from the last time statistics were taken.
The following switches control the behavior of TakeHeapStats (defaults in parens):
-d: (FALSE) print delta statistics
-f: (FALSE) include free objects in counts
-g: (TRUE) GC once before taking stats
-n: (FALSE) no save of stats for next time
-o: (FALSE) show objects rank
-r: (FALSE) reprint only, don't take stats
-s: (FALSE) silent, don't print stats
-w: (TRUE) show words rank
The following variables (in ExamineStorage) control the behavior of TakeHeapStats (defaults in parens):
typeDepth: (3) specifies depth of type printing
typeWidth: (4) specifies width of type printing
typeVerbose: (FALSE) specifies verbosity of type printing
typeRank: (10) specifies number of items to display in the type ranking
As a side benefit, the heap is validated when performing the scan.
FindCyclicTypes Command
FindCyclicTypes is similar to TakeHeapStats, but will only report on objects that are involved in cycles (i.e., that can reach themselves by following REFs). The output and switches are similar to TakeHeapStats.
The best way to use this command is to execute "TakeHeapStats" once to get base statistics, then to run the program you suspect of creating cyclic objects, then to execute "TakeHeapStats -d" to see the differences.
The following switches control the behavior of FindCyclicTypes (defaults in parens):
-d: (FALSE) print delta statistics
-g: (TRUE) GC once before taking stats
-n: (FALSE) no save of stats for next time
-o: (FALSE) show objects rank
-r: (FALSE) reprint only, don't take stats
-s: (FALSE) silent, don't print stats
-w: (TRUE) show words rank
Printing is governed by the same variables as TakeHeapStats. As a side benefit, the heap is validated when performing the scan.
FindBadGuys Command
FindBadGuys is a command to find suspect objects and print information about them. The "bad" predicate is based on the value of variables in the global frame ExamineStorage. These variables are:
minType: minimum type code to be found (default SafeStorage.nullType)
maxType: maximum type code to be found (default [LAST[SafeStorage.TypeIndex]])
allowF: TRUE => can find finalizable objects (default TRUE)
allowNotF: TRUE => can find non-finalizable objects (default TRUE)
allowOver: TRUE => can find objects with overflowed counts (default TRUE)
allowNotOver: TRUE => can find objects with non-overflowed counts (default TRUE)
minBsi: minimum block size for found objects (default FIRST[Allocator.BlockSizeIndex])
maxBsi: maximum block size for found objects (default LAST[Allocator.BlockSizeIndex])
minRC: minimum block size for found objects (default FIRST[Allocator.RefCount])
maxRC: maximum block size for found objects (default LAST[Allocator.RefCount])
An object must pass all of the tests in order to be found. However, as a minor implementation limitation, only the first 64 bad guys will be displayed. As an example, to find all objects with overflowed reference counts, do the following:
Run ExamineStorage
← ExamineStorage.allowNotOver ← FALSE
FindBadGuys
Note that the variables ExamineStorage.typeDepth and ExamineStorage.typeWidth specify the depth and width for the printing of types.
As a side benefit, the heap is validated when performing the scan.
ValidateHeap Command
ValidateHeap is a quick scan over the heap that validates many of the heap invariants. There is one switch (w) that forces a world-swap debugger call if a broken invariant is detected. Otherwise an error is raised (after the locks are released, of course) if a broken invariant is detected.
ExamineRefs Procedure
OK, so you've done TakeHeapStats, and you found out that most of your storage is going into ropes; the next question is, who owns those ropes? ExamineRefs is for getting the answer.
ExamineRefs: PROC [goalList, transparentList: TypeList, log: STREAM, gcFirst, wordRank: BOOL ← TRUE, objectRank: BOOL ← FALSE]
It will scan memory, looking for references to objects whose type is equivalent to one in the goalList. It keeps track of how many times such a reference is made from each type of object, and how many words worth of referent each such reference makes from each type of object. Now, the next thing you'd notice is that a lot of your interesting stuff is owned by the cons cells of LIST OF REF ANYs, which is also boring --- you want to credit the reference to whoever owns the LIST OF REF ANY. This is what the transparentList argument is for. The references credited to an object are not only those contained directly in that object, but also those reachable by tracing through REFs to objects whose types are equivalent to ones in the transparentList. A couple of convenient values for transparentList are provided in basicBores and moreBores.

% ← ExamineStorage.ExamineRefs[goalList: LIST[CODE[Rope.RopeRep]], transparentList: ExamineStorage.moreBores, log: &stdout[]]

-- Heap storage statistics (June 9, 1986 10:36:41 pm PDT) --
types: 98, objects: 51198, words: 858522 (words/obj: 17)
RC - 0: 0, 1: 0, 2: 0, 3: 0
RC - 4: 0, 5: 0, 6: 0, 7: 0
RC - 8: 0, 9: 0, 10: 0, 11: 0
RC - 12: 0, 13: 0, 14: 0, 15: 0
RC IN [16..31]: 0, IN [32..63]: 0, overflow: 0
inZCT: 0, inZCT0; 0

-- Rank by words --
rank 0, words: 260438, objects: 15765, type: 1421B
TextNode.Body
rank 1, words: 236900, objects: 14302, type: 752B
RECORD[in: STREAM ← NIL, out: STREAM ← NIL, err: STREAM ← NIL, commandLine: ROPE, ...]
rank 2, words: 123702, objects: 7178, type: 1424B
EditNotify.Change
rank 3, words: 109480, objects: 6041, type: 24B
SafeStorageOpsImpl.RefLitTable
rank 4, words: 28038, objects: 1603, type: 1360B
RopeReader.Body
rank 5, words: 22470, objects: 2215, type: 35B
AtomPrivate.AtomRec
rank 6, words: 15696, objects: 283, type: 535B
Commander.CommandProcObject
rank 7, words: 14392, objects: 730, type: 2770B
WalnutMsgSetDisplayerPrivate.MsgInfoRec
rank 8, words: 6036, objects: 354, type: 1461B
NodePropsImpl.NodePropsBody
rank 9, words: 4184, objects: 357, type: 1252B
ViewerClasses.ViewerRec
2. ReclaimFreePages.bcd
ReclaimFreePages.bcd is used to reclaim pages that are composed of entirely free collectible objects in the SafeStorage heap. This can significantly reduce the amount of VM used. As a side effect it verifies some integrity constraints for the free lists. Currently these commands only reclaim free pages from the small object free lists.
AutoReclaimFreePages
AutoReclaimFreePages is a command that forks a process that waits for a given number of seconds (specified on the command line), then waits for a GC to start and complete, then reclaims the free pages from the heap as described above. 60 seconds is the maximum wait and the default wait.
ForceReclaimFreePages
ForceReclaimFreePages is a command that performs a given number of GCs (specified on the command line), then reclaims the free pages from the heap as described above. 100 is the maximum number of GCs, and 1 is the default.