YggIndexMaintImpl.mesa
Copyright Ó 1988, 1989 by Xerox Corporation. All rights reserved.
Bob Hagmann February 10, 1989 2:16:01 pm PST
This module maintains the indices.
DIRECTORY
Atom USING [GetPName, MakeAtom],
Camelot USING [tidT],
Process USING [-- Detach, -- MsecToTicks, SetTimeout],
RedBlackTree USING [Compare, Create, Delete, GetKey, Insert, Lookup, Table],
Rope USING [Concat, Equal, Match, ROPE, Substr],
YggDID USING [DID, EqualDIDs],
YggDIDMap USING [AddComponentFile, GetComponentFiles, OpenDocumentFromDID, RemoveComponentFile],
YggDIDMapPrivate USING [Document, DocumentRep],
YggDIDPrivate USING [DIDRep],
YggEnvironment USING [nullTransID, TransID],
YggFile USING [Create, Delete, FileFromComponentFiles, FileUseFromComponentFiles],
YggFixedNames USING [ChildContainers, Children, IndexPrefix, IndexPrefixSize, ParentContainers, Parents],
YggIndex USING [DeleteEntry, EntryProc, EnumerateEntries, Index, Open, WriteEntry],
YggIndexMaint USING [],
YggInternal USING [FileHandle],
YggNav USING [GetAllProperties, GetContainersInContainer, GetObjectsInContainer, GetPatternsInContainer, GetSuperContainers],
YggRep USING [Attribute, AttributeValue, CompareTimeStamps, did, MaxTimeStamp, metaAttributeMod, NextTimeStamp, nullTimeStamp, TimeStamp, TypedPrimitiveElement, VDoc, VolatizeFromDID],
YggTransaction USING [CreateTrans, EqualTrans, Finish, Outcome];
YggIndexMaintImpl: CEDAR MONITOR
IMPORTS Atom, Process, RedBlackTree, Rope, YggDID, YggDIDMap, YggFile, YggFixedNames, YggIndex, YggNav, YggRep, YggTransaction
EXPORTS YggDID, YggInternal, YggIndexMaint
~
BEGIN
Maintenance of the indices is delayed. As each transaction commits, changes to properties, changes to the containment hierarchy (and hence the side effect of what is indexed where), and indexing requests (the patterns of properties to be indexed in a container) are sent to this module.
For changes to properties or containment, these calls are done during the precommit phase of the transaction. If the system stays up, the commit status of the transaction is later told to this module. Indexing requests calls are done "without" transactions: this module fabricates the transaction.
A list of things to do is constructed (WorkToDoList). This list has all changes: changes to properties, changes to the containment hierarchy, and indexing changes. The WorkToDoList is a list of DID's that have been changed. The requests are appended to the end of some part of the WorkToDoList. If this is for an unknown DID, then a new DID item (LIST OF WorkItem) is built. Otherwise, the proper DID item (LIST OF WorkItem) is appended to.
A single process looks at the WorkToDoList. It scans for work to do. It does all of the properties, containment hierarchy, and indexing requests. If a latch cannot be obtained, the next item is tried.
Apart from doing the updates sequentially, no other order in imposed on the updates. Using some anomolies can occur. The invariant that we wish to enforce is that the object is always indexed in at least all of the places required. Spurious extra indexing is OK as it should be quite infrequent and the indices are just supposed to be hints anyway (and we could write code to clean up the indicies periodically).
To do:
1) Only index the "primary" containers
2) Stabilize the work to do list. Recover it after crash recovery and start up. Write the list into the contents of the object with DID YggDIDPrivate.LowForIndexMaint. Only work items and their completion must be recoverable: the intermediate state of an index change can be redone (possibly with no change).
Use the contents to keep variable sized "WorkItem" records. Keep four (or more) free lists. To allocate, keep a rover and take the next list that is not locked. Lock the list, with unlock during commit/abort notification. If the update commits, add it to the WorkToDoList, with no stable storage update. When an update is done, remove it by trying to add it to the following record, if the following record is free and its list is unlocked. Don't do stable updates to indicate progress in indx updates: just redo the whole update after recovery. Periodically, when all of the lists are empty, reinitialize all of the lists (don't reinitialize again until an update is seen).
Header:
free list start pointers
Records:
0) size Used
1) free? if so, which list
2) sequence number (used when reconstructing lists)
3.. n-1) all the bits
n) size Used (if backward parsing of the lists is needed)
3) Rationalize the use of transactions
Types, global data, and constants
ROPE: TYPE = Rope.ROPE;
Document: TYPE = REF DocumentRep;
DocumentRep: PUBLIC TYPE = YggDIDMapPrivate.DocumentRep;
DID: PUBLIC TYPE ~ REF DIDRep;
DIDRep: PUBLIC TYPE ~ YggDIDPrivate.DIDRep;
DIDTidNotFound: PUBLIC ERROR = CODE;
DoingIndexMaint: BOOL ← FALSE;
MyCondition: CONDITION;
UpdateType: TYPE = {valueChange, addIndex, deleteIndex, metaChanged};
currentTimeStamp: YggRep.TimeStamp ← YggRep.nullTimeStamp;
WorkItem:
TYPE =
RECORD[
did: DID,
tid: Camelot.tidT,
transactionCommitted: {unknown, abort, commit},
updateType: UpdateType,
attributeOrPattern: ROPE,
oldValues: LIST OF LIST OF YggRep.AttributeValue,
newValues: LIST OF LIST OF YggRep.AttributeValue,
metaAttributesChanged: LIST OF YggRep.metaAttributeMod,
containersToApply: LIST OF ContainerItem,
maxTimeStamp: YggRep.TimeStamp ← YggRep.nullTimeStamp,
containersToIndex: LIST OF ContainerItem,
attributePatternDIDs: LIST OF AttrPatDID ← NIL
];
ContainerItem:
TYPE =
RECORD[
did: DID,
done: BOOL ← FALSE,
attributeNamePattern: LIST OF ROPE ← NIL
];
WorkItemsForDID:
TYPE =
RECORD[
stuffToDo: LIST OF WorkItem
];
WorkToDoList: LIST OF WorkItemsForDID ← NIL;
AttrPatDID:
TYPE =
RECORD[
attributeNamePattern: ROPE,
attributes: LIST OF AttrDIDVal
];
AttrDIDVal:
TYPE =
RECORD[
attributeName: ROPE,
didsAndValues: LIST OF DIDVal ← NIL
];
DIDVal:
TYPE =
RECORD[
did: DID,
values: LIST OF LIST OF YggRep.AttributeValue ← NIL
];
containerLock: RedBlackTree.Table;
numLockedContainers: INT ← 0;
containerLockEntry: TYPE ~ REF containerLockRep;
containerLockRep:
TYPE ~
RECORD [
did: DID,
users: INT ← 0
];
Exported procedures
NewValueForAttribute:
PUBLIC
PROC [did: YggDID.
DID, tid: Camelot.tidT, attributeName:
ROPE, oldValues:
LIST
OF
LIST
OF YggRep.AttributeValue, newValues:
LIST
OF
LIST
OF YggRep.AttributeValue] ~ {
Note that this did/attributeName has a new value.
addValueToWorkToDoList[did, tid, attributeName, oldValues, newValues];
};
NewValueForMetaAttribute:
PUBLIC PROC [did: YggDID.
DID, tid: Camelot.tidT, metaAttributesChanged:
LIST
OF YggRep.metaAttributeMod] ~ {
Note that this did/metaattribute has a new value. Called during Pre-Commit.
addNewValueForMetaAttributeToWorkToDoList[did, tid, metaAttributesChanged];
};
NewValueForAttributeCommitStatus:
PUBLIC PROC [did: YggDID.
DID, tid: Camelot.tidT, commited:
BOOL] ~ {
Note state of previous NewValueForAttribute or NewValueForMetaAttribute call(s). Called during Commit or Abort.
noteCommit[did, tid, commited];
};
AddOrRemoveIndexPattern:
PUBLIC
PROC [containerDID: YggDID.
DID, pattern:
ROPE, add:
BOOL]
RETURNS [ok:
BOOL ← TRUE]~ {
Note that this container now has different indexing.
addIndexUpdateToWorkToDoList[containerDID, pattern, add];
};
Utilities
addValueToWorkToDoList:
ENTRY
PROC[did: YggDID.
DID, tid: Camelot.tidT, attributeName:
ROPE, oldValues:
LIST
OF
LIST
OF YggRep.AttributeValue, newValues:
LIST
OF
LIST
OF YggRep.AttributeValue] ~ {
lastwtdl: LIST OF WorkItemsForDID ← NIL;
IF ~DoingIndexMaint THEN RETURN;
FOR wtdl:
LIST
OF WorkItemsForDID ← WorkToDoList, wtdl.rest
UNTIL wtdl =
NIL
DO
IF YggDID.EqualDIDs[wtdl.first.stuffToDo.first.did, did]
THEN {
lastWI: LIST OF WorkItem ← NIL;
FOR lowi:
LIST
OF WorkItem ← wtdl.first.stuffToDo, lowi.rest
UNTIL lowi =
NIL
DO
lastWI ← lowi;
ENDLOOP;
lastWI.rest ← CONS[[did, tid, unknown, valueChange, attributeName, oldValues, newValues, NIL, NIL], NIL];
EXIT;
};
lastwtdl ← wtdl;
REPEAT
FINISHED => {
vta: LIST OF WorkItemsForDID ← LIST[[LIST[[did, tid, unknown, valueChange, attributeName, oldValues, newValues, NIL]]]];
IF lastwtdl = NIL THEN WorkToDoList ← vta
ELSE lastwtdl.rest ← vta;
};
ENDLOOP;
NOTIFY MyCondition;
};
addIndexUpdateToWorkToDoList:
ENTRY
PROC[containerDID: YggDID.
DID, pattern:
ROPE, add:
BOOL] ~ {
lastwtdl: LIST OF WorkItemsForDID ← NIL;
IF ~DoingIndexMaint THEN RETURN;
FOR wtdl:
LIST
OF WorkItemsForDID ← WorkToDoList, wtdl.rest
UNTIL wtdl =
NIL
DO
IF YggDID.EqualDIDs[wtdl.first.stuffToDo.first.did, containerDID]
THEN {
lastWI: LIST OF WorkItem ← NIL;
FOR lowi:
LIST
OF WorkItem ← wtdl.first.stuffToDo, lowi.rest
UNTIL lowi =
NIL
DO
lastWI ← lowi;
ENDLOOP;
lastWI.rest ← CONS[[containerDID, YggEnvironment.nullTransID, commit, IF add THEN addIndex ELSE deleteIndex, pattern, NIL, NIL, NIL, NIL], NIL];
EXIT;
};
lastwtdl ← wtdl;
REPEAT
FINISHED => {
vta: LIST OF WorkItemsForDID ← LIST[[LIST[[containerDID, YggEnvironment.nullTransID, commit, IF add THEN addIndex ELSE deleteIndex, pattern, NIL, NIL, NIL, NIL]]]];
IF lastwtdl = NIL THEN WorkToDoList ← vta
ELSE lastwtdl.rest ← vta;
};
ENDLOOP;
bumpContainerUse[containerDID];
NOTIFY MyCondition;
};
addNewValueForMetaAttributeToWorkToDoList:
ENTRY
PROC[did: YggDID.
DID, tid: Camelot.tidT, metaAttributesChanged:
LIST
OF YggRep.metaAttributeMod] ~ {
lastwtdl: LIST OF WorkItemsForDID ← NIL;
IF ~DoingIndexMaint THEN RETURN;
FOR wtdl:
LIST
OF WorkItemsForDID ← WorkToDoList, wtdl.rest
UNTIL wtdl =
NIL
DO
IF YggDID.EqualDIDs[wtdl.first.stuffToDo.first.did, did]
THEN {
lastWI: LIST OF WorkItem ← NIL;
FOR lowi:
LIST
OF WorkItem ← wtdl.first.stuffToDo, lowi.rest
UNTIL lowi =
NIL
DO
lastWI ← lowi;
ENDLOOP;
lastWI.rest ← CONS[[did, tid, unknown, metaChanged, NIL, NIL, NIL, metaAttributesChanged, NIL], NIL];
EXIT;
};
lastwtdl ← wtdl;
REPEAT
FINISHED => {
vta: LIST OF WorkItemsForDID ← LIST[[LIST[[did, tid, unknown, metaChanged, NIL, NIL, NIL, metaAttributesChanged, NIL]]]];
IF lastwtdl = NIL THEN WorkToDoList ← vta
ELSE lastwtdl.rest ← vta;
};
ENDLOOP;
bumpContainerUse[did];
NOTIFY MyCondition;
};
noteCommit:
ENTRY
PROC [did: YggDID.
DID, tid: Camelot.tidT, commited:
BOOL] ~ {
ENABLE UNWIND => {};
gotOne: BOOL ← FALSE;
IF ~DoingIndexMaint THEN RETURN;
FOR wtdl:
LIST
OF WorkItemsForDID ← WorkToDoList, wtdl.rest
UNTIL wtdl =
NIL
DO
IF YggDID.EqualDIDs[wtdl.first.first.did, did]
THEN {
FOR lowi:
LIST
OF WorkItem ← wtdl.first.stuffToDo, lowi.rest
UNTIL lowi =
NIL
DO
IF YggTransaction.EqualTrans[lowi.first.tid, tid]
THEN {
lowi.first.transactionCommitted ← IF commited THEN commit ELSE abort;
gotOne ← TRUE;
};
ENDLOOP;
EXIT;
};
REPEAT
FINISHED => {
ERROR DIDTidNotFound;
};
ENDLOOP;
IF ~gotOne THEN ERROR DIDTidNotFound;
IF ~commited THEN bumpContainerUse[did];
NOTIFY MyCondition;
};
The process that does all the work
IndexMaintProcess:
PROC = {
DO
-- forever
waitForABit:
ENTRY
PROC ~ {
WAIT MyCondition;
};
didSomething: BOOL ← FALSE;
prevDID: LIST OF WorkItemsForDID ← NIL;
FOR wtdl:
LIST
OF WorkItemsForDID ← WorkToDoList, wtdl.rest
UNTIL wtdl =
NIL
DO
Flip through the work to do list
doneWithDID: BOOL ← FALSE;
entryRemoved: BOOL ← FALSE;
prevWorkItems: LIST OF WorkItemsForDID ← NIL;
removeEntry:
ENTRY
PROC ~ {
IF prevWorkItems # NIL THEN prevWorkItems.first.rest ← wtdl.first.rest -- previous work item for DID still active
ELSE {
-- first work item for this DID
IF prevDID =
NIL
THEN {
-- first DID on the list
IF wtdl.first.rest =
NIL
THEN {
-- all updates for did are done and this is the first did on the list
WorkToDoList ← wtdl.rest;
}
ELSE {
-- more updates for did, this is the first did on the list, and this is the first update on the list
wtdl.first.stuffToDo ← wtdl.first.stuffToDo.rest;
};
}
ELSE {
IF wtdl.first.rest =
NIL
THEN {
-- all updates for did are done and this is not the first did on the list
prevDID.rest ← wtdl.rest;
}
ELSE {
-- more updates for did and this is not the first did on the list
wtdl.first.stuffToDo ← wtdl.first.stuffToDo.rest;
};
};
};
};
vDoc: YggRep.VDoc ← NIL;
vDoc ← YggRep.VolatizeFromDID[YggEnvironment.nullTransID, wtdl.first.first.did, readOnly, [read, fail], TRUE];
IF vDoc = NIL THEN {prevWorkItems ← wtdl; LOOP;}; -- but what if the did has been deleted?
entryRemoved ← FALSE;
FOR lowi:
LIST
OF WorkItem ← wtdl.first, lowi.rest
UNTIL lowi =
NIL
DO
for a DID
doRemoveEntry: BOOL ← FALSE;
somethingChanged: BOOL ← FALSE;
IF lowi.first.transactionCommitted = unknown THEN LOOP; -- waiting for the commit state to be known
IF lowi.first.transactionCommitted = abort THEN {removeEntry[]; LOOP}; -- transaction didn't make it; dump item
SELECT lowi.first.updateType
FROM
valueChange => {
[doRemoveEntry, somethingChanged] ← doValueChange[lowi, vDoc];
};
addIndex => {
[doRemoveEntry, somethingChanged] ← doCreateIndexChange[lowi, vDoc];
};
deleteIndex => {
[doRemoveEntry, somethingChanged] ← doDeleteIndexChange[lowi, vDoc];
};
metaChanged => {
[doRemoveEntry, somethingChanged] ← doMetaChange[lowi, vDoc];
};
ENDCASE => ERROR;
IF doRemoveEntry
THEN {
entryRemoved ← TRUE;
removeEntry[];
didSomething ← TRUE;
};
IF ~entryRemoved THEN prevWorkItems ← wtdl;
prevDID ← wtdl;
ENDLOOP; -- for a DID
ENDLOOP; -- Flip through the work to do list
IF ~didSomething THEN waitForABit[];
ENDLOOP; -- forever
};
process update type procs
doValueChange:
PROC [lowi:
LIST
OF WorkItem, vDoc: YggRep.VDoc]
RETURNS [doRemoveEntry:
BOOL ←
FALSE, somethingChanged:
BOOL ←
FALSE] ~ {
keepTrying: BOOL ← TRUE;
IF lowi.first.containersToApply = NIL THEN getContainersToApply[vDoc, lowi, YggFixedNames.Parents]; -- just starting, so find parents
WHILE keepTrying
DO
FOR parentContainers:
LIST
OF ContainerItem ← lowi.first.containersToApply, parentContainers.rest
UNTIL parentContainers =
NIL
DO
doc: Document;
atomForIndex: ATOM;
transID: YggEnvironment.TransID;
componentFiles: LIST OF YggInternal.FileHandle;
outcome: YggTransaction.Outcome;
superDIDs: LIST OF DID ← NIL;
success: BOOL ← FALSE;
indexFile: YggInternal.FileHandle;
IF parentContainers.first.done THEN LOOP; -- already done
transID ← YggTransaction.CreateTrans[YggEnvironment.nullTransID];
doc ← YggDIDMap.OpenDocumentFromDID[parentContainers.first.did, transID];
lowi.first.maxTimeStamp ← YggRep.MaxTimeStamp[lowi.first.maxTimeStamp, doc.timeStampForIndexMaint];
componentFiles ← YggDIDMap.GetComponentFiles[doc];
atomForIndex ← Atom.MakeAtom[Rope.Concat[YggFixedNames.IndexPrefix, lowi.first.attributeOrPattern]];
indexFile ← YggFile.FileFromComponentFiles[componentFiles: componentFiles, fileUse: atomForIndex];
[superDIDs, success] ← YggNav.GetSuperContainers[transID, parentContainers.first.did, TRUE];
IF ~success
THEN {
outcome ← YggTransaction.Finish[transID: transID, requestedOutcome: abort];
keepTrying ← FALSE;
EXIT;
};
[keepTrying: keepTrying, outcome: outcome] ← doOneChange[transID: transID, doc: doc, indexFile: indexFile, attributeName: lowi.first.attributeOrPattern, atomForIndex: atomForIndex, containerDID: parentContainers.first.did, indexedDID: lowi.first.did, oldValues: lowi.first.oldValues, newValues: lowi.first.newValues];
IF ~keepTrying THEN EXIT;
outcome ← YggTransaction.Finish[transID: transID, requestedOutcome: commit];
IF outcome = commit
THEN {
somethingChanged ← TRUE;
parentContainers.first.done ← TRUE;
IF superDIDs # NIL THEN addSuperDIDs[superDIDs, lowi]; -- append supers to list of stuff to do, provided that they have not yet been seen
}
ELSE {
keepTrying ← FALSE;
EXIT;
};
IF ~keepTrying THEN EXIT;
REPEAT
FINISHED => {
-- all parents processed OK !!!
ok: BOOL ← TRUE;
ok ← checkTimeStamps[lowi, lowi.first.maxTimeStamp];
IF ok
THEN {
doRemoveEntry ← TRUE;
somethingChanged ← TRUE;
keepTrying ← FALSE;
}
ELSE {
-- something changed; force re-evaluation of my super contatiners and redo loop
FOR parentContainers:
LIST
OF ContainerItem ← lowi.first.containersToApply, parentContainers.rest
UNTIL parentContainers =
NIL
DO
parentContainers.first.done ← FALSE;
ENDLOOP;
};
};
ENDLOOP;
ENDLOOP;
};
doMetaChange:
PROC [lowi:
LIST
OF WorkItem, vDoc: YggRep.VDoc]
RETURNS [doRemoveEntry:
BOOL ←
FALSE, somethingChanged:
BOOL ←
FALSE] ~ {
FOR mam:
LIST
OF YggRep.metaAttributeMod ← lowi.first.metaAttributesChanged, mam.rest
UNTIL mam =
NIL
DO
SELECT
TRUE
FROM
Rope.Equal[YggFixedNames.Parents, mam.first.attributeName] => {
add/remove DID for WorkItem to this parent: do nothing; the next clause will handle it.
doRemoveEntry ← TRUE;
somethingChanged ← TRUE;
};
Rope.Equal[YggFixedNames.Children, mam.first.attributeName] => {
-- add/remove DID as child to this parent (in WorkItem)
keepTrying: BOOL ← TRUE;
didSomething: BOOL ← TRUE;
outcome: YggTransaction.Outcome;
containerDID: DID;
transID: YggEnvironment.TransID;
containerDoc: Document;
containerVDoc: YggRep.VDoc;
containerDID ← mam.first.didValue;
containerVDoc ← YggRep.VolatizeFromDID[YggEnvironment.nullTransID, containerDID, readOnly, [read, fail], TRUE];
IF containerVDoc = NIL THEN RETURN [FALSE, FALSE]; -- try again later
IF lowi.first.containersToApply = NIL THEN lowi.first.containersToApply ← LIST[[containerDID, FALSE]];
WHILE keepTrying
DO
FOR parentContainers:
LIST
OF ContainerItem ← lowi.first.containersToApply, parentContainers.rest
UNTIL parentContainers =
NIL
DO
superDIDs: LIST OF DID ← NIL;
success: BOOL ← FALSE;
IF parentContainers.first.done THEN LOOP; -- already done
transID ← YggTransaction.CreateTrans[YggEnvironment.nullTransID];
containerDoc ← YggDIDMap.OpenDocumentFromDID[parentContainers.first.did, transID];
lowi.first.maxTimeStamp ← YggRep.MaxTimeStamp[lowi.first.maxTimeStamp, containerDoc.timeStampForIndexMaint];
[superDIDs, success] ← YggNav.GetSuperContainers[transID, parentContainers.first.did, TRUE];
IF ~success
THEN {
outcome ← YggTransaction.Finish[transID: transID, requestedOutcome: abort];
keepTrying ← FALSE;
EXIT;
};
FOR aL:
LIST
OF YggRep.Attribute ← vDoc.attributes, aL.rest
UNTIL aL =
NIL
DO
attributeName: ROPE ← aL.first.attributeName;
[keepTrying: keepTrying, outcome: outcome, didSomething: didSomething] ← doOneChange [
transID: transID,
doc: containerDoc,
indexFile: NIL,
attributeName: attributeName,
atomForIndex: Atom.MakeAtom[Rope.Concat[
YggFixedNames.IndexPrefix, attributeName]],
containerDID: containerDID,
indexedDID: vDoc.did,
oldValues: IF mam.first.add THEN NIL ELSE LIST[aL.first.value],
newValues: IF mam.first.add THEN LIST[aL.first.value] ELSE NIL];
IF ~keepTrying THEN EXIT;
IF didSomething
THEN {
outcome ← YggTransaction.Finish[transID: transID, requestedOutcome: commit];
IF outcome = commit
THEN {
somethingChanged ← TRUE;
IF aL.rest # NIL THEN transID ← YggTransaction.CreateTrans[YggEnvironment.nullTransID];
}
ELSE {
keepTrying ← FALSE;
EXIT;
};
};
REPEAT
FINISHED => {
-- container processed OK !!!
parentContainers.first.done ← TRUE;
IF superDIDs # NIL THEN addSuperDIDs[superDIDs, lowi]; -- append supers to list of stuff to do, provided that they have not yet been seen
};
ENDLOOP;
outcome ← YggTransaction.Finish[transID: transID, requestedOutcome: abort];
IF ~keepTrying THEN EXIT;
REPEAT
FINISHED => {
-- all parents processed OK !!!
ok: BOOL ← TRUE;
ok ← checkTimeStamps[lowi, lowi.first.maxTimeStamp];
IF ok
THEN {
doRemoveEntry ← TRUE;
somethingChanged ← TRUE;
keepTrying ← FALSE;
}
ELSE {
-- something changed; force re-evaluation of my super contatiners and redo loop
FOR parentContainers:
LIST
OF ContainerItem ← lowi.first.containersToApply, parentContainers.rest
UNTIL parentContainers =
NIL
DO
parentContainers.first.done ← FALSE;
ENDLOOP;
};
};
ENDLOOP;
ENDLOOP;
};
Rope.Equal[YggFixedNames.ParentContainers, mam.first.attributeName] => {
add/remove DID for WorkItem as subcontainer to this parent: do nothing; the next clause will handle it.
doRemoveEntry ← TRUE;
somethingChanged ← TRUE;
};
Rope.Equal[YggFixedNames.ChildContainers, mam.first.attributeName] => {
add/remove DID as subcontainer to this parent (in WorkItem)
1) look up all super containers that have to be indexed
2) build a list of attribute patterns to index
3) scan container and all its children containers; for each attribute that is indexed remember the did of the object
4) for each super container, index matched attributes
5) check to be sure nothing changed in the super container DG; on change GO TO 1
keepTrying: BOOL ← TRUE;
didSomething: BOOL ← TRUE;
outcome: YggTransaction.Outcome;
containerDID: DID;
transID: YggEnvironment.TransID;
containerDoc: Document;
1) look up all super containers that have to be indexed
2) build a list of attribute patterns to index
containerDID ← mam.first.didValue;
WHILE keepTrying
DO
IF lowi.first.containersToApply =
NIL
THEN {
lastContainersToApply: LIST OF ContainerItem;
lastContainersToApply ← lowi.first.containersToApply ← LIST[[containerDID, FALSE]];
transID ← YggTransaction.CreateTrans[YggEnvironment.nullTransID];
FOR cta:
LIST
OF ContainerItem ← lowi.first.containersToApply, cta.rest
UNTIL cta =
NIL
DO
superDIDs: LIST OF DID ← NIL;
success: BOOL ← FALSE;
patterns: LIST OF ROPE;
containerDoc ← YggDIDMap.OpenDocumentFromDID[cta.first.did, transID];
lowi.first.maxTimeStamp ← YggRep.MaxTimeStamp[lowi.first.maxTimeStamp, containerDoc.timeStampForIndexMaint];
[superDIDs, success] ← YggNav.GetSuperContainers[transID, cta.first.did, TRUE];
IF ~success
THEN {
outcome ← YggTransaction.Finish[transID: transID, requestedOutcome: abort];
keepTrying ← FALSE;
EXIT;
};
FOR dids:
LIST
OF
DID ← superDIDs, dids.rest
UNTIL dids =
NIL
DO
FOR knownSupers:
LIST
OF ContainerItem ← lowi.first.containersToApply, knownSupers.rest
UNTIL knownSupers =
NIL
DO
IF YggDID.EqualDIDs[knownSupers.first.did, dids.first] THEN EXIT;
REPEAT
FINISHED => {
lastContainersToApply ← CONS[[dids.first, FALSE], lastContainersToApply];
};
ENDLOOP;
ENDLOOP;
[patterns, success] ← YggNav.GetPatternsInContainer[cta.first.did, TRUE];
IF ~success
THEN {
keepTrying ← FALSE;
EXIT;
};
cta.first.attributeNamePattern ← patterns;
FOR lop:
LIST
OF
ROPE ← patterns, lop.rest
UNTIL lop =
NIL
DO
FOR loapd:
LIST
OF AttrPatDID ← lowi.first.attributePatternDIDs, loapd.rest
UNTIL loapd =
NIL
DO
IF Rope.Equal[loapd.first.attributeNamePattern, lop.first] THEN EXIT;
REPEAT
FINISHED => {
lowi.first.attributePatternDIDs ← CONS[[loapd.first.attributeNamePattern, NIL], lowi.first.attributePatternDIDs];
};
ENDLOOP;
ENDLOOP;
ENDLOOP;
IF ~ keepTrying
THEN {
lowi.first.containersToApply ← NIL;
RETURN [FALSE, FALSE]; -- try again later
};
outcome ← YggTransaction.Finish[transID: transID, requestedOutcome: commit];
};
3) scan container and all its children containers; for each attribute that is indexed remember the did of the object
IF ~keepTrying THEN EXIT; -- should be redundant
IF lowi.first.containersToIndex = NIL THEN lowi.first.containersToIndex ← LIST[[lowi.first.did, FALSE]];
{
FOR cta:
LIST
OF ContainerItem ← lowi.first.containersToIndex, cta.rest
UNTIL cta =
NIL
OR ~keepTrying
DO
subDIDs: LIST OF DID ← NIL;
didsInContainer: LIST OF DID ← NIL;
success: BOOL ← FALSE;
IF cta.first.done THEN LOOP;
transID ← YggTransaction.CreateTrans[YggEnvironment.nullTransID];
containerDoc ← YggDIDMap.OpenDocumentFromDID[cta.first.did, transID];
[subDIDs, success] ← YggNav.GetContainersInContainer[transID, cta.first.did, TRUE];
IF ~success
THEN {
outcome ← YggTransaction.Finish[transID: transID, requestedOutcome: abort];
keepTrying ← FALSE;
EXIT;
};
FOR dids:
LIST
OF
DID ← subDIDs, dids.rest
UNTIL dids =
NIL
DO
lastKnownSubs: LIST OF ContainerItem ← NIL;
FOR knownSubs:
LIST
OF ContainerItem ← lowi.first.containersToIndex, knownSubs.rest
UNTIL knownSubs =
NIL
DO
lastKnownSubs ← knownSubs;
IF YggDID.EqualDIDs[knownSubs.first.did, dids.first] THEN EXIT;
REPEAT
FINISHED => {
lastKnownSubs ← CONS[[dids.first, FALSE], lastKnownSubs];
};
ENDLOOP;
ENDLOOP;
[dids: didsInContainer, success: success] ← YggNav.GetObjectsInContainer[trans: transID, containerDID: cta.first.did, dontWait: TRUE];
IF ~success
THEN {
outcome ← YggTransaction.Finish[transID: transID, requestedOutcome: abort];
keepTrying ← FALSE;
RETURN;
};
outcome ← YggTransaction.Finish[transID: transID, requestedOutcome: commit];
FOR nowDID:
LIST
OF
DID ← didsInContainer, nowDID.rest
UNTIL nowDID =
NIL
DO
skipDID: BOOL ← FALSE;
addedValueForDID: BOOL ← FALSE;
properties: LIST OF YggRep.Attribute ← NIL;
transID ← YggTransaction.CreateTrans[YggEnvironment.nullTransID];
[properties: properties] ← YggNav.GetAllProperties[trans: transID, did: nowDID.first];
outcome ← YggTransaction.Finish[transID: transID, requestedOutcome: commit];
FOR loa:
LIST
OF YggRep.Attribute ← properties, loa.rest
UNTIL loa =
NIL
OR skipDID
DO
-- for each attribute
FOR apd:
LIST
OF AttrPatDID ← lowi.first.attributePatternDIDs, apd.rest
UNTIL apd =
NIL
OR skipDID
DO
IF Rope.Match[apd.first.attributeNamePattern, loa.first.attributeName]
THEN {
-- if the attribute should be indexed
FOR apdItem:
LIST
OF AttrDIDVal ← apd.first.attributes, apdItem.rest
UNTIL apdItem =
NIL
OR skipDID
DO
-- look for the attribute name under the pattern
IF Rope.Equal[apdItem.first.attributeName, loa.first.attributeName]
THEN {
FOR didOnList:
LIST
OF DIDVal ← apdItem.first.didsAndValues, didOnList.rest
UNTIL didOnList =
NIL
DO
IF YggDID.EqualDIDs[didOnList.first.did, nowDID.first]
THEN {
IF addedValueForDID
THEN {
didOnList.first.values ← CONS[loa.first.value, didOnList.first.values];
}
ELSE {
skipDID ← TRUE;
EXIT;
};
};
REPEAT
FINISHED => {
addedValueForDID ← TRUE;
apdItem.first.didsAndValues ← CONS[[nowDID.first, LIST[loa.first.value]], apdItem.first.didsAndValues];
};
ENDLOOP;
EXIT;
};
REPEAT
FINISHED =>
IF ~skipDID
THEN {
-- attribute name not listed under pattern
addedValueForDID ← TRUE;
apd.first.attributes ← CONS[[loa.first.attributeName, LIST[[nowDID.first, LIST[loa.first.value]]]], apd.first.attributes];
};
ENDLOOP;
};
ENDLOOP;
ENDLOOP;
ENDLOOP;
cta.first.done ← TRUE;
ENDLOOP;
};
4) for each super container, index matched attributes
IF ~ keepTrying THEN EXIT;
{
FOR cta:
LIST
OF ContainerItem ← lowi.first.containersToApply, cta.rest
UNTIL cta =
NIL
DO
IF cta.first.done THEN LOOP;
FOR anp:
LIST
OF
ROPE ← cta.first.attributeNamePattern, anp.rest
UNTIL anp =
NIL
DO
FOR apd:
LIST
OF AttrPatDID ← lowi.first.attributePatternDIDs, apd.rest
UNTIL apd =
NIL
DO
IF Rope.Equal[apd.first.attributeNamePattern, anp.first]
THEN {
FOR apdItem:
LIST
OF AttrDIDVal ← apd.first.attributes, apdItem.rest
UNTIL apdItem =
NIL
DO
indexFile: YggInternal.FileHandle;
containerDoc: Document;
atomForIndex: ATOM;
componentFiles: LIST OF YggInternal.FileHandle;
transID ← YggTransaction.CreateTrans[YggEnvironment.nullTransID];
containerDoc ← YggDIDMap.OpenDocumentFromDID[cta.first.did, transID];
componentFiles ← YggDIDMap.GetComponentFiles[containerDoc];
atomForIndex ← Atom.MakeAtom[Rope.Concat[YggFixedNames.IndexPrefix, apdItem.first.attributeName]];
indexFile ← YggFile.FileFromComponentFiles[componentFiles: componentFiles, fileUse: atomForIndex];
FOR didsToIndex:
LIST
OF DIDVal ← apdItem.first.didsAndValues, didsToIndex.rest
UNTIL didsToIndex =
NIL
DO
[keepTrying: keepTrying, outcome: outcome, didSomething: didSomething] ← doOneChange [
transID: transID,
doc: containerDoc,
indexFile: indexFile,
attributeName: apdItem.first.attributeName,
atomForIndex: atomForIndex,
containerDID: cta.first.did,
indexedDID: didsToIndex.first.did,
oldValues: IF mam.first.add THEN NIL ELSE didsToIndex.first.values,
newValues: IF mam.first.add THEN didsToIndex.first.values ELSE NIL];
IF ~keepTrying THEN EXIT;
IF didSomething
AND indexFile =
NIL
THEN {
indexFile ← YggFile.FileFromComponentFiles[componentFiles: componentFiles, fileUse: atomForIndex];
};
ENDLOOP;
REPEAT
FINISHED => {
-- container processed OK !!!
cta.first.done ← TRUE;
};
ENDLOOP;
EXIT;
};
REPEAT FINISHED => ERROR;
ENDLOOP;
ENDLOOP;
ENDLOOP;
};
5) check to be sure nothing changed in the super container DG; on change GO TO 1
IF ~ keepTrying THEN EXIT;
{
ok: BOOL ← TRUE;
ok ← checkTimeStamps[lowi, lowi.first.maxTimeStamp];
IF ok THEN
FOR indexContainers:
LIST
OF ContainerItem ← lowi.first.containersToIndex, indexContainers.rest
UNTIL indexContainers =
NIL
DO
doc: Document;
doc ← YggDIDMap.OpenDocumentFromDID[indexContainers.first.did, YggEnvironment.nullTransID];
IF YggRep.CompareTimeStamps[doc.timeStampForIndexMaint, lowi.first.maxTimeStamp] = greater
THEN {
ok ← FALSE;
EXIT;
};
ENDLOOP;
IF ok
THEN {
doRemoveEntry ← TRUE;
somethingChanged ← TRUE;
keepTrying ← FALSE;
}
ELSE {
-- something changed; force re-evaluation of my super contatiners and index containter and redo loop (shucks)
lowi.first.containersToApply ← NIL;
lowi.first.containersToIndex ← NIL;
};
};
ENDLOOP;
};
ENDCASE => ERROR;
ENDLOOP;
};
doCreateIndexChange:
PROC [lowi:
LIST
OF WorkItem, vDoc: YggRep.VDoc]
RETURNS [doRemoveEntry:
BOOL ←
FALSE, somethingChanged:
BOOL ←
FALSE] ~ {
Follow this algorithm:
1) build up list of containers to index
2) build up list of attributes/dids
3) apply to index
It should not be necessary to check time stamps: any changes will be handled by other update
4) check container time stamps
5) if a problem, go to 1
keepTrying: BOOL ← TRUE;
didSomething: BOOL ← TRUE;
outcome: YggTransaction.Outcome;
containerDID: DID ← lowi.first.did;
transID: YggEnvironment.TransID;
containerDoc: Document;
newPatternToIndex: ROPE ← lowi.first.attributeOrPattern;
WHILE keepTrying
DO
success: BOOL ← FALSE;
fileUses: LIST OF ATOM ← NIL;
IF lowi.first.containersToIndex = NIL THEN lowi.first.containersToIndex ← LIST[[lowi.first.did, FALSE]];
IF lowi.first.attributePatternDIDs = NIL THEN lowi.first.attributePatternDIDs ← LIST[[newPatternToIndex, NIL]];
FOR cti:
LIST
OF ContainerItem ← lowi.first.containersToIndex, cti.rest
UNTIL cti =
NIL
OR ~keepTrying
DO
subDIDs: LIST OF DID ← NIL;
didsInContainer: LIST OF DID ← NIL;
success: BOOL ← FALSE;
IF cti.first.done THEN LOOP;
transID ← YggTransaction.CreateTrans[YggEnvironment.nullTransID];
containerDoc ← YggDIDMap.OpenDocumentFromDID[cti.first.did, transID];
1) build up list of containers to index
[subDIDs, success] ← YggNav.GetContainersInContainer[transID, cti.first.did, TRUE];
IF ~success
THEN {
outcome ← YggTransaction.Finish[transID: transID, requestedOutcome: abort];
keepTrying ← FALSE;
EXIT;
};
FOR dids:
LIST
OF
DID ← subDIDs, dids.rest
UNTIL dids =
NIL
DO
lastKnownSubs: LIST OF ContainerItem ← NIL;
FOR knownSubs:
LIST
OF ContainerItem ← lowi.first.containersToIndex, knownSubs.rest
UNTIL knownSubs =
NIL
DO
lastKnownSubs ← knownSubs;
IF YggDID.EqualDIDs[knownSubs.first.did, dids.first] THEN EXIT;
REPEAT
FINISHED => {
lastKnownSubs ← CONS[[dids.first, FALSE], lastKnownSubs];
};
ENDLOOP;
ENDLOOP;
2) build up list of attributes/dids
[dids: didsInContainer, success: success] ← YggNav.GetObjectsInContainer[trans: transID, containerDID: cti.first.did, dontWait: TRUE];
IF ~success
THEN {
outcome ← YggTransaction.Finish[transID: transID, requestedOutcome: abort];
keepTrying ← FALSE;
RETURN;
};
outcome ← YggTransaction.Finish[transID: transID, requestedOutcome: commit];
FOR nowDID:
LIST
OF
DID ← didsInContainer, nowDID.rest
UNTIL nowDID =
NIL
DO
skipDID: BOOL ← FALSE;
addedValueForDID: BOOL ← FALSE;
properties: LIST OF YggRep.Attribute ← NIL;
transID ← YggTransaction.CreateTrans[YggEnvironment.nullTransID];
[properties: properties] ← YggNav.GetAllProperties[trans: transID, did: nowDID.first];
outcome ← YggTransaction.Finish[transID: transID, requestedOutcome: commit];
FOR loa:
LIST
OF YggRep.Attribute ← properties, loa.rest
UNTIL loa =
NIL
OR skipDID
DO
-- for each attribute
IF Rope.Match[loa.first.attributeName, newPatternToIndex]
THEN {
FOR apdItem:
LIST
OF AttrDIDVal ← lowi.first.attributePatternDIDs.first.attributes, apdItem.rest
UNTIL apdItem =
NIL
OR skipDID
DO
-- look for the attribute name under the pattern
IF Rope.Equal[apdItem.first.attributeName, loa.first.attributeName]
THEN {
FOR didOnList:
LIST
OF DIDVal ← apdItem.first.didsAndValues, didOnList.rest
UNTIL didOnList =
NIL
DO
IF YggDID.EqualDIDs[didOnList.first.did, nowDID.first]
THEN {
IF addedValueForDID
THEN {
didOnList.first.values ← CONS[loa.first.value, didOnList.first.values];
}
ELSE {
skipDID ← TRUE;
EXIT;
};
};
REPEAT
FINISHED => {
addedValueForDID ← TRUE;
apdItem.first.didsAndValues ← CONS[[nowDID.first, LIST[loa.first.value]], apdItem.first.didsAndValues];
};
ENDLOOP;
EXIT;
};
REPEAT
FINISHED =>
IF ~skipDID
THEN {
-- attribute name not listed under pattern
addedValueForDID ← TRUE;
lowi.first.attributePatternDIDs.first.attributes ← CONS[[loa.first.attributeName, LIST[[nowDID.first, LIST[loa.first.value]]]], lowi.first.attributePatternDIDs.first.attributes];
};
ENDLOOP;
};
ENDLOOP;
ENDLOOP;
cti.first.done ← TRUE;
ENDLOOP;
3) apply to index
IF ~ keepTrying THEN EXIT;
{
FOR apdItem:
LIST
OF AttrDIDVal ← lowi.first.attributePatternDIDs.first.attributes, apdItem.rest
UNTIL apdItem =
NIL
DO
indexFile: YggInternal.FileHandle;
containerDoc: Document;
atomForIndex: ATOM;
componentFiles: LIST OF YggInternal.FileHandle;
transID ← YggTransaction.CreateTrans[YggEnvironment.nullTransID];
containerDoc ← YggDIDMap.OpenDocumentFromDID[lowi.first.did, transID];
componentFiles ← YggDIDMap.GetComponentFiles[containerDoc];
atomForIndex ← Atom.MakeAtom[Rope.Concat[YggFixedNames.IndexPrefix, apdItem.first.attributeName]];
indexFile ← YggFile.FileFromComponentFiles[componentFiles: componentFiles, fileUse: atomForIndex];
FOR didsToIndex:
LIST
OF DIDVal ← apdItem.first.didsAndValues, didsToIndex.rest
UNTIL didsToIndex =
NIL
DO
[keepTrying: keepTrying, outcome: outcome, didSomething: didSomething] ← doOneChange [
transID: transID,
doc: containerDoc,
indexFile: indexFile,
attributeName: apdItem.first.attributeName,
atomForIndex: atomForIndex,
containerDID: lowi.first.did,
indexedDID: didsToIndex.first.did,
oldValues: NIL,
newValues: didsToIndex.first.values];
IF ~keepTrying THEN EXIT;
IF didSomething
AND indexFile =
NIL
THEN {
indexFile ← YggFile.FileFromComponentFiles[componentFiles: componentFiles, fileUse: atomForIndex];
};
ENDLOOP;
REPEAT
FINISHED => {
-- container processed OK !!!
};
ENDLOOP;
};
IF ~ keepTrying THEN EXIT;
ENDLOOP;
};
doDeleteIndexChange:
PROC [lowi:
LIST
OF WorkItem, vDoc: YggRep.VDoc]
RETURNS [doRemoveEntry:
BOOL ←
FALSE, somethingChanged:
BOOL ←
FALSE] ~ {
keepTrying: BOOL ← TRUE;
didSomething: BOOL ← TRUE;
outcome: YggTransaction.Outcome;
containerDID: DID ← lowi.first.did;
componentFiles: LIST OF YggInternal.FileHandle;
transID: YggEnvironment.TransID;
containerDoc: Document;
WHILE keepTrying
DO
success: BOOL ← FALSE;
patterns: LIST OF ROPE;
fileUses: LIST OF ATOM ← NIL;
transID ← YggTransaction.CreateTrans[YggEnvironment.nullTransID];
[patterns, success] ← YggNav.GetPatternsInContainer[containerDID, TRUE];
IF ~success
THEN {
outcome ← YggTransaction.Finish[transID: transID, requestedOutcome: abort];
keepTrying ← FALSE;
RETURN;
};
containerDoc ← YggDIDMap.OpenDocumentFromDID[containerDID, transID];
componentFiles ← YggDIDMap.GetComponentFiles[containerDoc];
fileUses ← YggFile.FileUseFromComponentFiles[componentFiles];
FOR locf:
LIST
OF
ATOM ← fileUses, locf.rest
UNTIL locf =
NIL
DO
useName: ROPE ← Atom.GetPName[locf.first];
matchName: ROPE ← NIL;
IF Rope.Equal[YggFixedNames.IndexPrefix, Rope.Substr[useName, 0, YggFixedNames.IndexPrefixSize]]
THEN matchName ← Rope.Substr[useName, YggFixedNames.IndexPrefixSize]
ELSE LOOP;
FOR lop:
LIST
OF
ROPE ← patterns, lop.rest
UNTIL lop =
NIL
DO
IF Rope.Match[pattern: lop.first, object: matchName] THEN EXIT;
REPEAT
FINISHED => {
indexFile: YggInternal.FileHandle ← NIL;
indexFile ← YggFile.FileFromComponentFiles[componentFiles: componentFiles, fileUse: locf.first];
YggDIDMap.RemoveComponentFile[containerDoc, indexFile, transID];
YggFile.Delete[file: indexFile, tid: transID];
};
ENDLOOP;
ENDLOOP;
outcome ← YggTransaction.Finish[transID: transID, requestedOutcome: abort];
IF outcome = commit
THEN {
somethingChanged ← TRUE;
doRemoveEntry←TRUE;
keepTrying ← FALSE;
};
ENDLOOP;
};
doOneChange:
PROC [transID: YggEnvironment.TransID, doc: Document, indexFile: YggInternal.FileHandle, attributeName:
ROPE, atomForIndex:
ATOM, containerDID:
DID, indexedDID:
DID, oldValues:
LIST
OF
LIST
OF YggRep.AttributeValue, newValues:
LIST
OF
LIST
OF YggRep.AttributeValue]
RETURNS [keepTrying:
BOOL ←
TRUE, outcome: YggTransaction.Outcome ← active, didSomething:
BOOL ←
FALSE] ~ {
Using the supplied transID, modify an index (if needed) at the specified doc/indexFile/containerDID. The index to be changed is specified by attributeName/atomForIndex. Valued to be removed are in oldValues, while new values to index are in newValues. The object to be indexed has a did of indexedDID. Transaction failures are reported in outcome with the keepTrying set to FALSE.
IF indexFile =
NIL
THEN {
-- no index yet, but should there be?
success: BOOL ← FALSE;
patterns: LIST OF ROPE;
shouldBeIndexed: BOOL ← FALSE;
[patterns, success] ← YggNav.GetPatternsInContainer[containerDID, TRUE];
IF ~success
THEN {
outcome ← YggTransaction.Finish[transID: transID, requestedOutcome: abort];
keepTrying ← FALSE;
RETURN;
};
FOR lor:
LIST
OF
ROPE ← patterns, lor.rest
UNTIL lor=
NIL
DO
IF Rope.Match[pattern: lor.first, object: attributeName]
THEN {
shouldBeIndexed ← TRUE;
RETURN;
};
ENDLOOP;
IF shouldBeIndexed
THEN {
componentFiles: LIST OF YggInternal.FileHandle;
componentFiles ← YggDIDMap.GetComponentFiles[doc];
indexFile ← YggFile.FileFromComponentFiles[componentFiles: componentFiles, fileUse: atomForIndex];
IF indexFile =
NIL
THEN {
indexFile ← YggFile.Create[size: 32, did: containerDID, fileUse: atomForIndex, nearToDid: containerDID, tid: transID];
YggDIDMap.AddComponentFile[doc, indexFile, transID];
};
};
};
IF indexFile #
NIL
THEN {
index: YggIndex.Index;
index ← YggIndex.Open[did: containerDID, fileUse: atomForIndex, cacheSize: 8, initialize:
FALSE];
clean out old stuff
FOR loloav:
LIST
OF
LIST
OF YggRep.AttributeValue ←
oldValues, loloav.rest
UNTIL loloav =
NIL
DO
FOR listoav:
LIST
OF YggRep.AttributeValue ← loloav.first, listoav.rest
UNTIL listoav =
NIL
DO
FOR valueSet:
LIST
OF YggRep.TypedPrimitiveElement ← listoav.first.valueSet, valueSet.rest
UNTIL valueSet =
NIL
DO
enumProc: YggIndex.EntryProc ~ {
PROC [value: YggRep.TypedPrimitiveElement, did: YggDID.DID] RETURNS [continue: BOOL];
tpesToDelete ← CONS[value, tpesToDelete]
};
tpesToDelete: LIST OF YggRep.TypedPrimitiveElement ← NIL;
YggIndex.EnumerateEntries[index: index, start: valueSet.first, end: valueSet.first, proc: enumProc];
FOR tpes:
LIST
OF YggRep.TypedPrimitiveElement ← tpesToDelete, tpes.rest
UNTIL tpes =
NIL
DO
didSomething ← TRUE;
[] ← YggIndex.DeleteEntry[index: index, value: tpes.first, did: indexedDID, trans: transID];
ENDLOOP;
ENDLOOP;
ENDLOOP;
ENDLOOP;
inset new stuff
FOR loloav:
LIST
OF
LIST
OF YggRep.AttributeValue ←
newValues, loloav.rest
UNTIL loloav =
NIL
DO
FOR listoav:
LIST
OF YggRep.AttributeValue ← loloav.first, listoav.rest
UNTIL listoav =
NIL
DO
FOR valueSet:
LIST
OF YggRep.TypedPrimitiveElement ← listoav.first.valueSet, valueSet.rest
UNTIL valueSet =
NIL
DO
didSomething ← TRUE;
[] ← YggIndex.WriteEntry[index: index, value: valueSet.first, did: indexedDID, trans: transID];
ENDLOOP;
ENDLOOP;
ENDLOOP;
};
};
code before it was procedure-ized
IF indexFile = NIL THEN { -- no index yet, but should there be?
patterns: LIST OF ROPE;
shouldBeIndexed: BOOL ← FALSE;
[patterns, success] ← YggNav.GetPatternsInContainer[parentContainers.first.did, TRUE];
IF ~success THEN {
outcome ← YggTransaction.Finish[transID: transID, requestedOutcome: abort];
keepTrying ← FALSE;
EXIT;
};
FOR lor: LIST OF ROPE ← patterns, lor.rest UNTIL lor= NIL DO
IF Rope.Match[pattern: lor.first, object: lowi.first.attributeOrPattern] THEN {
shouldBeIndexed ← TRUE;
EXIT;
};
ENDLOOP;
IF shouldBeIndexed THEN {
indexFile ← YggFile.Create[size: 32, did: parentContainers.first.did, fileUse: atomForIndex, nearToDid: parentContainers.first.did, tid: transID];
YggDIDMap.AddComponentFile[doc, indexFile, transID];
};
};
IF indexFile # NIL THEN {
index: YggIndex.Index;
index ← YggIndex.Open[did: parentContainers.first.did, fileUse: atomForIndex, cacheSize: 8, initialize: FALSE];
clean out old stuff
FOR loloav: LIST OF LIST OF YggRep.AttributeValue ← lowi.first.oldValues, loloav.rest UNTIL loloav = NIL DO
FOR listoav: LIST OF YggRep.AttributeValue ← loloav.first, listoav.rest UNTIL listoav = NIL DO
FOR valueSet: LIST OF YggRep.TypedPrimitiveElement ← listoav.first.valueSet, valueSet.rest UNTIL valueSet = NIL DO
enumProc: YggIndex.EntryProc ~ {
PROC [value: YggRep.TypedPrimitiveElement, did: YggDID.DID] RETURNS [continue: BOOL];
tpesToDelete ← CONS[value, tpesToDelete]
};
tpesToDelete: LIST OF YggRep.TypedPrimitiveElement ← NIL;
YggIndex.EnumerateEntries[index: index, start: valueSet.first, end: valueSet.first, proc: enumProc];
FOR tpes: LIST OF YggRep.TypedPrimitiveElement ← tpesToDelete, tpes.rest UNTIL tpes = NIL DO
[] ← YggIndex.DeleteEntry[index: index, value: tpes.first, did: lowi.first.did];
ENDLOOP;
ENDLOOP;
ENDLOOP;
ENDLOOP;
inset new stuff
FOR loloav: LIST OF LIST OF YggRep.AttributeValue ← lowi.first.newValues, loloav.rest UNTIL loloav = NIL DO
FOR listoav: LIST OF YggRep.AttributeValue ← loloav.first, listoav.rest UNTIL listoav = NIL DO
FOR valueSet: LIST OF YggRep.TypedPrimitiveElement ← listoav.first.valueSet, valueSet.rest UNTIL valueSet = NIL DO
YggIndex.WriteEntry[index: index, value: valueSet.first, did: lowi.first.did];
ENDLOOP;
ENDLOOP;
ENDLOOP;
};
getContainersToApply:
PROC [seedVDoc: YggRep.VDoc, lowi:
LIST
OF WorkItem, attrToFollow:
ROPE ← YggFixedNames.Parents] ~ {
parents: LIST OF YggRep.AttributeValue ← NIL;
FOR loa:
LIST
OF YggRep.Attribute ← seedVDoc.metaAttributes, loa.rest
UNTIL loa =
NIL
DO
IF Rope.Equal[loa.first.attributeName, attrToFollow] THEN {parents ← loa.first.value; EXIT};
ENDLOOP;
IF parents #
NIL
THEN {
FOR parentListAsAttr:
LIST
OF YggRep.AttributeValue ← parents, parentListAsAttr.rest
UNTIL parentListAsAttr =
NIL
DO
FOR parentVS:
LIST
OF YggRep.TypedPrimitiveElement ← parentListAsAttr.first.valueSet, parentVS.rest
UNTIL parentVS =
NIL
DO
IF parentVS.first.docType # YggRep.did THEN LOOP; -- junk
lowi.first.containersToApply ← CONS[[NARROW[parentVS.first.bits, DID], FALSE], lowi.first.containersToApply];
ENDLOOP;
ENDLOOP;
};
};
checkTimeStamps:
PROC [lowi:
LIST
OF WorkItem, timeStamp: YggRep.TimeStamp] RETURNS [ok:
BOOL ←
TRUE] ~ {
doc: Document;
doc ← YggDIDMap.OpenDocumentFromDID[lowi.first.did, YggEnvironment.nullTransID];
IF YggRep.CompareTimeStamps[doc.timeStampForIndexMaint, timeStamp] = greater
THEN {
ok ← FALSE;
};
IF ok
THEN
FOR parentContainers:
LIST
OF ContainerItem ← lowi.first.containersToApply, parentContainers.rest
UNTIL parentContainers =
NIL
DO
doc: Document;
doc ← YggDIDMap.OpenDocumentFromDID[parentContainers.first.did, YggEnvironment.nullTransID];
IF YggRep.CompareTimeStamps[doc.timeStampForIndexMaint, timeStamp] = greater
THEN {
ok ← FALSE;
EXIT;
};
ENDLOOP;
};
addSuperDIDs:
PROC [superDIDs:
LIST
OF
DID ←
NIL, lowi:
LIST
OF WorkItem] ~ {
append supers to list of stuff to do, provided that they have not yet been seen
lastPC: LIST OF ContainerItem ← NIL;
FOR pC:
LIST
OF ContainerItem ← lowi.first.containersToApply, pC.rest
UNTIL pC =
NIL
DO
lastPC ← pC;
ENDLOOP;
FOR lod:
LIST
OF
DID ← superDIDs, lod.rest
UNTIL lod =
NIL
DO
FOR pC:
LIST
OF ContainerItem ← lowi.first.containersToApply, pC.rest
UNTIL pC =
NIL
DO
IF YggDID.EqualDIDs[pC.first.did, lod.first] THEN EXIT;
REPEAT
FINISHED => {
lastPC ← CONS[[lod.first, FALSE], lastPC];
};
ENDLOOP;
ENDLOOP;
};
red black procs
debumpContainerUse:
PROC [containerDID: YggDID.
DID] ~ {
val: REF;
val ← RedBlackTree.Lookup[self: containerLock, lookupKey: containerDID];
IF val #
NIL
THEN {
entry: containerLockEntry;
entry ← NARROW[val];
entry.users ← entry.users - 1;
IF entry.users = 0
THEN {
[] ← RedBlackTree.Delete[self: containerLock, deleteKey: containerDID];
numLockedContainers ← numLockedContainers - 1;
IF numLockedContainers < 0 THEN ERROR;
};
}
};
bumpContainerUse:
INTERNAL PROC [containerDID: YggDID.
DID] ~ {
val: REF;
val ← RedBlackTree.Lookup[self: containerLock, lookupKey: containerDID];
IF val #
NIL
THEN {
entry: containerLockEntry;
entry ← NARROW[val];
entry.users ← entry.users + 1;
}
ELSE {
numLockedContainers ← numLockedContainers + 1;
RedBlackTree.Insert[self: containerLock, insertKey: containerDID, dataToInsert: NEW[containerLockRep ← [containerDID, 1]]];
};
};
GetKeyProc: RedBlackTree.GetKey = {
PROC [data: UserData] RETURNS [Key]
entry: containerLockEntry ← NARROW[data];
RETURN[ entry.did ];
};
CompareProc: RedBlackTree.Compare = {
PROC [k: Key, data: UserData] RETURNS [Basics.Comparison]
entryData: containerLockEntry ← NARROW[data];
keyData: DID ← NARROW[k];
SELECT keyData.didHigh
FROM
> entryData.did.didHigh => RETURN [greater];
< entryData.did.didHigh => RETURN [less];
ENDCASE => {
SELECT keyData.didLow
FROM
> entryData.did.didLow => RETURN [greater];
< entryData.did.didLow => RETURN [less];
ENDCASE => RETURN [equal];
};
};
Initialization
NextTimeStamp:
ENTRY
PROC
RETURNS [cts: YggRep.TimeStamp] ← {
currentTimeStamp ← YggRep.NextTimeStamp[currentTimeStamp];
};
Initialization
containerLock ← RedBlackTree.Create[getKey: GetKeyProc, compare: CompareProc];
TRUSTED {
Uncomment the next two statements when ready
Process.Detach[FORK IndexMaintProcess];
DoingIndexMaint ← TRUE;
Process.SetTimeout[condition: @MyCondition, ticks: Process.MsecToTicks[50]];
};
[] ← NextTimeStamp[];
END.