Transformations performed
1. Variables & labels are canonicalized. This means that variables and labels are shared (one cannot change the location field of a variable without affecting the other uses of that variable).
2. Lambdas have stack models (IntCodeTwig.StackModel) associated with them. Those models are also marked to indicate which lambdas are actually catch phrases.
3. Procedures taking long argument records (more than IntCodeTarget.maxBitsArgumentRecord) are transformed to take a pointer to the arguments instead, and argument variables are transformed to have locations that are indirect off that pointer.
4. Procedures returning long return records (more than IntCodeTarget.maxBitsReturnRecord) are transformed to take an additional argument (the first) that is a pointer to the return record, and returns within those procedures are transformed to assign to the return record, then return nothing.
5. Calls to procedures taking long argument records (more than IntCodeTarget.maxBitsArgumentRecord) are transformed to put the arguments into a variable in the frame extension, and to pass the address of a that variable.
6. Calls to procedures returning long return records (more than IntCodeTarget.maxBitsReturnRecord) are transformed to pass in the address of the return record, which is allocated in the local frame extension.
7. Variable flags are set to reflect usage, assignment, addressing, and up-level referencing. See IntCodeDefs.VariableFlag for details.
8. Global variables are assigned globalVar locations. These are assigned linearly in the order of appearance in the variable list for the module node. Uses of global variables are transformed to use a global link, and the global link is created in procedures that use global variables. REF literals are not yet considered global variables.
9. Non-catch phrase nested procedures are transformed into procedures that take a final extra argument (the static link) which is their up-level access.
10. Local variables (including those in the frame extension) are allocated/freed on the basis of blocks. This is a simple approximation to enable reusing locations.
11. Uses of local variables in memory are transformed into accesses through the frame extension, which is a simple deref of the memoryLink variable.
12. Uplevel uses of variables are transformed to have locations that are indirect through the static link (which is in a variable with a localVar location). Uplevel uses of variables from non-catch phrases force the variables into the frame extension. Uplevel uses of variables from catch phrases use the static link as a pointer to the saved registers.
13. All procedures that allocate a frame extension are transformed to have UNWIND code to free up the extension on unexpected exit. Also, all such procedures must free up the extension before returning normally. The frame extension freeing is done as a dummy return value that returns 0 bits, but occurs as the last return value.
14. A catch phrase lambda looks like a nested procedure with four arguments (regsPtr, exception, retsPtr, argsPtr). This is verified, and the regsPtr is considered to be the static link.
15. RESUME, SIGNAL and ERROR applications are transformed to pass pointers to the argument and return records (which are in the frame extension).
16. Ensure that the stack depth will not exceed LAST[StackRange].
The resulting subset
The trees output by IntCodeTwigImpl should not have the following constructs.
1. Long argument or return records.
2. Stack frames that require more than the maximum number of register stack locations.
3. Stack frames that require more than 16 register variables.
4. Uplevel references to local or global variables (no upLevel or globalVar locations).