Page Numbers: Yes X: 527 Y: -.4 First Page: 11 Not-on-first-page
Margins: Top: 1.1" Bottom: 1" Binding: 5
Odd Heading: Not-on-first-page

2. SYSTEM FACILITIES

Even Heading: Not-on-first-page

IDL REFERENCE MANUAL

2. System Facilities

This chapter describes the primitives IDL provides for operating on labelled, rectangular aggregates. These include functions for accessing and modifying selected array components, the extension mechanism that determines how most functions handle arrays with too many dimensions, and facilities for printing arrays in a readable format.

2.1 Selection

Selection is the operation for accessing individual elements, whole subsections, or the labelling information of an array. It is effected with the function AT, which is associated with the CLISP operator @.

at[a;sltr] where a is an array and sltr is a selector describing which components of a are to be selected, returns that selection.

Selectors may take several forms depending on the information that is to be extracted. The various possibilities will be illustrated in the context of a variable A bound to the matrix

Another Random Matrix

Variable

SubjectSEXAGEVOTE

11242

23311

32283

41252

2.1.1 Elements

The simplest selector is a list of integers, one for each dimension of A, specifying the level to be selected on each dimension of A. The result of this selection will be the element that lies at the intersection of these levels. For example, since A is a [4,3]-array, one can access the element of A which is at level 3 of the first dimension and level 2 of the second dimension (the scalar 28) with the expression

(AT A ’(3 2)) or, using CLISP, A@’(3 2)

However, whole subarrays (rows, columns, planes, etc.) can be selected by using a vector, rather than a single integer, as the selector for a dimension. For example, one could select the first two elements of the first row of our [4,3]-array with

A@’(1 (1 2))

The list (1 2) is first converted into a vector as described in section 1.7. When applied as a selector, it will produce the 2-vector [1 24], the first element of which is A@’(1 1) and the second A@’(1 2). The size of such a multiple selection is the product of the number of elements selected on each dimension. An easy way to think of this process is to imagine the selected subscript values casting "shadows" down the rows and columns, etc. of the array. The result will be all elements that fall into exactly one "shadow" from every dimension.

The result of applying a vector selector to a dimension is formed by selecting the level corresponding to each of the vector’s elements and packing these selections together. The same principle allows higher-dimensional arrays to be used as selectors, but in this case the result may have more dimensions than the original! Thus, if the selector on some dimension is a matrix, each element of the matrix is considered separately as a level selector for that dimension. The result of all these selections is formed by fitting them together into a two-dimensional plane which appears in the output in place of the selected dimension of the original. Thus, for M the matrix

12
31

the expression

A@(LIST ’(1 2) M)

produces an object with a leading Subject dimension, but in place of the Variable dimension is a two-dimensional image of M:

Subject = 1

1 2

1124

22 1

Subject = 2
1 2

1331

21 3

Note that the column labels of A have been lost, because A’s second dimension does not appear directly in the result and labelling information is attached by dimension. Since M is providing the structure, it is M’s labels (in this case none) which are copied to the second and third dimensions of the result.

The list-structures in these examples are an easy way of typing in small selectors. However, each selector may be constructed by evaluating an expression that describes an arbitrary scalar or array computation. This technique can result in considerable economy of specification. For example, the vector generating function GENVEC (see section 4.4) produces vectors that are often useful as selectors:

A@(LIST (GENVEC 2 4) 3)

selects the second, third and fourth elements of the third column of A. (The Lisp function LIST is used instead of a quoted list so that the selector contains the value of the GENVEC expression, not the GENVEC expression itself.)

Two additional features in IDL’s selection mechanism provide even greater conciseness of specification. First, although a subarray that includes all levels on a given dimension can be selected by listing those levels explicitly, e.g.

A@’(1 (1 2 3))

to select all elements of the first row of A, the same effect may be achieved by selecting with the distinguished symbol ALL:

A@’(1 ALL)

Second, if the selector list has fewer items than the dimensionality of the array, the selectors for the leading dimensions are defaulted to ALL. Thus,

A@’(1) is equivalent to A@’(ALL 1)

These facilities make selection a much more general operation than simply a way of selecting subsets. For example, one useful special case is where the selector for one dimension is a permutation (i.e., a reordering without addition or deletion) of the complete set of levels for that dimension. The result of such a selection is the corresponding permutation of the array. For example,

A@’((4 3 2 1) ALL)

produces an image of A with the order of the rows reversed! Note also that entries within a multiple selector may be repeated or arbitrarily reordered. Thus,

A@’((2 2) ALL)

is a matrix, both rows of which are the second row of the matrix A.

The levels to be selected may also be specified symbolically, by means of the corresponding level labels. A label appearing as a selector for a dimension is interpreted as the corresponding level index on that dimension. Thus, the label SEX can be used instead of the integer 1 to select the first column of A:

A@’(ALL SEX)

Similarly, when list-structure is converted into an array selector, labels appearing in the list are treated as level labels on the corresponding dimension, so that

A@’(ALL (2 SEX))

would select A’s second and first columns. Note, however, that

A@’(SEX ALL)

could not be used to obtain the second row of A, because SEX is not a selector for A’s first dimension. If this expression were evaluated, an error would occur.

Formally, a selector for array elements is a list of n items, where n is less than or equal to the dimensionality of the array being selected from. Each of these items may be:

a.a single integer or atom label, indicating the level to be selected on that dimension. Note that this dimension will be dropped from the result. For instance,

A@’(1 ALL)

is a 3-vector, not a [1,3]-matrix as

A@’((1) ALL)

would be.

b.a list-structure of integers or atoms that are selector labels for the corresponding dimension, indicating the levels to be selected on this dimension. This will be converted into an array and treated as in (c).

c.an array, indicating that the corresponding dimension of A is to be represented in the result by a set of d dimensions, where d is the dimensionality of the array used as selector. The elements on those dimensions of the result will be the result of applying the elements of the selector array as a selector to the corresponding dimension of A, and forming the results into an image of the selector array.

d.the value ALL indicating all levels for that dimension. (ALL is bound to itself, so it may be used interchangeably in quoted and non-quoted contexts.)

If n is less than the dimensionality of the array, ALL is assumed for the leading dimensions.

The general principle governing element selection is that the result is an array whose shape is e1 !! e2 !! e3 !! ... eN where !! indicates concatenation and ei is the shape of the selector for the ith dimension. The behavior of the scalar and array selectors becomes clear when referred to this principle. The shape of a scalar is the empty vector, and when this is concatenated with other vectors it disappears. Thus, the result loses a dimension. The shape of an array is, in general, a vector of length equal to its dimensionality. When this is concatenated in the formula above, it lengthens the shape of the result, and thus increases the dimensionality.

2.1.2 Other array properties

In addition to being usable as selectors, the labels of an array are themselves accessible through selection. The appropriate selectors are constructed with the TITLE, LABEL and CODE functions. The selector produced by the TITLE function will access the title of an array, so that the value of

A@(TITLE)

is the string "Another Random Matrix".

The LABEL function is used to access the dimension and selector labels of an array. The label selected is determined by the number of arguments given to the function. A single argument to LABEL will select the corresponding dimension label. Thus,

A@(LABEL 1)

will return the atom Subject, the dimension label for the first dimension. In a similar fashion, one can select the selector labels by using LABEL with two arguments, a dimension and a level. Thus,

A@(LABEL 2 3)

would select the selector label for the third level of the second dimension, namely, VOTE.

The arguments to LABEL can be either integers or labels. The type of the last argument determines the result. If, as above, the last argument is an integer, then the corresponding label will be returned. If, on the other hand, the last argument is a label, the corresponding integer index is returned. For our example array above

A@(LABEL ’Subject)

would evaluate to one, the number of the dimension with label Subject. However,

A@(LABEL ’Subject 3)

would still evaluate to VOTE. The dimension label is used only to specify the dimension being accessed and does not affect the kind of value returned.

The value labels of an array are accessed in a similar manner using CODE. To illustrate how it works, let us assume that value labels are attached to the Variable dimension of A, and that the variable SEX is coded in the usual way. The matrix would thus print as

Another Random Matrix

Variable

SubjectSEXAGEVOTE

1MALE242

23311

3FEMALE283

4MALE252

The function CODE can be called with zero, one, or two arguments. If called with none, it returns a selector that specifies the index of the dimension to which value labels are attached, so that

A@(CODE)

would evaluate to two. If called with one argument, that argument is interpreted as a selector for the value labelled dimension, and the result will access the codebook for that selector. A codebook is represented as a list of (code value) two-element lists, so the expression

A@(CODE ’SEX)

would evaluate to the list ((1 MALE) (2 FEMALE)). If CODE is given two arguments, then the value of the resulting selector depends on whether its second argument is a numeric value or a label. If the argument is a label, then the selector will cause AT to scan the codebook indicated by its first argument, and return the value associated with that label. Thus, in our example,

A@(CODE ’SEX ’FEMALE)

would evaluate to two. If the second argument is numeric, then the result will be the label paired with that value. Thus,

A@(CODE ’SEX 2)

would evaluate to FEMALE.

If there is no title, label, index, or code corresponding to a selector for those properties, the selection will return NIL.

The value of the function AT is always a "virtual" object, a window onto the array from which the selection is made. This means that any change made to the underlying array (via the function ASSIGN described below) will be visible through the selection. The only way to break this connection is to use the Lisp function COPY, which forms a new object from the components visible through the selection.

2.2 Assignment

Assignment is the operation of changing one of the components of an array. Assignment is the only IDL function that causes a change in an existing value. All other functions produce new values and do not have any "side effects" on their arguments. Rebinding of a variable is carried out with the Lisp function SETQ:

(SETQ A B) or A ← B

causes the value of B to become the new value of A and the old value of A to be lost. Aggregate objects such as IDL arrays raise the possibility of making partial changes instead of wholesale replacements. For example, some of the elements in an array may be changed while the organization of the aggregate (the shape, element type, storage format, etc.) is preserved. Such selective modifications are done with the ASSIGN function:

assign[target;source] where target is a selection (i.e., the value of a call to AT), and source is a value to be stored into target.

For example, (ASSIGN A@’(1 3) 7.6) will store the value 7.6 into A’s [1,3] cell, assuming, of course, that A is a matrix (the length of the subscript list implies A has at least two dimensions) which has a [1,3] cell. If the element type of A is INTEGER, the floating value 7.6 will be rounded to 8 before storing.

Whole subarrays may be modified with one assignment. For example,

(ASSIGN A@’((1 2 3) ALL) 0)

will set the first three rows of the matrix A to zero. The right hand argument of the assignment need not be a scalar. Thus, if A and B are both [4,4]-arrays, one could assign the second row of B into the third column of A with

(ASSIGN A@’(ALL 3) B@’(2 ALL))

Such assignments are done serially in row-major order (i.e., both the source and the target are enumerated with the last subscript varying fastest). This can lead to counter-intuitive results when a segment of an array is assigned into an overlapping segment of itself.

The labels of an array can be changed by assigning into the result of a TITLE, LABEL, or CODE selection. For example,

(ASSIGN A@(TITLE) "New title")

replaces A’s title, leaving the rest of A unchanged. Most IDL functions compose titles for their resulting arrays that indicate the function and arguments that produced the array. The internal convention for abbreviating such titles is available to the user: ASSIGN accepts list-structure titles as being equivalent to function-argument strings. Thus,

(ASSIGN A@(TITLE) (LIST ’Percentages X))

has the same effect as

(ASSIGN A@(TITLE) (CONCAT "Percentages of " X@(TITLE)))

(The Lisp function CONCAT places its argument strings together end to end to form a new string.)

The only information obtainable with a label selector but not changeable simply by assigning into it is the index of a dimension or selector. Thus, LABEL selectors appearing on the left hand side of assignments are considered to specify the corresponding label, even if they would evaluate to integers elsewhere. For example,

(ASSIGN A@(LABEL ’Varaible) ’Variable) and
(ASSIGN A@(LABEL 2) ’Variable)

are equivalent methods of changing the (misspelt) dimension label Varaible, whereas

(ASSIGN A@(LABEL ’Varaible) 2)

(or any other number) is in error.

One can change the dimension to which value labels are to be attached by assigning a dimension specification (either integer or atom) into A@(CODE). This must be done before using other CODE assignments to set other value labels. If some other dimension is value labelled at the time this assignment is done, the old value labels will be removed. Codebooks and individual codes or value labels may be changed in the obvious way. For example,

(ASSIGN A@(CODE ’SEX) ’((1 MALE) (2 FEMALE)) )

attaches our familiar codebook for sex to the SEX selector of the value labelled dimension of A, removing any previous value labels for SEX.

Any label or code may be removed by simply assigning NIL to the appropriate selection. Value labels may be removed in larger chunks: assigning NIL as a codebook for a level on the value labelled dimension causes it to have no value labels; changing the value labelled dimension removes all existing value labels, as noted above.

2.3 Function extension

The extension mechanism, briefly introduced in section 1.6, is a system facility that intercepts array arguments of greater dimensionality than a function is programmed to process. Such arguments are decomposed into "slices" each of the dimensionality required by the function. The function is then applied to each slice in turn and the results from each application are assembled into a single array value.

Such a decomposition can take place in several different ways. A [2,3]-array, for example, may be regarded as a two by three array of scalars, as two 3-vectors, or as three 2-vectors. The way the arguments are decomposed affects the number of times the function is applied, the arguments that it receives for each application, and the shape and labels of the result. This section discusses the extension mechanism in some detail, outlining the factors that determine how an argument will be regarded. The extension of single argument functions is considered first, followed by multi-argument functions, the devices by which the user can control the decomposition, and finally, the facilities by which the user may embed the extension mechanism in functions that he himself writes.

2.3.1 Single-argument functions

The function INVERT is a good example of an extended single argument function. It expects a matrix argument and returns a new matrix which is the inverse of its argument. If it is given a three-dimensional array, that array will be split into matrices along its leading dimension, these will be inverted separately, and the results will be fitted back together again in an image of the original argument. In a sense, the leading the dimension is withheld (or kept) from the inversion operation, and INVERT only sees objects that it is prepared to handle. Suppose A is the [2,3,3]-array

49129036

547213256

573273567

The two levels on the first dimension are represented by the two panels, and the rows and columns in each panel correspond to A’s second and third dimension. If the expression

(INVERT A)

is evaluated, A will be broken into slices along its first dimension, corresponding to the panels above, each slice will be inverted, and the results pasted back together to form the array

-.020.019.006.011-.001-.000

.164-.006-.061-.000 .032-.003

-.033-.002.044-.000-.000.002

each panel of which is the inverse of the corresponding panel of A above.

The general rules for a single-argument function F expecting an array of e dimensions and given an array of g dimensions are as follows:

If g is less than or equal to e, then F is simply applied once to its argument, and the resulting value is the value of the (vacuous) extension.

Otherwise, g is greater than e, and the first g–e dimensions of the argument are considered to be in excess. The function will be invoked a number of times equal to the number of cells represented by the excess dimensions (i.e., the product of the numbers of levels for those dimensions). The argument seen on each invocation will be a slice of the argument formed by selecting all levels on the non-excess, trailing dimensions and one level for each of the leading dimensions.

The result of the extension is constructed from the values returned by the repeated calls on the extended function. These values are combined into an array whose leading dimensions have the extents and labels of the argument’s excess dimensions. The extents for the trailing dimensions come from the common shape of the objects produced by each call. Their labels are taken from the value returned on the last call. Thus the shape of the result is the first g–e elements of the shape of the argument, concatenated with the common shape of the values. The elements of the array are obtained by placing the values as trailing subsections in the larger array.

As mentioned in section 1.6, the extension will fail if the value shape for one of the argument slices differs from the shape for other slices. This requirement is necessary so that the partial results can be formed into an IDL array, which must be rectangular.

For the INVERT example, the original argument is a [2,3,3]-array and the function expects a 2-array, so the two-level first dimension is in excess. The function will be called twice, receiving X@’(1 ALL ALL) the first time and X@’(2 ALL ALL) the second time. If A were a [4,2,3,3]-array, there would be two excess dimensions and the inversion would be done eight (= 4 x 2) times. The slices would be A@’(1 1 ALL ALL), A@’(1 2 ALL ALL), A@’(2 1 ALL ALL), etc. The final value would be another [4,2,3,3]-array.

2.3.2 Multiple argument functions

If F has multiple arguments, then both the dimensionality of the array given and the dimensionality expected of it must be considered for each argument. Let gi be the dimensionality of the ith argument given to a function which expects its ith argument to have dimensionality ei. The case of vacuous extension is the same as before: If, for each i, gi is less than or equal to ei, then all the arguments are passed directly to the function, it is invoked once, and the single value is returned.

The situation is a little more complicated if there are excess dimensions but the number of them is the same for all arguments, i.e., gi–ei is the same positive number for all i. As in the single argument case, the leading dimensions are considered to be in excess. The left-most argument is taken to be the controlling argument of the extension, and to a certain degree the extension mechanism acts as if F were a function of only that argument. As above, the number of times that F is invoked is the product of the number of levels for the excess dimensions. It will receive a different slice of that argument on each invocation, and leading dimensions of the extension result will have the levels and labels of the controlling argument’s excess dimensions. The other arguments are sliced up in parallel with the controlling argument, so the function will receive a different slice of each argument on each invocation. As before, the value of each invocation must have the same shape so that the final result is rectangular. There is an additional conformability requirement that the number of levels for the leading dimensions be the same for all arguments; otherwise, the parallel slicing is ill-defined.

As an example, suppose F is a three argument function expecting 3-arrays, matrices, and vectors. If it is applied to a [3,2,4,5,6]-array A, a [3,2,2,2]-array B, and a [3,2,7] array C, the decomposition will proceed in the following way: The number of excess dimensions for each argument is 2 (= 5–3 = 4–2 = 3–1), and the excess dimension levels have the same [3,2] extents. F will therefore be applied six times, receiving different argument slices on each application. The first time it will be given

A@’(1 1 ALL ALL ALL)
B@’(1 1 ALL ALL)
C@’(1 1 ALL)

the second time

A@’(1 2 ALL ALL ALL)
B@’(1 2 ALL ALL)
C@’(1 2 ALL)

the third time

A@’(2 1 ALL ALL ALL)
B@’(2 1 ALL ALL)
C@’(2 1 ALL)

and so on. The labels of A will be used for the leading dimensions of the result, since it is left-most and thus the controlling argument. Note that the leading subscripts for the selections are enumerated in parallel. The extension would fail if, for example, B were a [4,2,2,2]-array, as there would be no slices of A and C corresponding to the [4,--] slices of B.

In the most general situation, the number of excess dimensions is not the same for all arguments. The one with the greatest number of excess dimensions is taken as the controlling argument (the left-most one if there is a tie), and again, it determines the number of times the extended function will be called, the decomposition of arguments for each call, and the shape and labels of the result. The other arguments are "expanded" so that their excesses match the controlling argument’s; the computation then proceeds as for the equal-excess case.

An argument is expanded by replication. Suppose that C in the example above were a [3,7]-matrix instead of a [3,2,7]-array. A is still the controlling argument, and it has a [3,2] excess. The extension will thus replicate C twice, treating it as the needed [3,2,7]-array. The same 7-vector will be given to F no matter what second-dimension subscript is involved in slicing A and B. Thus on one invocation F will receive A@’(1 1 ALL ALL ALL), B@’(1 1 ALL ALL), and C@’(1 ALL). That same slice of C will be given in the call involving A@’(1 2 ALL ALL ALL) and B@’(1 2 ALL ALL). The general requirement for conformability among arguments is that the left-most excess dimensions (which are themselves leading dimensions) must have the same extents for all arguments.

To take a concrete example, let A be the [4,3]-matrix introduced in section 2.1:

Another Random Matrix

Variable

SubjectSEXAGEVOTE

11242

23311

32283

41252

The function DIFFERENCE expects both of its arguments to be scalars (ei = 0) and returns their arithmetic difference. For the expression

(DIFFERENCE 50 A)

A is the controlling argument, since it has two excess dimensions and the scalar 50 has none. The scalar will be replicated to form the [4,3] array:

505050

505050

505050

505050

and the elementwise subtraction will be performed. The values are scalars with no dimensionality, so the result of the extension will simply be an image of A, complete with its labels:

Difference of 50 and
Another Random Matrix

Variable

Subject SEXAGE VOTE

1492648

2471949

3482247

4492548

The extension mechanism would also work if the first argument were a 4-vector:

(DIFFERENCE ’(2 4 6 8) A)

The vector has one excess dimension, so A is still the controlling argument. The vector is treated as the [4,3]-matrix

222

444

666

888

and the 12 subtractions are performed. The result, again taking labels from A, is

Difference of Array 15
and Another Random Matrix

Variable

SubjectSEX AGEVOTE

11–220

21–273

34–223

47–176

On the other hand, the extension would fail if the first argument were a 3-vector: even though one dimension of A does have extent three, it is not the dimension being aligned with the vector so the conformability requirement would not be satisfied.

2.3.3 Affecting the decomposition (KEEP and LEAVE)

Whereas function extension automatically handles the case of subtracting each column of A from the same 4-vector, subtracting each row of A from the same 3-vector is less conveniently specified with this mechanism. The dimensions of A must be re-ordered so that the rows and columns are reversed, giving a [3,4]-matrix which the normal extension will be able to handle. Dimension permutations are computed by the function TRANSPOSE, described in Chapter 4. Thus

(DIFFERENCE ’(1 3 5) (TRANSPOSE A))

produces the matrix

Difference of Array 17 and
Transpose of Another Random Matrix

Subject

Variable  1  2  3  4

SEX 𔁆 –2 –1 𔁆

AGE–21–28–25–22

VOTE 𔁉 𔁊 𔁈 𔁉

Note that the desired subtractions have been performed but the result is tipped on its side, reflecting the shape of the transposed A, not the original. TRANSPOSE must also be applied to the output of the extension in order to get a more intuitive result.

Transpositions are a general way of affecting argument conformance and decomposition when the default behavior of the extension mechanism is inappropriate. By rearranging the leading dimensions, they alter which dimensions will be considered to be in excess and how they will be aligned. They do not change the number of excess dimensions; that is still determined by the argument expectations of the particular function involved.

There are occasions, however, when the arguments to a function must be decomposed into slices smaller than the function expects. This happens because an expectation defines the maximum number of dimensions that the function can handle, but many functions will produce reasonable results from smaller objects. Those results will never be computed for an over-sized object unless a function’s permanent expectations can be overridden on a particular invocation.

As one example, the function ADJOIN, described in Chapter 4, produces vectors by joining its argument vectors together end to end:

(ADJOIN ’(1 2) ’(3 4))

is the 4-vector [1 2 3 4]. Though it expects vectors, ADJOIN accepts scalars and treats them as 1-vectors. This property provides great flexibility in the arrays that ADJOIN can produce. By the ordinary rules of extension, joining a 4-vector and the [4,3]-matrix A above gives a [4,7]-matrix with a copy of the vector appearing on the front of each row. Because the first dimension of A is the only one in excess, the 4-vector will be expanded to a [4,4]-matrix, and each call to ADJOIN will receive a row of that matrix and a row of A. Suppose that the extension mechanism could be told that on this invocation, ADJOIN’s first argument is expected to be a scalar, so that the first dimension of the vector must also be withheld. Then both arguments would have a single excess dimension of extent 4, and no replication would be needed. Each element of the vector would be paired with the corresponding row of A, and the result would be a [4,4]-matrix with the vector added as a new column to the front of A.

The need for locally affecting a function’s expectations is most clearly visible with functions that have generic arguments. These arguments are expected to be arrays, but there is no further constraint on their dimensionality. RPLUS is an example of a generic-argument function: it accepts an arbitrary array and produces the scalar sum of all the elements. Thus, if B is the matrix

134

275

the expression (RPLUS B) would evaluate to 22, the grand total of the cell values. This quantity can be used to determine the proportion of the total that is due to each cell:

(QUOTIENT B (RPLUS B))

QUOTIENT, like DIFFERENCE, is a scalar-to-scalar operation. Its arguments in this example are the matrix, and the scalar total. The matrix will be broken down into scalars, each one of which will be divided by the total, and the result will be an array with the same shape as B:

.045.136 .182
.091.318 .227

Suppose, however, that the cell proportions within columns are needed. In this case, each element of B must be divided not by the grand total, but by the total of all elements within its column. These numbers will result from collapsing just across the rows to produce a 3-vector of column totals (i.e., [3 10 9]). In other words, even though RPLUS will accept an arbitrary array, in certain situations it must behave as though it expected three 2-vectors (or two 3-vectors). The matrix B could then be sliced appropriately, the slices totaled, and the results pasted together to form the triple or pair of interest, which could then be given to the QUOTIENT function.

The behavior needed in examples like these is obtained by marking arrays so that certain dimensions will be withheld, or kept, no matter what the expectations might be of the function they are given to. The function KEEP provides this marking facility:

keep[a;dims...] where a is an array and dims... is an indefinite number of dimension specifications (either integers, dimension labels, or ALL) for a, produces a copy of a with dims added to (the front of) its kept dimensions. This will decompose in an extended function so that the kept dimensions will be preserved in the output.

If no dimensions are specified, KEEP returns a vector containing the indices of the kept dimensions of a.

Marking dimensions in this way causes the extension mechanism to treat an array as follows. First, the number of dimensions withheld will be the greater of the number determined by the argument-expectation discrepancy and the number of kept dimensions. Second, for the purposes of selecting the excess dimensions and aligning them across different arguments, the kept dimensions are considered to have been transposed to become the leading dimensions. In other words, all the kept dimensions will be withheld, and other dimensions may also be withheld in order to guarantee that the slices given to the extended function will not exceed its expectations. The "transposition" is carried out purely for alignment purposes, and the kept dimensions will appear in the result in exactly the same order that they appeared in the controlling argument, as before.

With this facility, the column-joining operation can be requested by

(ADJOIN (KEEP ’(1 2 3 4) 1) A)

The KEEP causes the first dimension to be withheld, so that ADJOIN behaves as if it expected a scalar first argument. KEEPing the generic argument of RPLUS yields the column proportions:

(QUOTIENT (KEEP B 2) (RPLUS (KEEP B 2)))

Since the second dimension is kept, the RPLUS in the denominator collapses across rows, giving [3 10 9]. The KEEP in the numerator moves the second dimension to the front, so that for purposes of argument conformance, the numerator is treated as a [3,2]-matrix. The 3 extent on the first excess dimension matches the 3 for the denominator excess. This is the only effect of the numerator KEEP, since both dimensions must be withheld anyway so that QUOTIENT receives only scalars. The row proportions for this array can be computed by the analogous expression:

(QUOTIENT (KEEP B 1) (RPLUS (KEEP B 1)))

It is occasionally necessary to undo the effect of keeping some dimensions of an array. The function LEAVE is the inverse of KEEP, producing a copy of an array without the marks that KEEP puts on.

leave[a;dims...] where a is an array and dims... is an indefinite number of dimension specifications (either integers, dimension labels, or ALL) for a, produces a copy of a with the specified dimensions not kept. LEAVE has no effect for dimensions that are specified in dims but not marked as kept in a.

2.3.4 User-written functions

The same mechanism that IDL uses to extend its own functions is available for user functions as well. This mechanism is embodied in the system functions EAPPLY and EAPPLY*, which apply a function to arguments decomposed according to the rules described above.

eapply[fn;expects;args] and eapply*[fn;expects;a1,a2,...] apply fn to the arguments args or a1,a2,... decomposed according to their keeps and the expected dimensionality as given in the list expects.

An expects entry of either SCALAR, VECTOR, MATRIX, or an integer indicates the expected dimensionality. The entry ARRAY indicates a generic argument whose decomposition will be controlled entirely by the actual argument’s keeps. The entry NIL indicates an argument that the extension mechanism will not examine at all.

If fn is a no-spread function, the expectations for its (arbitrarily many) trailing arguments can be specified by having the last expects entry be the atom ... in which case the preceding expects entry is used for all subsequent arguments.

The difference between EAPPLY and EAPPLY* is analogous to the difference between the Lisp functions APPLY and APPLY*. EAPPLY accepts a list of (already-evaluated) arguments for fn; EAPPLY* accepts a sequence of argument expressions which will be evaluated before the extension mechanism is invoked.

To simplify the task of defining new extended functions, IDL provides a new kind of function object, the ELAMBDA. The key-word ELAMBDA is used instead of LAMBDA in the function definition, and the expected dimensionality is specified with each argument. For example,

(ELAMBDA ((M MATRIX) (V VECTOR)) (ADJOIN V (MOMENTS M)))

is a function object that specifies that it expects its first argument to be a matrix and its second to be a vector. Such a function object may either be associated with some name, using DEFINEQ, e.g.,

[DEFINEQ (FOO (ELAMBDA ((M MATRIX) (V VECTOR)) (ADJOIN V (MOMENTS M]

or applied directly to a set of arguments. For example, for two arrays A1 and A2 of arbitrary dimensionality,

((ELAMBDA ((M MATRIX) (V VECTOR)) (ADJOIN V (MOMENTS M))) A1 A2)

is equivalent to

(EAPPLY* (FUNCTION (LAMBDA (M V) (ADJOIN V (MOMENTS M))))
’(MATRIX VECTOR)
A1 A2)

The technique of direct application of an ELAMBDA is an elegant way of forcing the repeated evaluation of some computation (see Chapter 8).

The function EXTEND provides a convenient way of extending some previously defined function. EXTEND operates by altering the function’s body so that it calls EAPPLY* with the appropriate information.

extend[fn;expects] alters the body of fn to be eapply*[fn;expects;a1,a2,...]. As EXTEND can detect if a function has previously been extended, a function may be re-extended simply by calling EXTEND with a different expects. An expects of NIL clears the extension.

2.4 Printing

Lisp objects may be printed in both simple and complex (or pretty) modes, and there are two corresponding modes for IDL arrays. The simple representation of an array is called its Lisp "print-name". This is printed whenever an object is returned to the Lisp executive as the value of some expression. The print-names for IDL arrays are designed to uniquely identify the array (via a serial number assigned to each array as it is created) and to briefly summarize the properties that directly affect how subsequent operations will apply to it. These properties are its shape and its kept dimensions. Suppose A is a [4,3]-matrix with serial number 7. Its print-name is

[Array 7: 1=4 2=3]

The integers before the equal-signs are the dimension indices, and the numbers afterwards are the number of levels on the corresponding dimensions. If the second dimension of A is kept, then the print-name would be

[Array 7: 1=4 2=3; kept 2]

If the array has dimension labels, these are used instead of the dimension indices to provide a more informative description of the array. If A were a Subject by Variable matrix, its print-name would be

[Array 7: Subject=4 Variable=3; kept Variable]

If Lisp is asked to pretty-print an IDL array, then all the properties of the array are displayed, including the title, dimension and level labels, and the values in the cells of the array. The examples in preceding sections and subsequent chapters illustrate the format in which this information is presented. Printing of this sort will result from applying the Lisp function PRINTDEF to an array, or by invoking the IDL function PPA:

ppa[a;file] pretty-prints the array a on the primary output file, or file if specified. If file is not open, it is opened, the array is printed, and then file is closed. Value is a.

The Lisp primary output file is usually the user’s terminal (denoted by the special file name T), so omitting a file specification will cause the array to be displayed directly to the user. IDL automatically maintains a "dribble" file (named IDL.TYPESCRIPT), a complete transcript of the user’s commands and the system’s responses throughout a session. This provides an enduring record of the arrays printed to the terminal and of the computational context in which those arrays are produced. The primary output file may be changed by the Lisp functions OUTFILE and OUTPUT.

The actual format in which the array appears may be controlled in a number of ways. Arrays may be printed with or without labels, and the precision and column-widths may be varied, depending on the values of the following global variables:

LABELPRINTFLAG

An integer which encodes how much of an array’s labeling the PPA routine will print. Initially 4, it is encoded as follows

0 – print no labels
1 – print only the array title
2 – print array title and dimension labels
3 – print title, dimension, and selector labels
4 – print title, dimension, selector, and value labels

When label printout is requested, missing labels will usually print as the corresponding integer index. An exception is that a totally unlabelled, small array (i.e., one that fits on a single panel of printout) will print without row and column indices, as they are both uninformative and apt to be confused for data values in this case. If there is no value label to correspond to a value found in an array, the actual score for which no label was found is printed under the prevailing rules for numeric printout. This is often a convenient way to find wild scores as the number stands out clearly in the value-labelled array.

linelength[n]

A Lisp function that indicates the number of character positions on a line. This determines the number of print positions that will be used before an array will be "folded" and subsequent columns printed as a separate section further down the page. If n is NIL, value is the current setting; otherwise, value is the previous setting.

PRECISION

A list of the form (L R) – initially, (4 3). L and R specify, respectively, the number of digits to be printed on the left and right of the decimal point. The field width is always L+R+2, providing for a sign and the decimal point. Column selector labels and value labels will be truncated on the right so that they fit within the field.

ROWLABELWIDTH

An integer, initially 8, which specifies the number of columns to be used for printing the selector labels for rows.