Page Numbers: Yes X: 527 Y: -.4 First Page: 29 Not-on-first-page
Margins: Top: 1.1" Bottom: 1" Binding: 5
Odd Heading: Not-on-first-page
4. ARRAY FUNCTIONS
Even Heading: Not-on-first-page
IDL REFERENCE MANUAL
4. Array Functions
The functions described in this chapter are IDL’s basic tools for manipulating arrays. None of these are specific to data analysis, but all of them are designed to make it easier to deal with the rectangular data aggregates commonly encountered in data analysis.
Unless stated otherwise, all the functions in this chapter are extended. Thus, if a function is described as expecting an array of a certain dimensionality, an argument of higher dimensionality will be broken into slices of that size before the function is applied.
4.1 ADJOIN
ADJOIN joins an indefinite number of vectors together end to end.
adjoin[vectors...] produces a vector of length equal to the sum of the lengths of the arguments which consists of the elements of the first argument, followed by the elements of the second, and so on.
With the use of appropriate keeps, ADJOIN can be used to add sections to higher dimensional objects. For example, to form two conformable matrices A and B into a 3-array with two planes on the third dimension ("laying A flat on top of B"), one might use the expression
(ADJOIN (KEEP A ALL) (KEEP B ALL))
This breaks A and B down into scalars, the ADJOIN assembles each pair into a 2-vector, and the extension mechanism assembles these results into an image of A and B.
4.2 DEAL
DEAL generates random permutation vectors.
deal[n] where n is a positive integer, returns a vector of length n consisting of the set of integers from 1 to n in a random order.
The name DEAL reflects the fact that applying the resulting vector as a selector to, say, a deck of cards shuffles the deck. DEAL vectors are useful for randomizing treatments in experimental design (see section 8.1). Note that (probably) different vectors are returned from separate calls to DEAL with the same argument.
As the length of its result varies with the value of its input, DEAL is not extended.
4.3 FORMAT and ELEMENTTYPE
format[a] returns the storage format (FULL or SYMMETRIC) of the array a.
elementtype[a] returns the type (INTEGER or FLOATING) of the elements of the array a.
4.4 GENVEC
GENVEC (for generate vector) produces a vector whose elements form a linear sequence, i.e., each element differs from its predecessor by a fixed amount. Such a sequence is completely determined by its first two elements and its last element.
genvec[initial;end] where initial is either a 2-vector or a scalar and end is a scalar, returns a vector generated as follows:
If initial is a 2-vector, its first element is the first element of the sequence, and the difference between its two elements is added to the ith element of the sequence to form the i+1st element, until end is reached.
If initial is a scalar, it is treated as the first element of a 2-vector whose second element is 1 if end is greater than initial and –1 otherwise. The sequence is then constructed as described above.
Thus, (GENVEC 4 7) generates the vector [4 5 6 7], and (GENVEC ’(1 3) 9) yields [1 3 5 7 9]. Such vectors are particularly useful for selection. For example, if A is a vector of length N, then
A@(LIST (GENVEC N 1)) produces the reverse of A, and
A@(LIST (GENVEC ’(1 3) N)) produces the vector containing every second element.
Note that for short sequences, it may be more convenient to simply type in the elements as a list.
As the length of its result varies with the value of its input, GENVEC is not extended.
4.5 IDLARRAYP
idlarrayp[a] is T if a is an IDL array, NIL otherwise.
4.6 INVERT
invert[m] returns the inverse of its argument matrix.
INVERT uses Gaussian elimination with total pivoting and is thus somewhat slow but numerically very stable. If the argument is not square, its square top left-hand corner is interpreted as a coefficient matrix and the rest of the matrix is taken as a set of value vectors. The inverse of the square corner appears in the result, along with the solutions to the set of linear equations corresponding to the value vectors.
4.7 MPROD
mprod[a;b] where a and b are conformable matrices or vectors, returns the usual matrix product of the two matrices.
The conformability requirement on the two arguments is that the second dimension of the first must have the same number of levels as the first dimension of the second. However, the vector case is treated specially so that a useful orientation of vector arguments is achieved. For example, an R-vector multiplied by an S-vector is treated as if it were an [R,1]-matrix by a [1,S]-matrix, resulting in a [R,S]-matrix. Consequently, the conventional dot-product of two equal-length vectors cannot be obtained with MPROD; the expression (RPLUS R*S) will produce that scalar quantity, since TIMES will be applied in an elementwise fashion.
4.8 ORDER
order[v;cfn] for v a vector and cfn an (optional) comparison function, returns the permutation vector which would, if applied as a selector, sort v into order according to cfn. ORDER does not actually sort v. If no comparison function is supplied, LESSP is used and the resulting permutation vector would sort the elements into ascending order.
For example, (ORDER ’(4 6 –3 9 –100)) returns the vector [5 3 1 2 4]. To actually sort a vector A, one might use A@(LIST (ORDER A)).
4.9 RANK
rank[a] returns an image of the array a with each element being the rank of the corresponding element among the other elements of a. Although a similar result could be obtained using ORDER (specifically, order[order[a]]), RANK both corrects for ties by assigning the average rank to each member of a group of tied observations and treats cells containing NIL specially.
Cells containing NIL are ignored for the purpose of computing the ranks, so the ranks will range from 1 to the number of non-missing observations. Furthermore, the corresponding cells in the output array will also be NIL. That is, (RANK ’(4 NIL 1 2)) is [3 NIL 1 2]. This is done because ranks are often used as data transformations, in which case the NIL should propagate. The arithmetic interpretation of NIL is, in any case, irrelevant in this context.
RANK accepts arrays of any dimension and constructs one set of ranks for the entire object. Thus, within-column ranks for a matrix M cannot be obtained simply by calling (RANK M). If the ranks are to be determined independently for each column, the larger structure must be explicitly broken down into vectors with KEEP, as in (RANK (KEEP M 2)).
RANK is often used in computing the statistics for non-parametric tests of significance (see Chapter 8).
4.10 REDUCE, RPLUS, RTIMES
reduce[a;f;startingval] where a is an array, f is a scalar function of two arguments, and startingval is a (optional) scalar, returns the value of applying f to the members of a in row-major order (i.e., for vector a, f[...f[f[startingval;a1];a2] ... an] ). Thus, the first argument of f will be the result of previous applications, and the second argument will be the next element of a.
If startingval is specified and a has no elements, startingval is returned.
If startingval is not specified, then the initial invocation of f will be f[a1;a2] instead of f[startingval;a1], provided a has at least two elements. Otherwise, a1 is returned if it exists, else NIL.
For convenience, the reductions of addition and multiplication have been given special function definitions:
(RPLUS A) is equivalent to (REDUCE A ’PLUS 0), and
(RTIMES A) is equivalent to (REDUCE A ’TIMES 1).
Thus (RPLUS A) will return the sum of the elements of A, 0 if A is empty, or NIL if A contains any missing values. (REDUCE A ’MAX) yields the maximum element in A, while (REDUCE A ’NCONC1 NIL) returns a Lisp list of the elements of A in row-major order.
4.11 RESHAPE
RESHAPE forms a new array of a given shape out of the elements of another.
reshape[a;newshape;newformat] where a is an array, newshape is an (optional) vector, and newformat is an (optional) format specification (either FULL or SYMMETRIC, default is FULL), returns an array of the shape and format given, whose elements are those of a. This is done by enumerating a in row-major order to fill the places in the result, which is also filled in row-major order. If a is exhausted before the newly constructed object is filled, it is repeated from the beginning in the same sequence.
For example, (RESHAPE 0 ’(2 3)) produces a [2,3]-matrix of zeros. In the special case when newshape is NIL, a is "flattened" to a vector of the same length as there are elements in a.
4.12 SEEK
The SEEK function searches a vector for occurrences of specified value(s).
seek[sought;vec] where sought is either a scalar, a vector, or a function of one argument, and vec is a vector, returns a vector (possibly empty) of all indices i such that
vec@’(i) equals sought for scalar sought,
vec@’(i) is a member of sought for vector sought, or
apply[sought;vec@’(i)] is not NIL for function sought.
The indices are returned in the result in ascending order. For example
(SEEK ’(1 3 7) ’(6 5 4 3 2 1)) is [4 6] and
(SEEK ’PLUSP ’(–4 2 5 –1)) is [2 3].
As the length of the result returned by SEEK varies with the values of its inputs, it is not extended.
4.13 SHAPE
shape[a] returns the shape of its argument as a vector. The shape of a scalar is the empty vector, a vector of no elements. (SHAPE (SHAPE A)) is always a 1-vector of the dimensionality of A.
4.14 SHIFT
shift[v;shift;fill] returns a vector formed by shifting vector v shift positions (positive shift shifts to the right; negative to left). The elements vacated from one end of v are filled by values shifted from the opposite end of fill.
If fill is a scalar, then that scalar will replace all vacated elements. If fill is omitted, then v itself is used, and SHIFT has the effect of rotating the elements of v.
If the absolute value of shift is greater than the length of fill, it is reused as many times as needed.
Examples:
(SHIFT ’(1 2 3 4) 2 ’(11 12 13)) is [12 13 1 2]
(SHIFT ’(1 2 3 4) 2) is [3 4 1 2]
(SHIFT ’(1 2 3 4) –2 0) is [3 4 0 0]
SHIFT provides a way of generating cumulative reductions for scalar functions. For example, the cumulative sum vector for a vector V is formed by
C ← (RPLUS (KEEP (SHIFT V (GENVEC (SHAPE V)–1 0) 0) 1) )
This constructs a matrix whose rows contain successively longer initial subsequences of V, padded on the left with zeros. PLUS-reduction along the rows of this matrix yields a vector of the cumulative sums. V can be recovered from C by
C – (SHIFT C 1 0)
which lines up each element in C with the cumulative sum of all levels before it. The subtraction yields the original values.
This technique can be used to convert between vectors containing cumulative and raw frequency counts.
4.15 TRANSPOSE
The TRANSPOSE function permutes the dimensions of an IDL array.
transpose[a;perm] where a is an n-array, and perm is an (optional) n-vector (for arbitrary n). TRANSPOSE treats the vector perm as a mapping from the dimensions of a onto the dimensions of the result. Dimensions may be repeated or arbitrarily permuted in perm, provided only that, if the largest element of perm is x (which will be the dimensionality of the result), then all the integers between 1 and x appear in perm.
If perm is omitted, it is assumed to be (GENVEC n 1) and the dimensions of a are reversed. Thus (TRANSPOSE M) for matrix M produces the conventional transposition.
The result is the obvious dimension rearrangement when perm is a permutation vector of the integers from 1 to n. However, more complex rearrangements are possible. For example,
(TRANSPOSE A ’(1 1))
with A an [m,m]-matrix will return the m-vector major diagonal of A.
As TRANSPOSE contains facilities for selecting the dimensions on which it is to operate, it is not extended.