Page Numbers: Yes X: 527 Y: -.4 First Page: 43 Not-on-first-page
Margins: Top: 1.1" Bottom: 1" Binding: 5
Odd Heading: Not-on-first-page
7. DATA INPUT AND OUTPUT
Even Heading: Not-on-first-page
IDL REFERENCE MANUAL
7. Data Input and Output
The material in the preceding chapters assumed that the data to be analyzed already existed as an IDL array. This chapter considers how data may be entered into an array from an external representation, either from list structure residing in memory or from a sequence of characters on a file. Functions for converting an array back into an external representation are also described, so that, for example, objects created by IDL computations may be preserved from one session to the next.
7.1 To and from list structure
The easiest way of entering data is simply to type it in as list structure at the point in the IDL session when it is needed. Thus, typing
X ← ’(4 2 6 9 3 11 2 8 31 0)
saves the ten observations as a list in the variable X. As all IDL functions will convert numeric lists to arrays, X may be used whenever a vector would be appropriate (see section 1.7). This method has the disadvantages that the conversion would be carried out every time X was used as an array, and the resulting arrays have no labels or titles. Thus, IDL provides special functions for explicitly constructing arrays from list structure that also includes labelling.
idlarray[data] returns an array constructed according to the description found in data. data is a list in the following format
({title} organization {keeps} {format} elements)
The braces {} indicate that an item is optional and need not be specified. The title, if present, is simply a string enclosed in double-quotes. The organization specifies the shape and labelling of the array, and the optional keeps indicates its kept dimensions. The optional format is either SYMMETRIC or FULL (the default), and elements is a list of the values that will fill up the array in row-major order.
The organization is a list containing one sub-list for each dimension of the array. This is of the form
(dim = nlevels {level-labels1 level-labels2 ...} )
dim is either the dimension number or the dimension label, nlevels is the number of levels for that dimension, and level-labelsi specifies the level label and, for the value-labelled dimension, the value-labels for level i. If there are no value-labels, the specification may simply be the atomic level label, or NIL or the level index if the level is unlabelled. If there are value-labels, then the specification is a list consisting of the level label followed by the codebook for that level.
keeps, if present, is a list of the form (kept k1 k2 ...), where each ki is either a dimension number of label specifying a kept dimension.
For example, the expression
(IDLARRAY ’("Another random matrix"
((Subject = 4)
(Variable = 3 (SEX (1 MALE) (2 FEMALE)) AGE VOTE))
(kept Variable)
(1 24 2 3 31 1 2 28 3 1 25 2)))
produces the matrix
Another Random Matrix
Kept: Variable
Variable
SubjectSEXAGEVOTE
1MALE242
23311
3FEMALE283
4MALE252
The inverse of IDLARRAY is also provided:
listarray[a] produces a list-structure description of the array a in the form outlined above. This can be given back to IDLARRAY to reconstruct an array equivalent to the original.
Usually, data is collected and initially entered in the form of a subjects by variables matrix. These arrays can be described in the format above, but, for convenience, a more perspicuous list-structure representation is also allowed for such arrays. This representation groups the rows into individual lists and permits the row-labels to be specified next to the corresponding data values. The functions IDLMATRIX and LISTMATRIX process this special form of description.
idlmatrix[data] returns a matrix constructed according to the instructions and data found in data. data is a list each element of which is a list representing one row of the output. The list is a sequence of data values optionally preceded by the level label for that row. Before the row specifications there may be one list of the form
(TITLE title-string dimension1-label dimension2-label)
and another of the form
(LABELS level1-labels level2-labels ...)
These level labels are for levels on the second dimension of the result, and each one may be NIL (meaning no level label), a non-NIL literal atom which provides a level label but no codebook for the corresponding level, or a list of the form
(level-label codepair1 codepair2 ...)
For example, an array with two subjects, John, a 32 year old male, and Susan, a 27 year old female, could be created by
(IDLMATRIX ’( (TITLE "An array with two subjects" Subjects Variables)
(LABELS (Sex (1 Male)(2 Female)) Age)
(John 1 32)
(Susan 2 27) ))
The function LISTMATRIX is the inverse of IDLMATRIX. It produces the list structure corresponding to an IDL matrix:
listmatrix[m] returns a list structure representing the elements and labels of the IDL matrix m. This structure can be passed to IDLMATRIX to reconstruct a matrix equivalent to m.
List-structure descriptions can be written on files and read back in to form arrays, and this is one way of preserving arrays. However, the functions described below achieve the same effect without allocating intermediate list-structure and are thus more efficient. The main advantage of list structure representations is that they can be manipulated by the large number of Lisp list-processing functions, including the powerful editor (see Teitelman, 1978).
7.2 To and from files
Data can be stored on files in either the full-array format or the special matrix format. Arrays or matrices can be read into memory by reading the appropriate expression from the file to form the list structure, and then giving that structure to either IDLARRAY or IDLMATRIX. For example, suppose the file MYDATA contained only expressions representing the matrix form of an array, inserted by means of a text-editor familiar to the user. The data could be formed into an IDL matrix and bound to the variable D by means of the following expression:
D ← (IDLMATRIX (READFILE ’MYDATA))
The Lisp function READFILE opens a file, reads all the expressions from it and forms them into a list structure, closes the file, and returns that list structure. For this example, the file MYDATA must not have the outer parentheses enclosing the titles and rows of the matrix, since READFILE will itself insert those.
If the more general form of description is on the file, the function IDLARRAY may be used instead of IDLMATRIX to construct the array. However, it is more efficient to use the function READIDARRAY, for this constructs the array in one step, without building the intervening list structure:
readidlarray[file] constructs an IDL array from an IDLARRAY-type expression read from file (or the Lisp primary input file if file is NIL). If file is not an open file, it is opened, the expression is read, and then file is closed.
The information in arrays may be saved on a file by first converting it to list structure and then using any of the Lisp printing functions PRIN2, PRINT, PRINTDEF, etc. to print the structure on the open file. For the general array format, the function DUMPIDLARRAY may be used instead:
dumpidlarray[a;file] writes an IDLARRAY-type list expression on file (or the Lisp primary output file if file is NIL) from which an array equivalent to a can be constructed. If file is not an open file, it is opened, the expression is printed, and then file is closed.
The Lisp file package command IDLARRAYS can be used to dump and restore IDL arrays on LOADable files, via DUMPIDLARRAY and READIDLARRAY. This permits arrays to be saved on the same files as functions, variables, and other information, and to be restored by the simple Lisp function LOAD. The appropriate dumping and reading functions are automatically supplied. To include an array on a LOADable file SURVEYINFO, the Lisp variable SURVEYINFOCOMS must be a list of commands one of whose elements begins with IDLARRAYS and specifies the name of a variable whose value is the array to be dumped. Other commands might specify other objects to be saved. For example, if SURVEYINFOCOMS were bound to the command list
((VARS X Y) (IDLARRAYS SDATA) (FNS RECODE))
then the expression
(MAKEFILE ’SURVEYINFO)
would save on the file SURVEYINFO the values of the (non-array) variables X and Y, the array bound to SDATA, and the user function named RECODE. If in another IDL session the expression
(LOAD ’SURVEYINFO)
were evaluated, the variable values and function would be restored. The Interlisp Reference Manual (Teitelman, 1978) has a very extensive discussion of file commands, MAKEFILE, and LOAD.
Rather than saving particular arrays from one session to be used in another, it is also possible to preserve the complete state of the current session on a file. This file can be run at a later time to resume the suspended analysis session. The Lisp function SYSOUT moves the state of the current session to a file:
sysout[file] saves the state of the current IDL session on file. If file does not include an extension in its name, the extension SAV will be supplied by default.
A SYSOUT file may be resumed simply by running that file instead of IDL to start the session. For details, see Teitelman (1978).