5 Importing Gridded Data

It is usually possible to input a file written by some other program into DX. The procedure is simply to create a file which describes the contents of the data file to DX. This description file is called a ‘header file’ and is simply a text file created with an editor.

As an example this recipe shows how to import a formatted text file containing two data cubes. One data cube is a simple Gaussian field, the other is a Gaussian superimposed on a sloping background. Note that though the values are calculated using Fortran double precision variables they are written out as floating point numbers. DX interprets the ensuing file as containing single precision values. Each data cube is represented as a single DX field (see Section 17). The program to generate the data file is listed in Figure 7.

(1)

The DX header file for this data file is shown in Figure 5. These header files have file type ‘.general’. Thus the present example is called ‘field.general’.

---

  file=field.lis
  format=text
  grid=50x50x50
  majority=row
  interleaving=field
  field=Gaussian, Sloping
  structure=scalar, scalar
  type=float, float
  dependency=positions, positions

Figure 5: Header file for gridded data.

---

(2)

Each line of the header file consists of a keyword followed by one or more values. The purpose of the various keywords are as follows.

file

is the name of the data file. If the file name is not preceded by a directory specification then it is assumed to be in the same directory as the header file.

format

is the format of the file; the current example is a formatted file rather than a binary file.

grid

is the size of the grid of data. In the present case both data cubes comprise grids with fifty elements in each side.

majority

specifies the organisation of multi-dimensional arrays. ‘Column’ majority means that the first dimension varies fastest (as in Fortran); ‘row’ majority means that the last dimension varies fastest (as in C).

interleaving

specifies how the data for the two fields are interleaved within the file. DX permits various possibilities. For example, all the values for the first field could occur first, followed by all the values for the second field. In the present case the data for the two fields are more closely intertwined. Each record in the file contains a value for a position in the first field and a value for the corresponding position in the second field (see Figures 6 and 7). ‘interleaving=field’ specifies this sort of interleaving.

---

0.15655673        0.25655673
0.16572675        0.26572675
0.17514088        0.27514088
0.18476781        0.28476781
0.19457102        0.29457102
0.20450866        0.30450866
$⋮$                 $⋮$

Figure 6: The first few records of file field.lis (written by program field.f, Figure 7). The first column is values for the simple Gaussian data cube, the second column values for the Gaussian superimposed on a sloping background.

---

field

specifies names for the fields in the file. Here they are called ‘Gaussian’ (the simple Gaussian data cube) and ‘Sloping’ (the Gaussian superimposed on a sloping background). Note the use of a comma to separate the two names.

structure

specifies the structure of the two fields in the file. In the current example both the fields are simple scalars.

type

specifies the data types of the two fields. In the current example both fields are single precision real numbers.

dependency

denotes whether the data are position or connection dependent. See Section 17.4 for an explanation of these two cases. Here both fields are position dependent (which is probably the more common case in astronomy).

A full description of all the possible keywords is given in Section 4.3 Header File Syntax: Keyword Statements of the IBM QuickStart Guide[1].

(3)

Once a suitable description file has been been created the data can be imported into DX by including the ‘Import’ module in your network. Note that it is the name of the description file, not the name of the data file which must be supplied to ‘Import’.

5.1 Variations

(1)

DX includes ‘Data Prompter’, a windows-based application for interactively creating the header file for a data set. However, often it is no easier to use than creating the header file with an editor. Section 4.4 Data Prompter, of the QuickStart Guide[1] gives details.

(2)

Unformatted binary files can also be imported using the same header file mechanism. The format keyword should be set to ‘binary’. Binary files written with a C program can be imported directly. However, unfortunately, binary files written with a Fortran program contain record-control bytes which must be removed prior to importing the file. The Starlink extensions to DX include SXUnfort for this purpose; see SUN/203[11] for details. Of course, an unformatted file written on a Digital alpha will differ from the corresponding file written on a Sun because of the different byte order of the machines. It is possible to input an unformatted file written on a Sun into DX running on a Digital alpha, or vice versa, by using the ‘msb’ and ‘lsb’ modifiers to the format keyword⁴. See Section 4.3 Header File Syntax: Keyword Statements of the IBM QuickStart Guide[1] for details.

(3)

It is also possible to import files in the Starlink NDF format (and other common astronomical formats, such as FITS images or Figaro DST files). Again see SUN/203[11] for details.