E File naming and data structures

Unless instructed otherwise in the observation definition file (ODF) the SCUBA software writes a demodulated data file (essentially the raw data) and a reduced observation (RO) file to disk.

E.1 Naming Conventions

SCUBA data files have the following naming convention: string_obs type_run number.sdf. Where string is either the UT date of the observation in YYYYMMDD format or the PATT ID for the observation.

Therefore the first data from 23rd June 1997 will be in files

for the demodulated and reduced data respectively.

The RO file contains the reduced data calculated by the telescope during the observation and supplies a first look at the data. At present the on-line system does not produce calibrated publishable quality images – the data should be re-processed using the demodulated data and Surf.

E.2 Structure of RO files

Since SCUBA can produce data from more than one sub-instrument simultaneously and yet only one file is wanted for each observation, the RO files are in fact HDS container files containing NDFs. In general there is one NDF for each sub-instrument used plus one NDF containing all the observation parameters (it is a blank NDF containing a FITS extension).

For example, a map observation with the LONG and SHORT wave array would produce an RO file with the following structure:

The image from the long wave array would be displayed in Kappa with:

and the FITS information would be displayed with:

In addition to one NDF per sub-instrument, there is also one NDF per measurement. For example, the RO file from a FOCUS observation would contain an NDF for each FOCUS position names MEAS_1, MEAS_2 etc. – this is the case for FOCUS and ALIGN observations. Skydip data is slightly different since the names of the NDFs are of the form SUB_WAVELENGTH (i.e. SHORT_450 and LONG_850).

If an observation is only using a particular bolometer (eg a FLATFIELD or PHASE observation) there will be an NDF for the bolometer (eg H7) rather than for the array as a whole.

E.3 Structure of demodulated files

Demodulated data files are much more complicated as they have to contain a full description of the state of SCUBA for each observation. Here is an example trace (the map of 3C279 used in the example):

From this we can see that there is a 3 dimensional data array and 4 NDF extensions. Each will be described in turn:

DATA_ARRAY

The data array contains three dimensions. The first dimension is a vector of size 5 and relates to the chop data, the chop error, the calibrator data the calibrator error and the quality respectively. The second dimension is the bolometers and the third is the number of measurements (ie number of jiggle positions $\times$ number of integrations $\times$ 2 (number of nods)). These data can be displayed to examine each component in turn (for example checking whether the OFF position contains a source) by using NDF sections.

FIGARO extension

The Figaro extension is not used by the Surf package.

SCUCD extension

The SCUCD extension contains information on the position of the telescope for each sample along with the local sidereal time of each sample.

SCUBA extension

The SCUBA extension contains the flatfield information (position of the bolometers, sub-intrument of bolometers, quality of bolometers and A-to-D card). There are also arrays to aid in the location of each jiggle in the data array (DEM_PNTR).

FITS extension

The FITS Extension is described in the next appendix.

In addition a new extension is created by the Surf package itself. This extension (called REDS for historical reasons) contains beam weights for chopping, and pointing corrections. In general this extension is created for PHOTOMetry observations and by the change_pointing task.

A more detailed description of the file format can be found in Starlink System Note 72.