13 Processing lists of data

Perhaps the most noticeable ‘feature’ of CCDPACK programs is their ability to accept, and process, lists of data files and other associated parameters, such as exposure times and position list names (see “§8.2.5”). A list of ‘names’ can be supplied in response to one prompt, or can be supplied on successive lines using the continuation character ‘-’ to force a reprompt for more specifications. A list of names consists of a series of character strings separated by commas. Note that the list itself is really a string not a vector and should be enclosed in quotes "". The quotes are not necessary if the list consists of only one name, or if given in response to a prompt. When using the C-shell it is necessary to protect the "" so that the final string passed to the application still contains these (so a suitable response would be ’"ndf1,ndf2"’).

13.1 Input wildcards

The names of images (NDFs or some foreign format) given to CCDPACK routines may include wildcard specifications such as:

all of which have usual meanings, i.e. any number of characters, a single character, a range of characters, and a list of strings. The simplest return would then be:

and all the images in the current directory would be accessed. Other possibilities include specifications such as:

If any of the image names you specify is an HDS container file holding more than one NDF structure, then each NDF contained directly within that file is processed as a separate image. If the data product you are using is supplied in this form, which for instance is sometimes the case for a set of frames from one exposure of a mosaic camera, this makes it much easier to process a group of images at once. So if you had an appropriate container file expos1234.sdf, then just supplying:

would allow processing of a whole group of related images. Note that it is easy to convert Multi-Extension FITS files (MEFs) into HDS container files — this is explained in section 6.

The names of images given to programs (except for XREDUCE) do not normally require the addition of the file extension. This is only necessary when there is some ambiguity over which files to use (when for instance several images of the same name, but of different types are available). However, the file extension will be accepted if given. So for instance repeating the last examples for IRAF data frames could look like:

13.2 Indirection

Names can also be stored in ordinary text files. Indirection through a text file is indicated by the character.

  $$^  (tophat)

The names may include wildcards (for images) and other indirections (up to 7 deep). A typical response might be:

And the rflatfields.lis file would contain something like:

Indirection can be mixed with other specifications in response to a prompt (or on the command line) i.e.:

etc.

13.3 Output names

All output names may be created from wildcards and/or formed through indirection. However, this is not as flexible as the input scheme, and wildcards and indirection cannot be mixed. An example of an output specification is:

This means call all the output images the same as the input images and put them in the same directory. Not necessarily what you want to do. Alternatively:

means call all the output images the same name as the associated input images, but append the string ‘_debias’ to the names. A third option using wildcard methods is to replace the occurrences of a particular string within the input names with a new string, e.g.:

This will end with the image names having the string ‘debias’ replaced with ‘flattened’. Indirection files follow the usual rules — but if a wildcard is used in the file this must be the only entry — and of course explicit names can be always be given for the output images in response to a prompt or on the command line.

When image names include the directories too, only the file name itself may be modified. Changing the directory of the output images (which otherwise will always be the same as the input images) is achieved by commands like:

Which in this case will take all the images ‘*_ccd’ from one directory and create new images in the second directory replacing any occurrences of the string ‘_ccd’ with ‘-pr’. Remember the ‘*’ in output expressions represents only the names of the input images not the directory or any other information (such as the extension or the NDF’s name within an HDS container file), these are only used if no ‘preferences’ are shown in the output expression. To keep images from other directories in the current directory use commands like:

If the input group was a set of images in an HDS container file, the output group will have the same structure. So if the file expo1234.sdf contains two NDFs called I1 and I2, then specifying

will write a new container file expo1234-out.sdf holding the output images as NDFs called I1 and I2.

In general the same rules apply for non-image output names (such as when position list access routines are not using image association to supply the name of the appropriate file), the only real difference is that when dealing with file names the complete name will be used (including the file type and directory information) and any substitutions must take this into account.

Output data frames are written as the format determined by your foreign data access setup. If you want to be sure of the type of your output images then append the appropriate file type (if you use the CONVERT defaults then output images without types are created as NDFs). So for instance repeating the above commands with FITS output images, you’d use: