6 Data formats

The “native” data format that CCDPACK uses is the Starlink NDF (the N-dimensional data format (SUN/33), which is based on HDS, the Hierarchical Data System (SUN/92)). This is portable between the operating systems that CCDPACK runs on (Digital UNIX, Solaris and Linux at this time) so it can be copied, accessed via NFS and ftp’d (using binary transfer) between these systems.

If you have your data stored on tapes in FITS format then you can use the KAPPA application FITSIN to read them into NDFs. You can also get a complete list of all the FITS headers of all the files on a tape (or list of files) using the KAPPA application FITSHEAD.

If you already have (or want to keep) your data in another astronomical format such as IRAF, disk FITS or old FIGARO then CCDPACK can also process these, but with an additional processing overhead. To use data in these formats you should initialise the CONVERT package (SUN/55):

  % convert

and the necessary facilities will be set up. Having done this, the simplest way to proceed is to use the full name of the unconverted files (i.e. including an extension such as ‘.fits’) when giving filenames to CCDPACK; it will transparently convert them to NDF format as required without any further effort from you.

An alternative is to convert the files explicitly to NDF format before using CCDPACK commands on them. Depending on how you are using the commands, this may save processing time by preventing CCDPACK from doing the same conversion more than once. A full description of how to do this is given in SUN/55, but normally it just consists of running a command called ‘ <TYPE >2NDF’. For instance converting all the FITS files in the current directory to NDF files with the same name can be done like this:

  % fits2ndf in=’*.fits’ out=’*’

One useful tip for FITS2NDF is to set the CONTAINER parameter to true if you are using Multi-Extension FITS files (MEFs), i.e. type instead

  % fits2ndf in=’*.fits’ out=’*’ container=true

This will convert an MEF into a single HDS container file containing each of the HDUs in the MEF — the upshot of this is that you can pass a single file name to CCDPACK programs and it will process all the images contained therein. It may also make Set processing (see section 9) easier.

One point that you should take note of is that not all formats are as flexible as NDF and cannot therefore store all the information that can be generated by CCDPACK. In particular IRAF data files cannot store additional data arrays such as variance and quality, so for instance you cannot gain any useful information about how errors propagate in your data. Also the extension information stored by CCDPACK has to be converted into native headers, this makes it less obvious what header information CCDPACK is using.

If you are unfortunate enough to have data in a format not supported by the CONVERT package then you will need to consult SSN/20 about how to proceed. You should bear in mind that the requirements of CCDPACK are that your format supports the storage of an image and some associated header information (this is essential for registration).