16 Storing AST Objects in FITS Headers (FitsChans)

 16.1 The Native FITS Encoding
 16.2 The FitsChan Model
 16.3 Creating a FitsChan
 16.4 Addressing Cards in a FitsChan
 16.5 Writing Native Objects to a FitsChan
 16.6 Extracting Individual Cards from a FitsChan
 16.7 The Native FitsChan Output Format
 16.8 Adding Individual Cards to a FitsChan
 16.9 Adding Concatenated Cards to a FitsChan
 16.10 Reading Native Objects From a FitsChan
 16.11 Saving and Restoring Multiple Objects in a FitsChan
 16.12 Mixing Native Objects with Other FITS Cards
 16.13 Finding and Changing Cards in a FitsChan
 16.14 Source and Sink Functions for FitsChans

A FITS header is a sequence of 80-character strings, formatted according to particular rules defined by the Flexible Image Transport System (FITS). FITS24 is a widely-used standard for data interchange in astronomy and has also been adopted as a data processing format in some astronomical data reduction systems. The individual 80-character strings in a FITS header are usually called cards or header cards (for entirely anachronistic reasons).

A sequence of FITS cards appears as a header at the start of every FITS data file, and sometimes also at other points within it, and is used to provide ancillary information which qualifies or describes the main array of data stored in the file. As such, FITS headers are prime territory for storing information about the coordinate systems associated with data held in FITS files.

In this section, we will examine how to store information in FITS headers directly in the form of AST Objects—a process which is supported by a specialised class of Channel called a FitsChan. Our discussion here will turn out to be a transitional step that emphasises the similarities between a FitsChan and a Channel (§15). At the same time, it will prepare us for the next section (§17), where we will examine how to use a FitsChan to tackle some of the more difficult problems that FITS headers can present.

16.1 The Native FITS Encoding

As it turns out, we are not the first to have thought of storing WCS information in FITS headers. In fact, the original FITS standard (1981 vintage) defined a set of header keywords for this purpose which have been widely used, although they have proved too limited for many practical purposes.

At the time of writing, a number of different ways of using FITS headers for storing WCS information are in use, most (although not all) based on the original standard. We will refer to these alternative ways of storing the information as FITS encodings but will defer a discussion of their advantages and limitations until the next section (§17).

Here, we will examine how to store AST Objects directly in FITS headers. In effect, this defines a new encoding, which we will term the native encoding. This is a special kind of encoding, because not only does it allow us to associate conventional WCS calibration information with FITS data, but it also allows any other information that can be expressed in terms of AST Objects to be stored as well. In fact, the native encoding provides us with facilities roughly analogous to those of the Channel15)—i.e. a lossless way of transferring AST Objects from program to program—but based on FITS headers instead of free-format text.

16.2 The FitsChan Model

I/O between AST Objects and FITS headers is supported by a specialised form of Channel called a FitsChan. A FitsChan contains a buffer which may hold any number, including zero, of FITS header cards. This buffer forms a workspace in which you can assemble FITS cards and manipulate them before writing them out to a file.

By default, when a FitsChan is first created, it contains no cards and there are five ways of inserting cards into it:

(1)
You may add cards yourself, one at a time, using astPutFits16.8).
(2)
You may add cards yourself, supplying all cards concatenated into a single string, using astPutCards16.9).
(3)
You may write an AST Object to the FitsChan (using astWrite), which will have the effect of creating new cards within the FitsChan which describe the Object (§16.5).
(4)
You may assign a value to the SourceFile attribute of the FitsChan. The value should be the path to a text file holding a set of FITS header cards, one per line. When the SourceFile value is set (using astSetC or astSet), the file is opened and the headers copied from it into the FitsChan. The file is then immediately closed.
(5)
You may specify a source function which reads data from some external store of FITS cards, just like the source associated with a basic Channel (§15.13). If you supply a source function, it will be called when the FitsChan is created in order to fill it with an initial set of cards (§16.14).

There are also four ways of removing cards from a FitsChan:

(1)
You may delete cards yourself, one at a time, using astDelFits16.13).
(2)
You may read an AST Object from the FitsChan (using astRead), which will have the effect of removing those cards from the FitsChan which describe the Object (§16.10).
(3)
You may assign a value to the FitsChan’s SinkFile attribute. When the FitsChan is deleted, any remaining headers are written out to a text file with path equal to the value of the SinkFile attribute.
(4)
Alternatively, you may specify a sink function which writes data to some external store of FITS cards, just like the sink associated with a basic Channel (§15.14). If you supply a sink function, it will be called when the FitsChan is deleted in order to write out any FITS cards that remain in it (§16.14). Note, the sink function is not called if the SinkFile attribute has been set.

Note, in particular, that reading an AST Object from a FitsChan is destructive. That is, it deletes the FITS cards that describe the Object. The reason for this is explained in §17.5.

In addition to the above, you may also read individual cards from a FitsChan using the function astFindFits (which is not destructive). This is the main means of writing out FITS cards if you have not supplied a sink function. astFindFits also provides a means of searching for particular FITS cards (by keyword, for example) and there are other facilities for overwriting cards when required (§16.13).

16.3 Creating a FitsChan

The FitsChan constructor function, astFitsChan, is straightforward to use:

#include "star/ast.h"
AstFitsChan *fitschan;

...

fitschan = astFitsChan( NULL, NULL, "Encoding=NATIVE" );

Here, we have omitted any source or sink functions by supplying NULL pointers for the first two arguments. We have also initialised the FitsChan’s Encoding attribute to NATIVE. This indicates that we will be using the native encoding (§16.1) to store and retrieve Objects. If this was left unspecified, the default would depend on the FitsChan’s contents. An attempt is made to use whatever encoding appears to have been used previously. For an empty FitsChan, the default is NATIVE, but it does no harm to be sure.

16.4 Addressing Cards in a FitsChan

Because a FitsChan contains an ordered sequence of header cards, a mechanism is needed for addressing them. This allows you to specify where new cards are to be added, for example, or which card is to be deleted.

This role is filled by the FitsChan’s integer Card attribute, which gives the index of the current card in the FitsChan. You can nominate any card you like to be current, simply by setting a new value for the Card attribute, for example:

int icard;

...

astSetI( fitschan, "Card", icard )

where “icard” contains the index of the card on which you wish to operate next. Some functions will update the Card attribute as a means of advancing through the sequence of cards, when reading them for example, or to indicate which card matches a search criterion.

The default value for Card is one, which is the index of the first card. This means that you can “rewind” a FitsChan to access its first card by clearing the Card attribute:

astClear( fitschan, "Card" );

The total number of cards in a FitsChan is given by the integer Ncard attribute. This is a read-only attribute whose value is automatically updated as you add or remove cards. It means you can address all the cards in sequence using a loop such as the following:

int ncard;

...

ncard = astGetI( fitschan, "Ncard" );
for ( icard = 1; icard <= ncard; icard++ ) {
   astSetI( fitschan, "Card", icard );
   <access the current card>
}

However, it is usually possible to write slightly tidier loops based on the astFindFits function described later (§16.6 and §16.13).

If you set the Card attribute to a value larger than Ncard, the FitsChan is regarded as being positioned at its end-of-file. In this case there is no current card and an attempt to obtain a value for the Card attribute will always return the value Ncard + 1. When a FitsChan is empty, it is always at the end-of-file.

16.5 Writing Native Objects to a FitsChan

Having created an empty FitsChan16.3), you can write any AST Object to it in the native encoding using the astWrite function. Let us assume we are writing a SkyFrame,25 as follows:

AstSkyFrame *skyframe;
int nobj;

...

nobj = astWrite( fitschan, skyframe );

Since we have selected the native encoding (§16.1), there are no restrictions on the class of Object we may write, so astWrite should always return a value of one, unless an error occurs. Unlike a basic Channel15.3), this write operation will not produce any output from our program. The FITS headers produced are simply stored inside the FitsChan.

After this write operation, the Ncard attribute will be updated to reflect the number of new cards added to the FitsChan and the Card attribute will point at the card immediately after the last one written. Since our FitsChan was initially empty, the Card attribute will, in this example, point at the end-of-file (§16.4).

The FITS standard imposes a limit of 68 characters on the length of strings which may be stored in a single header card. Sometimes, a description of an AST Object involves the use of strings which exceed this limit (e.g. a Frame title can be of arbitrary length). If this occurs, the long string will be split over two or more header cards. Each “continuation” card will have the keyword CONTINUE in columns 1 to 8, and will contain a space in column 9 (instead of the usual equals sign). An ampersand (“&”) is appended to the end of each of the strings (except the last one) to indicate that the string is continued on the next card.

Note, this splitting of long strings over several cards only occurs when writing AST Objects to a FitsChan using the astWrite function and the native encoding. If a long string is stored in a FitsChan using (for instance) the astPutFits or astPutCards function, it will simply be truncated.

16.6 Extracting Individual Cards from a FitsChan

To examine the contents of the FitsChan after writing the SkyFrame above (§16.5), we must write a simple loop to extract each card in turn and print it out. We must also remember to rewind the FitsChan first, e.g. using astClear. The following loop would do:

#include <stdio.h>
char card[ 81 ];

...

astClear( fitschan, "Card" );
while ( astFindFits( fitschan, "%f", card, 1 ) ) (void) printf( "%s\n", card );

Here, we have used the astFindFits function to find a FITS card by keyword. It is given a keyword template of “%f”, which matches any FITS keyword, so it always finds the current card, which it returns. Its fourth argument is set to 1, to indicate that the Card attribute should be incremented afterwards so that the following card will be found the next time around the loop. astFindFits returns zero when it reaches the end-of-file and this terminates the loop.

If we were storing the FITS headers in an output FITS file instead of printing them out, we might use a loop like this but replace “printf” with a suitable data storage operation. This would only be necessary if we had not provided a sink function for the FitsChan (§16.14).

16.7 The Native FitsChan Output Format

If we print out the FITS header cards describing the SkyFrame we wrote earlier (§16.5), we should obtain something like the following:

COMMENT AST ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ AST
COMMENT AST            Beginning of AST data for SkyFrame object             AST
COMMENT AST ................................................................ AST
BEGAST_A= ’SkyFrame’           / Description of celestial coordinate system
NAXES_A =                    2 / Number of coordinate axes
AX1_A   = ’        ’           / Axis number 1
BEGAST_B= ’SkyAxis ’           / Celestial coordinate axis
ENDAST_A= ’SkyAxis ’           / End of object definition
AX2_A   = ’        ’           / Axis number 2
BEGAST_C= ’SkyAxis ’           / Celestial coordinate axis
ENDAST_B= ’SkyAxis ’           / End of object definition
ISA_A   = ’Frame   ’           / Coordinate system description
SYSTEM_A= ’FK4-NO-E’           / Celestial coordinate system type
EPOCH_A =               1958.0 / Besselian epoch of observation
ENDAST_C= ’SkyFrame’           / End of object definition
COMMENT AST ................................................................ AST
COMMENT AST               End of AST data for SkyFrame object                AST
COMMENT AST ---------------------------------------------------------------- AST

As you can see, this resembles the information that would be written to a basic Channel to describe the same SkyFrame (§15.8), except that it has been formatted into 80-character header cards according to FITS conventions.

There are also a number of other differences worth noting:

(1)
There is no unnecessary information about default values provided for the benefit of the human reader. This is because the Full attribute for a FitsChan defaults to 1, thus suppressing this information (c.f. §15.9). You can restore the information if you wish by setting Full to 0 or +1, in which case additional COMMENT cards will be generated to hold it.
(2)
The information is not indented, because FITS does not allow this. However, if you change the Full attribute to 0 or +1, comments will be included that are intended to help break up the sequence of headers and highlight its structure. This will probably only be of use if you are attempting to track down a problem by examining the FITS cards produced in detail.
(3)
The FITS keywords which appear to the left of the “ =” signs have additional characters (“_A”, “_B”, etc.) appended to them. This is done in order to make each keyword unique.

This last point is worth further comment and is necessary because the FITS standard only allows for certain keywords (such as COMMENT and HISTORY) to appear more than once. astWrite therefore appends an arbitrary sequence of two characters to each new keyword it generates in order to ensure that it does not duplicate any already present in the FitsChan.

The main risk from not following this convention is that some software might ignore (say) all but the last occurrence of a keyword before passing the FITS headers on. Such an event is unlikely, but would obviously destroy the information present, so astWrite enforces the uniqueness of the keywords it uses. The extra characters added are ignored when the information is read back.

As with a basic Channel, you can also suppress the comments produced in a FitsChan by setting the boolean (integer) Comment attribute to zero (§15.10). However, FITS headers are traditionally generously commented, so this is not recommended.

16.8 Adding Individual Cards to a FitsChan

To insert individual cards into a FitsChan, prior to reading them back as Objects for example, you should use the astPutFits function. You can insert a card in front of the current one as follows:

astPutFits( fitschan, card, 0 );

where the third argument of zero indicates that the current card should not be overwritten. Note that facilities are not provided by AST for formatting the card contents.

After inserting a card, the FitsChan’s Card attribute points at the original Card, or at the end-of-file if the FitsChan was originally empty. Entering a sequence of cards is therefore straightforward. If “cards” is an array of pointers to strings containing FITS header cards and “ncards” is the number of cards, then a loop such as the following will insert the cards in sequence into a FitsChan:

#define MAXCARD 100
char *cards[ MAXCARD ];
int ncard;

...

for ( icard = 0; icard < ncard; icard++ ) astPutFits( fitschan, cards[ icard ], 0 );

The string containing a card need not be null terminated if it is at least 80 characters long (we have not allocated space for the strings themselves in this brief example).

Note that astPutFits enforces the validity of a FitsChan by rejecting any cards which do not adhere to the FITS standard. If any such cards are detected, an error will result.

16.9 Adding Concatenated Cards to a FitsChan

If you have all your cards concatenated together into a single long string, each occupying 80 characters (with no delimiters), you can insert them into a FitsChan in a single call using astPutCards. This call first empties the supplied FitsChan of any existing cards, then inserts the new cards, and finally rewinds the FitsChan so that a subsequent call to astRead will start reading from the first supplied card. The astPutCards function uses astPutFits internally to interpret and store each individual card, and so the caveats in §16.8 should be read.

For instance, if you are using the CFITSIO library for access to FITS files, you can use the CFITSIO fits_hdr2str function to obtain a string suitable for passing to astPutCards:



if( !fits_hdr2str( fptr, 0, NULL, 0, &header, &nkeys, &status ) )
   fitschan = astFitsChan( NULL, NULL, "" );
   astPutCards( fitschan, header );
   header = free( header );
   wcsinfo = astRead( fitschan );

   ...
}

16.10 Reading Native Objects From a FitsChan

Once you have stored a FITS header description of an Object in a FitsChan using the native encoding (§16.5), you can read it back using astRead in much the same way as with a basic Channel15.4). Similar comments about validating the Object you read also apply (§15.6). If you have just written to the FitsChan, you must remember to rewind it first:

AstObject *object;

...

astClear( fitschan, "Card" );
object = astRead( fitschan );

An important feature of a FitsChan is that read operations are destructive. This means that if an Object description is found, it will be consumed by astRead which will remove all the cards involved, including associated COMMENT cards, from the FitsChan. Thus, if you write an Object to a FitsChan, rewind, and read the same Object back, you should end up with the original FitsChan contents. If you need to circumvent this behaviour for any reason, it is a simple matter to make a copy of a FitsChan using astCopy4.13). If you then read from the copy, the original FitsChan will remain untouched.

After a read completes, the FitsChan’s Card attribute identifies the card immediately following the last card read, or the end-of-file of there are no more cards.

Since the native encoding is being used, any long strings involved in the object description will have been split into two or more adjacent contuation cards when the Object was stored in the header using function astWrite. The astRead function reverses this process by concatenating any such adjacent continuation cards to re-create the original long string.

16.11 Saving and Restoring Multiple Objects in a FitsChan

When using the native FITS encoding, multiple Objects may be stored and all I/O operations are sequential. This means that you can simply write a sequence of Objects to a FitsChan. After each write operation, the Card attribute will be updated so that the next write appends the next Object description to the previous one.

If you then rewind the FitsChan, you can read the Objects back in the original order. Reading them back will, of course, remove their descriptions from the FitsChan (§16.10) but the behaviour of the Card attribute is such that successive reads will simply return each Object in sequence.

The only thing that may require care, given that a FitsChan can always be addressed randomly by setting its Card attribute, is to avoid writing one Object on top of another. For obvious reasons, the Object descriptions in a FitsChan must remain separate if they are to make sense when read back.

16.12 Mixing Native Objects with Other FITS Cards

Of course, any real FITS header will contain other information besides AST Objects, if only the mandatory FITS cards that must accompany all FITS data. When FITS headers are read in from a real dataset, therefore, any native AST Object descriptions will be inter-mixed with many other cards.

Because this is the normal state of affairs, the boolean (integer) Skip attribute for a FitsChan defaults to one. This means that when you read an Object From a FitsChan, any irrelevant cards will simply be skipped over until the start of the next Object description, if any, is found. If you start reading part way through an Object description, no error will result. The remainder of the description will simply be skipped.

Setting Skip to zero will change this behaviour to resemble that of a basic Channel15.12), where extraneous data are not permitted by default, but this will probably rarely be useful.

16.13 Finding and Changing Cards in a FitsChan

You can search for, and retrieve, particular cards in a FitsChan by keyword, using the function astFindFits. This performs a search, starting at the current card, until it finds a card whose keyword matches the template you supply, or the end-of-file is reached.

If a suitable card is found, astFindFits optionally returns the card’s contents and then sets the FitsChan’s Card attribute either to identify the card found, or the one following it. The way you want the Card attribute to be set is indicated by the final boolean (int) argument to astFindFits. A value of one is returned to indicate success. If a suitable card cannot be found, astFindFits returns a value of zero to indicate failure and sets the FitsChan’s Card attribute to the end-of-file.

Requesting that the Card attribute be set to indicate the card that astFindFits finds is useful if you want to replace that card with a new one, as in this example:

char newcard[ 81 ];

...

(void) astFindFits( fitschan, "AIRMASS", NULL, 0 );
astPutFits( fitschan, newcard, 1 );

Here, astFindFits is used to search for a card with the keyword AIRMASS, with a NULL pointer being given to indicate that we do not want the card’s contents returned. If the card is found, astPutFits then overwrites it with a new card. Otherwise, the Card attribute ends up pointing at the end-of-file and the new card is simply appended to the end of the FitsChan.

A similar approach can be used to delete selected cards from a FitsChan using astDelFits, which deletes the current card:

if ( astFindFits( fitschan, "BSCALE", NULL, 0 ) ) astDelFits( fitschan );

This deletes the first card, if any, with the BSCALE keyword.

Requesting that astFindFits increments the Card attribute to identify the card following the one found is more useful when writing loops. For example, the following loop extracts each card whose keyword matches the template “CD%6d” (that is, “CD” followed by six decimal digits):

while ( astFindFits( fitschan, "CD%6d", card, 1 ) {
   <process the card’s contents>
}

For further details of keyword templates, see the description of astFindFits in Appendix B.

16.14 Source and Sink Functions for FitsChans

The use of source and sink functions with a FitsChan is optional. This is because you can always arrange to explicitly fill a FitsChan with FITS cards (§16.8 and §16.9) and you can also extract any cards that remain and write them out yourself (§16.6) before you delete the FitsChan.

If you choose to use these functions, however, they behave in a very similar manner to those used by a Channel15.13 and §15.14). You supply pointers to these functions, as arguments to the constructor function astFitsChan when you create the FitsChan (§16.3). The source function is invoked implicitly at this point to fill the FitsChan with FITS cards and the FitsChan is then rewound, so that the first card becomes current. The sink function is automatically invoked later, when the FitsChan is deleted, in order to write out any cards that remain in it.

The only real difference between the source and sink functions for a FitsChan and a basic Channel is that FITS cards are limited in length to 80 characters, so the choice of buffer size is simplified. The “Source” and “Sink” functions in §15.13 and §15.14 could therefore be used to access FITS headers stored in text files simply by changing LEN to be 80. If you were not accessing a text file, however, appropriate changes to the I/O statements would be needed since the separating newline characters would be absent. The details obviously depend on the format of the file you are handling, which need not necessarily be a true FITS file.

24http://fits.gsfc.nasa.gov/

25More probably, you would want to write a FrameSet, but for purposes of illustration a SkyFrame contains a more manageable amount of data.