17 Using Foreign FITS Encodings

 17.1 The Foreign FITS Encodings
 17.2 Limitations of Foreign Encodings
 17.3 Identifying Foreign Encodings on Input
 17.4 Reading Foreign WCS Information from a FITS Header
 17.5 Removing WCS Information from FITS Headers—the Destructive Read
 17.6 Propagating WCS Information through Data Processing Steps
 17.7 Writing Foreign WCS Information to a FITS Header

We saw in the previous section (§16) how to store and retrieve any kind of AST Object in a FITS header by using a FitsChan. To achieve this, we set the FitsChan’s Encoding attribute to NATIVE. However, the Objects we wrote could then only be read back by other programs that use AST.

In practice, we will also encounter FITS headers containing WCS information written by other software systems. We will probably also need to write FITS headers in a format that can be understood by these systems. Indeed, this interchange of data is one of the main reasons for the existence of FITS, so in this section we will examine how to accommodate these requirements.

17.1 The Foreign FITS Encodings

As mentioned previously (§16.1), there are a number of conventions currently in use for storing WCS information in FITS headers, which we call encodings. Here, we are concerned with those encodings defined by software systems other than AST, which we term foreign encodings.

Currently, AST supports six foreign encodings, which may be selected by setting the Encoding attribute of a FitsChan to one of the following (character string) values:

DSS

This encoding stores WCS information using the convention developed at the Space Telescope Science Institute for the Digitised Sky Survey (DSS) astrometric plate calibrations. DSS images which use this convention are widely available and it is understood by a number of important and well-established astronomy applications.

However, the calibration model used (based on a polynomial fit) is not easily applicable to other types of data and creating the polynomial coefficients needed to calibrate your own images can prove difficult. For this reason, the DSS encoding is probably best viewed as a “read-only” format. It is possible, however, to read in WCS information using this encoding and then to write it back out again, so long as only minor changes have been made.

FITS-WCS

This encoding is very important because it is based on a new FITS standard which should, for the first time, address the problem of celestial coordinate systems in a proper manner, by considerably extending the original FITS standard.

The conventions used are described in a series of papers by E.W. Greisen, M. Calabretta, et. al., often referred to as the “FITS-WCS papers”. They are described at http://fits.gsfc.nasa.gov/fits_wcs.html. Now that the first two papers in this series have been agreed, this encoding should be understood by any FITS-WCS compliant software and it is likely to be adopted widely for FITS data in future. For details of the coverage of these conventions provided by the FitsChan class, see Appendix F.

FITS-IRAF

This encoding is based on the conventions described in the document “World Coordinate Systems Representations Within the FITS Format” by R.J. Hanisch and D.G. Wells, 1988.29 It is employed by the IRAF data analysis facility, so its use will facilitate data exchange with IRAF. This encoding is in effect a sub-set of the current FITS-WCS encoding.
FITS-PC

This encoding is based on a previous version of the proposed new FITS WCS standard which used PCjjjjiii and CDELTj keywords to describe axis rotation and scaling. Versions of AST prior to V1.5 used this scheme for the FITS-WCS encoding. As of V1.5, FITS-WCS uses CDi_j keywords instead.30 The FITS-PC encoding is included in AST V1.5 only to allow FITS-WCS data created with previous versions to be read. It should not, in general, be used to create new data sets.
FITS-AIPS

This encoding is based on the conventions described in the document “Non-linear Coordinate Systems in AIPS” by Eric W. Greisen (revised 9th September, 1994).31 It is currently employed by the AIPS data analysis facility, so its use will facilitate data exchange with AIPS. This encoding uses CROTAi and CDELTi keywords to describe axis rotation and scaling.
FITS-AIPS++

Encodes coordinate system information in FITS header cards using the conventions used by the AIPS++ project. This is an extension of FITS-AIPS which includes some of the features of FITS-PC and FITS-IRAF.

For more detail about the above encodings, see the description of the Encoding attribute in Appendix C.

17.2 Limitations of Foreign Encodings

The foreign encodings available for storing WCS information in FITS headers have a number of limitations when compared with the native encoding of AST Objects (§16). The main ones are:

(1)
Only one class of AST Object, the FrameSet, may be represented using a foreign FITS encoding. This should not come as a surprise, because the purpose of storing WCS information in FITS headers is to attach coordinate systems to an associated array of data. Since the FrameSet is the AST Object designed for the same purpose (§13.4), there is a natural correspondence.

The way in which a FrameSet is translated to and from the foreign encoding also follows from this correspondence. The FrameSet’s base Frame identifies the data grid coordinates of the associated FITS data. These are the same as FITS pixel coordinates, in which the first pixel (in 2 dimensions) has coordinates (1,1) at its centre. Similarly, the current Frame of the FrameSet identifies the FITS world coordinate system associated with the data.

(2)
You may store a representation of only a single FrameSet in any individual set of FITS header cards (i.e. in a single FitsChan) at one time. If you attempt to store more than one, you may over-write the previous one or generate an invalid representation of your WCS information.

This is mainly a consequence of the use of fixed FITS keywords by foreign encodings and the fact that you cannot, in general, have multiple FITS cards with the same keyword.

(3)
In general, it will not be possible to store every possible FrameSet that you might construct. Depending on the encoding, only certain FrameSets that conform to particular restrictions can be represented and, even then, some of their information may be lost. See the description of the Encoding attribute in Appendix C for more details of these limitations.

It should be understood that using foreign encodings to read and write information held in AST Objects is essentially a process of converting the data format. As such, it potentially suffers from the same problems faced by all such processes, i.e. differences between the AST data model and that of the foreign encoding may cause some information to be lost. Because the AST model is extremely flexible, however, any data loss can largely be eliminated when reading. Instead, this effect manifests itself in the form of the above encoding-dependent restrictions on the kind of AST Objects which may be written.

One of the aims of the AST library, of course, is to insulate you from the details of these foreign encodings and the restrictions they impose. We will see shortly, therefore, how AST provides a mechanism for determining whether your WCS information satisfies the necessary conditions and allows you to make an automatic choice of which encoding to use.

17.3 Identifying Foreign Encodings on Input

Let us now examine the practicalities of extracting WCS information from a set of FITS header cards which have been written by some other software system. We will pretend that our program does not know which encoding has been used for the WCS information and must discover this for itself. In order to have a concrete example, however, we will use the following set of cards. These use the FITS-AIPS encoding and contain a typical mix of other FITS cards which are irrelevant to the WCS information in which we are interested:

  SIMPLE  =                    T / Written by IDL:  30-Jul-1997 05:35:42.00
  BITPIX  =                  -32 / Bits per pixel.
  NAXIS   =                    2 / Number of dimensions
  NAXIS1  =                  300 / Length of x axis.
  NAXIS2  =                  300 / Length of y axis.
  CTYPE1  = ’GLON-ZEA’           / X-axis type
  CTYPE2  = ’GLAT-ZEA’           / Y-axis type
  CRVAL1  =           -149.56866 / Reference pixel value
  CRVAL2  =           -19.758201 / Reference pixel value
  CRPIX1  =              150.500 / Reference pixel
  CRPIX2  =              150.500 / Reference pixel
  CDELT1  =             -1.20000 / Degrees/pixel
  CDELT2  =              1.20000 / Degrees/pixel
  CROTA1  =              0.00000 / Rotation in degrees.
  SURVEY  = ’COBE DIRBE’
  BUNITS  = ’MJy/sr  ’           /
  ORIGIN  = ’CDAC    ’           / Cosmology Data Analysis Center
  TELESCOP= ’COBE    ’           / COsmic Background Explorer satellite
  INSTRUME= ’DIRBE   ’           / COBE instrument [DIRBE, DMR, FIRAS]
  PIXRESOL=                    9 / Quad tree pixel resolution [6, 9]
  DATE    = ’27/09/94’           / FITS file creation date (dd/mm/yy)
  DATE-MAP= ’16/09/94’           / Date of original file creation (dd/mm/yy)
  COMMENT     COBE specific keywords
  DATE-BEG= ’08/12/89’           / date of initial data represented (dd/mm/yy)
  DATE-END= ’25/09/90’           / date of final data represented   (dd/mm/yy)

The first step is to create a FitsChan and insert these cards into it. If CARDS is an array of character strings holding the header cards and NCARDS is the number of cards, this could be done as follows:

        INCLUDE ’AST_PAR’
        INTEGER FITSCHAN, ICARD, NCARD, STATUS
        CHARACTER * ( 80 ) CARDS( NCARD )
  
        STATUS = 0
  
        ...
  
        FITSCHAN = AST_FITSCHAN( AST_NULL, AST_NULL, ’ ’, STATUS )
        DO 1 ICARD = 1, NCARD
           CALL AST_PUTFITS( FITSCHAN, CARDS( ICARD ), .FALSE., STATUS )
   1    CONTINUE

Note that we have not initialised the Encoding attribute of the FitsChan as we did in §16.3 when we wanted to use the native encoding. This is because we are pretending not to know which encoding to use and want AST to determine this for us. By leaving the Encoding attribute un-set, its default value will adjust to whichever encoding AST considers to be most appropriate, according to the FITS header cards present. For details of how this choice is made, see the description of the Encoding attribute in Appendix C.

This approach has the obvious advantages of making our program simpler and more flexible and of freeing us from having to know about the different encodings available. As a bonus, it also means that the program will be able to read any new encodings that AST may support in future, without needing to be changed.

At this point, we could enquire the default value of the Encoding attribute, which indicates which encoding AST intends to use, as follows:

        CHARACTER * ( 20 ) ENCODE
  
        ...
  
        ENCODE = AST_GETC( FITSCHAN, ’Encoding’, STATUS )

The result of this enquiry would be the string “FITS-AIPS”. Note that we could also have set the FitsChan’s Encoding attribute explicitly, such as when creating it:

        FITSCHAN = AST_FITSCHAN( AST_NULL, AST_NULL, ’Encoding=FITS-AIPS’, STATUS )

If we tried to read information using this encoding (§17.4), but failed, we could then change the encoding and try again. This would allow our program to take control of how the optimum choice of encoding is arrived at. However, it would also involve using explicit knowledge of the encodings available and this is best avoided if possible.

17.4 Reading Foreign WCS Information from a FITS Header

Having stored a set of FITS header cards in a FitsChan and determined how the WCS information is encoded (§17.3), the next step is to read an AST Object from the FitsChan using AST_READ. We must also remember to rewind the FitsChan first, if necessary, such as by clearing its Card attribute, which defaults to 1:

        INTEGER WCSINFO
  
        ...
  
        CALL AST_CLEAR( FITSCHAN, ’Card’, STATUS )
        WCSINFO = AST_READ( FITSCHAN, STATUS )

If the pointer returned by AST_READ is not equal to AST__NULL, then an Object has been read successfully. Otherwise, there was either no information to read or the choice of FITS encoding (§17.3) was inappropriate.

At this point you might like to indulge in a little data validation along the lines described in §15.6, for example:

        IF ( AST_GETC( WCSINFO, ’Class’, STATUS ) .EQ. ’FrameSet’ ) THEN
           <the Object is a FrameSet, so use it>
        ELSE
           <something unexpected was read>
        END IF

If a foreign encoding has definitely been used, then the Object will automatically be a FrameSet17.2), so this stage can be omitted. However, if the native encoding (§16.1) might have been employed, which is a possibility if you accept the FitsChan’s default Encoding value, then any class of Object might have been read and a quick check would be worthwhile.

If you used AST_SHOW4.4) to examine the FrameSet which results from reading our example FITS header (§17.3), you would find that its base Frame describes the image’s pixel coordinate system and that its current Frame is a SkyFrame representing galactic coordinates. These two Frames are inter-related by a Mapping (actually a CmpMap) which incorporates the effects of various rotations, scalings and a “zenithal equal area” sky projection, so that each pixel of the FITS image is mapped on to a corresponding sky position in galactic coordinates.

Because this FrameSet may be used both as a Mapping (§13.6) and as a Frame (§13.8), it may be employed directly to perform many useful operations without any need to decompose it into its component parts. These include:

If the FrameSet contains other Frames (apart from the base and current Frames), then you would also have access to information about other coordinate systems associated with the image.

17.5 Removing WCS Information from FITS Headers—the Destructive Read

It is instructive at this point to examine the contents of a FitsChan after we have read a FrameSet from it (§17.4). The following would rewind our FitsChan and display its contents:

        CHARACTER CARD * ( 80 )
  
        ...
  
        CALL AST_CLEAR( FITSCHAN, ’Card’, STATUS )
   2    CONTINUE
        IF ( AST_FINDFITS( FITSCHAN, ’%f’, CARD, .TRUE., STATUS ) ) THEN
           WRITE ( *, ’(A)’ ) CARD
           GO TO 2
        END IF

The output, if we started with the example FITS header in §17.3, might look like this:

  SIMPLE  =                    T /  Written by IDL:  30-Jul-1997 05:35:42.00
  BITPIX  =                  -32 /  Bits per pixel.
  NAXIS   =                    2 /  Number of dimensions
  NAXIS1  =                  300 /  Length of x axis.
  NAXIS2  =                  300 /  Length of y axis.
  SURVEY  = ’COBE DIRBE’
  BUNITS  = ’MJy/sr  ’
  ORIGIN  = ’CDAC    ’           /  Cosmology Data Analysis Center
  TELESCOP= ’COBE    ’           /  COsmic Background Explorer satellite
  INSTRUME= ’DIRBE   ’           /  COBE instrument [DIRBE, DMR, FIRAS]
  PIXRESOL=                    9 /  Quad tree pixel resolution [6, 9]
  DATE    = ’27/09/94’           /  FITS file creation date (dd/mm/yy)
  DATE-MAP= ’16/09/94’           /  Date of original file creation (dd/mm/yy)
  COMMENT     COBE specific keywords
  DATE-BEG= ’08/12/89’           /  date of initial data represented (dd/mm/yy)
  DATE-END= ’25/09/90’           /  date of final data represented   (dd/mm/yy)

Comparing this with the original, you can see that all the FITS cards that represent WCS information have been removed. They have effectively been “sucked out” of the FitsChan by the destructive read that AST_READ performs and converted into an equivalent FrameSet. AST remembers where they were stored, however, so that if we later write WCS information back into the FitsChan (§17.7) they will, as far as possible, go back into their original locations. This helps to preserve the overall layout of the FITS header.

You can now see why AST_READ performs destructive reads. It is a mechanism for removing WCS information from a FITS header while insulating you, as a programmer, from the details of the encoding being used. It means you can ensure that all relevant header cards have been removed, giving you a clean slate, without having to know which FITS keywords any particular encoding uses.

Clearing this WCS information out of a FITS header is particularly important when considering how to write new WCS information back after processing (§17.7). If any relevant FITS cards are left over from the input dataset and find their way into the new processed header, they could interfere with the new information being written.32 The destructive read mechanism ensures that this doesn’t happen.

17.6 Propagating WCS Information through Data Processing Steps

One of the purposes of AST is to make it feasible to propagate WCS information through successive stages of data processing, so that it remains consistent with the associated image data. As far as possible, this should happen regardless of the FITS encoding used to store the original WCS information.

If the data processing being performed does not change the relationship between image pixel and world coordinates (whatever these may be), then propagation of the WCS information is straightforward. You can simply copy the FITS header from input to output.

If this relationship changes, however, then the WCS information must be processed alongside the image data and a new FITS header generated to represent it. In this case, the sequence of operations within your program would probably be as follows:

(1)
Read the image data and associated FITS header from the input dataset, putting the header cards into a FitsChan17.3).
(2)
Read an AST Object, a FrameSet, from the FitsChan (typically using a foreign FITS encoding—§17.4).
(3)
Process the image data and modify the FrameSet accordingly (e.g. §14.5).
(4)
Write the FrameSet back into the FitsChan (§17.7).
(5)
Perform any other modification of FITS header cards your program may require.
(6)
Write the FitsChan contents (i.e. processed header cards) and image data to the output dataset.

In stage (2), the original WCS information will be removed from the FitsChan by a destructive read. Later, in stage (4), new WCS information is written to replace it. This is the process which we consider next (§17.7).

17.7 Writing Foreign WCS Information to a FITS Header

Before we can write processed WCS information held in a FrameSet back into a FitsChan in preparation for output, we must select the FITS encoding to use. Unfortunately, we cannot simply depend on the default value of the Encoding attribute, as we did when reading the input information (§17.3), because the destructive action of reading the WCS data (§17.5) will have altered the FitsChan’s contents. This, in turn, will have changed the choice of default encoding, probably causing it to revert to NATIVE.

We will return to the question of the optimum choice of encoding below. For now, let’s assume we want to use the same encoding for output as we used for input. Since we enquired what that was before we read the input WCS data from the FitsChan (§17.3), we can now set that value explicitly. We can also set the FitsChan’s Card attribute back to 1 at the same time (because the write will fail if the FitsChan is not rewound). AST_WRITE can then be used to write the output WCS information into the FitsChan:

        INTEGER NOBJ
  
        ...
  
  
        CALL AST_SET( FITSCHAN, ’Card=1, Encoding=’ // ENCODE, STATUS )
        NOBJ = AST_WRITE( FITSCHAN, WCSINFO, STATUS )

The value returned by AST_WRITE (assigned to NOBJ) indicates how many Objects were written. This will either be 1 or zero. A value of zero is used to indicate that the information could not be encoded in the form you requested. If this happens, nothing will have been written.

If your choice of encoding proves inadequate, the probable reason is that the changes you have made to the FrameSet have caused it to depart from the data model which the encoding assumes. AST knows about the data model used by each encoding and will attempt to simplify the FrameSet you provide so as to fit into that model, thus relieving you of the need to understand the details and limitations of each encoding yourself.33 When this attempt fails, however, you must consider what alternative encoding to use.

Ideally, you would probably want to try a sequence of alternative encodings, using an approach such as the following:

  *  1.
        CALL AST_SET( FITSCHAN, ’Card=1, Encoding=FITS-WCS’, STATUS )
        IF ( AST_WRITE( FITSCHAN, WCSINFO, STATUS ) .EQ. 0 ) THEN
  
  *  2.
           CALL AST_SETC( FITSCHAN, ’Encoding’, ENCODE, STATUS )
           IF ( AST_WRITE( FITSCHAN, WCSINFO, STATUS ) .EQ. 0 ) THEN
  
  *  3.
              CALL AST_SET( FITSCHAN, ’Encoding=NATIVE’, STATUS )
              NOBJ = AST_WRITE( FITSCHAN, WCSINFO, STATUS )
           END IF
        END IF

That is:

(1)
Start by trying the FITS-WCS encoding, on the grounds that FITS should provide a universal interchange standard in which all WCS information should be expressed if possible.
(2)
If that fails, then try the original encoding used for the input WCS information, on the grounds that you are at least not making the information any harder for others to read than it originally was.
(3)
If that also fails, then you are probably trying to store fairly complex information for which you need the native encoding. Only other AST programs will then be able to read this information, but these are probably the only programs that will be able to do anything sensible with it anyway.

An alternative approach might be to encode the WCS information in several ways, since this gives the maximum chance that other software will be able to read it. This approach is only possible if there is no significant conflict between the FITS keywords used by the different encodings34. Adopting this approach would simply require multiple calls to AST_WRITE, rewinding the FitsChan and changing its Encoding value before each one.

Unfortunately, however, there is a drawback to duplicating WCS information in the FITS header in this way, because any program which modifies one version of this information and simply copies the remainder of the header will risk producing two inconsistent sets of information. This could obviously be confusing to subsequent software. Whether you consider this a worthwhile risk probably depends on the use to which you expect your data to be put.

29Available by ftp from fits.cv.nrao.edu /fits/documents/wcs/wcs88.ps.Z

30There are many other differences between the previous and the current FITS-WCS encodings. The keywords to describe axis rotation and scaling is used purely as a label to identify the scheme.

31Available by ftp from fits.cv.nrao.edu /fits/documents/wcs/aips27.ps.Z

32This can happen if a particular keyword is present in the input header but is not used in the output header (whether particular keywords are used can depend on the WCS information being stored). In such a case, the original value would not be over-written by a new output value, so would remain erroneously present.

33Storing values in the FitsChan for FITS headers NAXIS1, NAXIS2, etc. (the grid dimensions in pixels), before invoking AST_WRITE can sometimes help to produce a successful write.

34In practice, this means you should avoid mixing FITS-IRAF, FITS-WCS, FITS-AIPS, FITS-AIPS++ and FITS-PC encodings since they share many keywords.