Converts NDFs into FITS files NDF2FITS
You can select which NDF array components to export to the FITS file, and choose the data type of the data and variance arrays. You can control whether or not to propagate extensions and history information.
The application also accepts NDFs stored as top-level components of an HDS container file.
Both NDF and FITS use the term extension, and they mean different things. Thus to avoid confusion in the descriptions below, the term ‘sub-file’ is used to refer to a FITS IMAGE, TABLE or BINTABLE Header and Data Unit (HDU).
TRUE
, tables of world co-ordinates
may be written using the TAB algorithm as defined in the FITS-WCS Paper III. Examples
where such a table might be present in the WCS include wavelengths of pre-scrunched
spectra, and the presence of distortions that prevent co-ordinates being defined by analytical
expressions. Since many FITS readers are yet to support the TAB algorithm, which uses a FITS
binary-table extension to store the co-ordinates, this parameter permits this facility to be
disabled. [TRUE]
!
), "Copy"
or a space-separated list of axis
symbols (case insensitive). If it is null, the order is determined automatically so that the
th
WCS axis is the WCS axis that is most nearly parallel to the
th pixel axis. If it is "Copy"
,
the th WCS axis in the
FITS header is the th
WCS axis in the NDF’s current WCS Frame. Otherwise, the string must be a space-separated
list of axis symbols that gives the order for the WCS axes. An error is reported if the list
does not contain any of the axis symbols present in the current WCS Frame, but no error is
reported if the list also contains other symbols. [!]
8
for unsigned byte, 16
for signed word, 32
for integer, 64
for
64-bit integer, -32
for real, -64
for double precision. There are three other special values.
BITPIX=0
will cause the output file to have the data type equivalent to that of the input
NDF.
BITPIX=-1
requests that the output file has the data type corresponding to the value of the BITPIX
keyword in the NDF’s FITS extension. If the extension or BITPIX keyword is absent, the output file
takes the data type of the input array.
BITPIX="Native"
requests that any scaled arrays in the NDF be copied to the scaled data type.
Otherwise behaviour reverts to BITPIX=-1
, which may in turn be effectively BITPIX=0
. The
case-insensitive value may be abbreviated to "n"
.
BITPIX values must be enclosed in double quotes and may be a list of comma-separated values to be
applied to each conversion in turn. An error results if more values than the number of input NDFs are
supplied. If too few are given, the last value in the list is applied to the remainder of the
NDFs; thus a single value is applied to all the conversions. The given values must be in
the same order as that of the input NDFs. Indirection through a text file may be used. If
more than one line is required to enter the information at a prompt then type a "-"
at the
end of each line where a continuation line is desired. [0]
TRUE
, each header and data unit in the FITS file will contain the integrity-check
keywords CHECKSUM and DATASUM immediately before the END card. [TRUE]
"D"
for the main data array "V"
for variance, "Q"
for quality, or any
permutation thereof. The special value "A"
means all components, i.e. COMP="DVQ"
. Thus
COMP="VD"
requests that both the data array and variance are to be converted if present.
During processing at least one, if not all, of the requested components must be present,
otherwise an error is reported and processing turns to the next input NDF. If the DATA
component is in the list, it will always be processed first into the FITS primary array. The
order of the variance and quality in COMP decides the order they will appear in the FITS
file.
The choice of COMP may affect automatic quality masking. See "
Quality Masking"
for the
details.
COMP may be a list of comma-separated values to be applied to each conversion in turn. The list must
be enclosed in double quotes. An error results if more values than the number of input NDFs are
supplied. If too few are given, the last value in the list is applied to the remainder of the NDFs; thus a
single value is applied to all the conversions. The given values must be in the same order as that of the
input NDFs. Indirection through a text file may be used. If more than one line is required to enter the
information at a prompt then type a "-"
at the end of each line where a continuation line is
desired. ["A"]
TRUE
, the supplied IN files are any
multi-NDF HDS container files, in which the NDFs reside as top-level components. This
option is primarily intended to support the UKIRT format where the NDFs are named
.I,
,
and one named HEADER containing global metadata in its FITS airlock. The
.I
NDFs may also contain FITS airlocks, storing metadata pertinent to that NDF, such as
observation times. The individual NDFs often represent separate integrations nodded
along a slit or spatially. Note that this is not a group, so a single value applies to all the
supplied input files. [FALSE]
TRUE
. DUPLEX=FALSE
means that the airlock headers only appear with the primary array.
DUPLEX=TRUE
, propagates the FITS airlock headers for other array components of the NDF.
[FALSE]
"
World Co-ordinate Systems"
section below. In addition, the value "Auto"
may also be supplied, in which case a suitable
default encoding is chosen based on the contents of the NDF’s FITS extension and WCS
component. ["Auto"]
"*"
, "?"
).
Indirection may occur through text files (nested up to seven deep). The indirection character is
""
. If
extra prompt lines are required, append the continuation character "-"
to the end of the line.
Comments in the indirection file begin with the character "#"
. TRUE
value is given for Parameter NATIVE, then World Co-ordinate System (WCS) information will be
written to the FITS header in the form of a ‘native’ encoding (see "
World Co-ordinate Systems"
below). This will be in addition to the encoding specified using Parameter ENCODING, and will
usually result in two descriptions of the WCS information being stored in the FITS header (unless
ENCODING parameter produces a native encoding in which case only one native encoding is stored
in the header). Including a native encoding in the header will enable other AST-based software (such
as FITS2NDF) to reconstruct the full details of the WCS information. The other non-native encodings
will usually result in some information being lost. [FALSE]
TRUE
; and when the container file only has two
component NDFs: one data NDF of arbitrary name, and the other called HEADER that stores the
global headers of the dataset. [TRUE]
"Starlink Software"
. [!]
""
. This
character is a token which represents the name of the corresponding input NDF, but with a file type of
".fit"
instead of ".sdf"
, and with no directory specification. Thus, simply supplying "*"
for
this parameter will create a group of output files in the current directory with the same
names as the input NDFs, but with file type ".fit"
. You can also specify some simple
editing to be performed. For instance, "new-*|.fit|.fits|"
will add the string "new-"
to
the start of every file name, and will substitute the string ".fits"
for the original string
".fit"
.
NDF2FITS will not permit you to overwrite an existing FITS file, unless you supply an exclamation mark prefix (suitably escaped if you are using a UNIX shell).
TRUE
, the NDF extensions (other than the FITS extension) are propagated to the FITS files as FITS
binary-table sub-files, one per structure of the hierarchy. [FALSE]
TRUE
, the contents of the FITS extension of the NDF are merged with the header information derived
from the standard NDF components. See the "
Notes"
for details of the merger. [TRUE]
TRUE
, any NDF history records are written to the primary FITS header as
HISTORY cards. These follow the mandatory headers and any merged FITS-extension headers
(see Parameter PROFITS). [TRUE]
"None"
— No provenance is written.
"CADC"
— The CADC headers are written. These record the number and paths of both the direct
parents of the NDF being converted, and its root ancestors (the ones without parents). It also modifies
the PRODUCT keyword to be unique for each FITS sub-file.
"Generic"
— Encapsulates the entire PROVENANCE structure in FITS headers in sets of five
character-value indexed headers. there is a set for the current NDF and each parent. See Section
"
Provenance"
for more details.
["None"]
TRUE
or FALSE
selection demands the chosen setting irrespective of how the current NDF stores co-ordinate
information. [!]
logo.fit
. The data type of the FITS primary data array matches that
of the NDF’s data array. The FITS extension in the NDF is merged into the FITS header
of logo.fit
. logo.fit
. An existing logo.fit
will be overwritten. The data type of
the FITS primary data array matches that of the NDF’s data array. The FITS extension in
the NDF is merged into the FITS header of logo.fit
. In addition any NDF extensions
(apart from FITS) are turned into binary tables that follow the primary header and data
unit. logo.fit
. The data type of the FITS primary data array matches that of
the NDF’s data array. The FITS extension in the NDF is merged into the FITS header of
logo.fit
. Should horse contain variance and quality arrays, these are written in IMAGE
sub-files. Any history information in the NDF is not relayed to the FITS file. "a"
and ending in
"z"
in the directory called data
into FITS files of the same name and with a file extension of ".fit"
.
The variance array becomes the data array of each new FITS file. The data type of the FITS primary
data array single-precision floating point. Any FITS extension in the NDF is ignored. jvp1.fit
and jvp2.fit
respectively. The data type of
the FITS primary data array is signed integer words in jvp1.fit
, and double-precision
floating point in jvp2.fit
. The FITS extension in each NDF is merged into the FITS header of
the corresponding FITS file. u20040730_00675.sdf
to new FITS file u20040730_00675.fit
, merging its .I1 and
.HEADER structures into a single NDF before the conversion. The output file has only
one header and data unit. c20011204_00016.sdf
to the multiple-extension FITS file
cgs4_16.fit
. The primary HDU has the global metadata from the .HEADER’s FITS airlock. The
four integrations in I1, I2, I3, and I4 components of the container file are converted to FITS
IMAGE sub-files. huge.fits
. The data type of the FITS primary
data array matches that of the NDF’s scaled data array. The scale and offset coefficients
used to form the FITS array are also taken from the NDF’s scaled array. The NDF main data array becomes the primary data array of the FITS file if it is in value of Parameter COMP, otherwise the first array defined by Parameter COMP will become the primary data array. A conversion from floating point to integer or to a shorter integer type will cause the output array to be scaled and offset, the values being recorded in keywords BSCALE and BZERO. There is an offset (keyword BZERO) applied to signed byte and unsigned word types to make them unsigned-byte and signed-word values respectively in the FITS array (this is because FITS does not support these data types).
The FITS keyword BLANK records the bad values for integer output types. Bad values in floating-point output arrays are denoted by IEEE not-a-number values.
The NDF’s quality and variance arrays appear in individual FITS IMAGE sub-files immediately following the primary header and data unit, unless that component already appears as the primary data array. The quality array will always be written as an unsigned-byte array in the FITS file, regardless of the value of the Parameter BITPIX.
Here are details of the processing of standard items from the NDF into the FITS header, listed by FITS keyword.
SIMPLE, EXTEND, PCOUNT, GCOUNT — all take their default values.
BITPIX, NAXIS, NAXISn — are derived directly from the NDF data array; however the BITPIX in the
FITS airlock extension is transferred when Parameter BITPIX=-1
.
CRVALn, CDELTn, CRPIXn, CTYPEn, CUNITn — are derived from the NDF WCS component if
possible (see "
World Co-ordinate Systems"
). If this is not possible, and if PROFITS is TRUE
, then it
copies the headers of a valid WCS specified in the NDF’s FITS airlock. Should that attempt fail, the last
resort tries the NDF AXIS component, if it exists. If its co-ordinates are non-linear, the
AXIS co-ordinates may be exported in a -TAB sub-file subject to the value of Parameter
USEAXIS.
OBJECT, LABEL, BUNIT — the values held in the NDF’s TITLE, LABEL, and UNITS components
respectively are used if they are defined; otherwise any values found in the FITS extension are
used (provided Parameter PROFITS is TRUE
). For a variance array, BUNIT is assigned to
()**2
,
where
is the DATA unit; the BUNIT header is absent for a quality array.
DATE — is created automatically.
ORIGIN — inherits any existing ORIGIN card in the NDF FITS extension, unless you supply a value
through parameter ORIGIN other than the default "Starlink Software"
.
EXTNAME — is the array-component name when the EXTNAME appears in the primary header or an IMAGE sub-file. In a binary-table derived from an NDF extension, EXTNAME is the path of the extension within the NDF, the path separator being the usual dot. The path includes the indices to elements of any array structures present; the indices are in a comma-separated list within parentheses.
If the component is too long to fit within the header (68 characters), EXTNAME is set to @EXTNAMEF
.
The full path is then stored in keyword EXTNAMEF using the HEASARC Long-string CONTINUE
convention
(http://fits.gsfc.nasa.gov/registry/continue_keyword.html
).
EXTVER — is only set when EXTNAME (q.v.) cannot accommodate the component name, and it is assigned the HDU index to provide a unique identifier.
EXTLEVEL — is the level in the hierarchical structure of the extension. Thus a top-level extension has value 1, sub-components of this extension have value 2 and so on.
EXTTYPE — is the data type of the NDF extension used to create a binary table.
EXTSHAPE — is the shape of the NDF extension used to create a binary table. It is a comma-separated list of the dimensions, and is 0 when the extension is not an array.
HDUCLAS1, HDUCLASn — "NDF"
and the array-component name respectively.
LBOUNDn — is the pixel origin for the dimension when any of the pixel origins is not equal to 1. (This is not a standard FITS keyword.)
XTENSION, BSCALE, BZERO, BLANK and END — are not propagated from the NDF’s FITS extension. XTENSION will be set for any sub-file. BSCALE and BZERO will be defined based on the chosen output data type in comparison with the NDF array’s type, but cards with values 1.0 and 0.0 respectively are written to reserve places in the header section. These ‘reservation’ cards are for efficiency and they can always be deleted later. BLANK is set to the Starlink standard bad value corresponding to the type specified by BITPIX, but only for integer types and not for the quality array. It appears regardless of whether or not there are bad values actually present in the array; this is for the same efficiency reasons as before. The END card terminates the FITS header.
HISTORY — HISTORY headers are propagated from the FITS airlock when PROFITS is TRUE
, and
from the NDF history component when PROHIS is TRUE
.
DATASUM and CHECKSUM — data-integrity keywords are written when Parameter CHECKSUM is
TRUE
, replacing any existing values. When Parameter CHECKSUM is FALSE
and PROFITS is TRUE
any
existing values inherited from the FITS airlock are removed to prevent storage of invalid checksums
relating to another data file.
See also the sections "
Provenance"
and "
World Co-ordinate Systems"
for details of headers used to
describe the PROVENANCE extension and WCS information respectively.
Extension information may be transferred to the FITS file when PROEXTS is TRUE
. The whole
hierarchy of extensions is propagated in order. This includes substructures, and arrays of extensions
and substructures. However, at present, any extension structure containing only substructures is not
propagated itself (as zero-column tables are not permitted), although its substructures may be
converted.
Each extension or substructure creates a one-row binary table, where the columns of the table correspond to the primitive (non-structure) components. The name of each column is the component name. The column order is the same as the component order. The shapes of multi-dimensional arrays are recorded using the TDIMn keyword, where n is the column number. The HEASARCH convention for specifying the width of character arrays (keyword TFORMn=’rAw’, where r is the total number of characters in the column and w is the width of an element) is used. The EXTNAME, EXTTYPE, EXTSHAPE and EXTLEVEL keywords (see above) are written to the binary-table header.
There are additional rules if a multi-NDF container file is being converted (see Parameter CONTAINER). This excludes the case where there are but two NDFs—one data and the other just headers—that have already been merged (see Parameter MERGE):
For multiple NDFs a header-only HDU may be created followed by an IMAGE sub-file containing the data array (or whichever other array is first specified by COMP).
I2
. NDF
for a component NDF in a multi-NDF container file. "FITS-IRAF"
— This uses keywords CRVALi, CRPIXi, and CDi_j, and is the system commonly used
by IRAF. It is described in the document World Coordinate Systems Representations Within the
FITS Format by by R.J. Hanisch and D.G. Wells, 1988, available by ftp from fits.cv.nrao.edu
/fits/documents/wcs/wcs88.ps.Z
.
"FITS-WCS"
— This is the FITS standard WCS encoding scheme described in the paper Representation
of celestial coordinates in FITS.
(http://www.atnf.csiro.au/people/mcalabre/WCS/
) It is very similar to "FITS-IRAF"
but supports
a wider range of projections and co-ordinate systems.
"FITS-WCS(CD)"
— This is the same as "FITS-WCS"
except that the scaling and rotation of the
data array is described by a CD matrix instead of a PC matrix with associated CDELT
values.
"FITS-PC"
— This uses keywords CRVALi, CDELTi, CRPIXi, PCiiijjj, etc, as described in a previous
(now superseded) draft of the above FITS world co-ordinate system paper by E.W.Greisen and
M.Calabretta.
"FITS-AIPS"
— This uses conventions described in the document "Non-linear Coordinate Systems in
AIPS" by Eric W. Greisen (revised 9th September, 1994), available by ftp from fits.cv.nrao.edu
/fits/documents/wcs/aips27.ps.Z.
It is currently employed by the AIPS data-analysis facility
(amongst others), so its use will facilitate data exchange with AIPS. This encoding uses CROTAi and
CDELTi keywords to describe axis rotation and scaling.
"FITS-AIPS++"
— This is an extension to FITS-AIPS which allows the use of a wider range of celestial
as used by the AIPS++ project.
"FITS-CLASS"
— This uses the conventions of the CLASS project. CLASS is a software package for
reducing single-dish radio and sub-mm spectroscopic data. It supports double-sideband spectra. See
the GILDAS manual.
"DSS"
— This is the system used by the Digital Sky Survey, and uses keywords AMDXn, AMDYn,
PLTRAH, etc.
"NATIVE"
— This is the native system used by the AST library (see SUN/210) and provides a loss-free
method for transferring WCS information between AST-based applications. It allows more
complicated WCS information to be stored and retrieved than any of the other encodings.
Values for FITS keywords generated by the above encodings will always be used in preference to any
corresponding keywords found in the FITS extension (even if PROFITS is TRUE
). If this is not what is
required, the WCS component of the NDF should be erased using the Kappa command ERASE before
running NDF2FITS. Note, if PROFITS is TRUE
, then any WCS-related keywords in the FITS extension
which are not replaced by keywords derived from the WCS component may appear in the output
FITS file. If this causes a problem, then PROFITS should be set to FALSE
or the offending
keywords removed using Kappa FITSEDIT for example.
"Generic"
.
PRVP — is the path of the th NDF.
PRVIn — is a comma-separated list of the identifiers of the direct parents for the th ancestor.
PRVDn — is the creation date in ISO order of the th ancestor.
PRVCn — is the software used to create the th ancestor
PRVMn — lists the contents of the MORE structure of the th parent.
<unknown>
if the information could not be found, except for the
PRVM
header, which is omitted if there is no MORE information to record. The index
used in
each keyword’s name is the provenance identifier for the NDF, and starts at 0 for the NDF being
converted to FITS.
The following PROVENANCE headers are written if parameter PROVENANCE is set to "CADC"
.
PRVCNT — is the number of immediate parents.
PRV — is name of the th immediate parent.
OBSCNT — is the number of root ancestor OBS headers.
OBS — is th root ancestor identifier from its MORE.OBSIDSS component.
FILEID — is the name of the output FITS file.
PRODUCT is modified or added to each sub-file’s header to be the primary header’s value of PRODUCT with a
_extnam
suffix, where extnam
is the extension name in lowercase.
When PROFITS is TRUE
any existing provenance keywords in the FITS airlock are not copied to the
FITS file.
’A’
or contains ’Q’
. TRUE
. This preserves the information, but it may
not be accessible to the recipient’s FITS reader. Therefore, in some cases it is desirable to
understand the meanings of certain NDF extensions, and create standard FITS products for
compatibility.
At present only one product is supported, but others may be added as required.
AAO 2dF
Standard processing is used except for the 2dF FIBRES extension and its constituent structures. The NDF may be restored from the created FITS file using FITS2NDF. The FIBRES extension converts to the second binary table in the FITS file (the NDF_CLASS extension appears in the first).
To propagate the OBJECT substructure, NDF2FITS creates a binary table of constant width (224 bytes) with one row per fibre. The total number of rows is obtained from component NUM_FIBRES. If a possible OBJECT component is missing from the NDF, a null column is written for that component. The columns inherit the data types of the OBJECT structure’s components. Column meanings and units are assigned based upon information in the reference given below.
The FIELD structure components are converted into additional keywords of the same name in the binary-table header, with the exception that components with names longer than eight characters have abbreviated keywords: UNALLOCxxx becomes UNAL-xxx (xxx=OBJ, GUI, or SKY), CONFIGMJD becomes CONFMJD, and xSWITCHOFF becomes xSWTCHOF (x=X or Y). If any FIELD component is missing it is ignored.
Keywords for the extension level, name, and type appear in the binary-table header.
JCMT SMURF
Standard processing is used except for the SMURF-type extension. This contains NDFs such as
EXP_TIME and TSYS. Each such NDF is treated like the main NDF except that it is assumed that these
extension NDFs have no extensions of their own. FITS airlock information and HISTORY are inherited
from the parent NDF. Also the extension keywords are written: EXTNAME gives the path to the NDF,
EXTLEVEL records the extension hierarchy level, and EXTTYPE is set to "NDF"
. Any non-NDF
components of the SMURF extension are written to a binary table in the normal fashion.
Bailey, J.A. 1997, 2dF Software Report 14, version 0.5.
NASA Office of Standards and Technology, 1994, A User’s Guide for the Flexible Image Transport System (FITS), version 3.1.
NASA Office of Standards and Technology, 1995, Definition of the Flexible Image Transport System (FITS), version 1.1.
All NDF data types are supported.