This appendix contains information on version 1.1 of the Specdre package (for SPEctroscopy Data Reduction) which has been added to Figaro. Sections B.6 and B.7 give examples of using several applications together.
Specdre is a package for spectroscopy data reduction and analysis. Some of the general features of the package are:
The topics addressed by the applications are mainly:
Specdre uses the NDF data access library, which allows you to specify sections rather than the whole data set. Also, for the special requirements of spectroscopy data reduction and analysis, an extension to the NDF format is used which stores additional information with the data, thus allowing much enhanced communication between Specdre applications.
Some parameters used by Specdre are common to several commands. The device parameter is
sometimes associated with the global parameter GRAPHICS_DEVICE. When it is, it usually defaults. And
these parameters are really global, in the sense that other packages may use and change them,
too.
Where in and/or out are NDFs, they are mostly associated with the global DATA_ARRAY. The effect is
that the default input is usually the output of the previous command.
info and dialog are always associated with SPECDRE_INFO and SPECDRE_DIALOG. These
parameters control the amount of information and user interaction of many applications. Once
info is switched off all applications will become quiet until the parameters are set true
again.
varuse is a defaulted parameter to many applications, but not associated with a global parameter. By
default it is true. Sometimes it has to be set false in order to ignore variance information in the input
data.
Other parameters like start, step, end occur naturally in several applications. In some instances
they may be scalars, in others vectors. Often their defaults are set by the application with knowledge
of the data set at hand.
The management of graphics output closely follows that of KAPPA. To make full use of the graphics capabilities, you will need to use some KAPPA commands; and you will find Pongo extremely useful to add almost anything to your plots. For normal use you can get along without KAPPA or Pongo. The actual plots are achieved through a combination of AGI, SGS, PGPLOT, and GKS. The graphics is done with PGPLOT, but the device is handled via AGI and SGS, and GKS is the low level package underlying them all. AGI stores information about the graphs produced in a data base. This data base tells co-operating applications where and on which device a plot was made and what its world coordinates were. Other packages use and update the same data base so that a consistent display administration can be achieved, even when different packages are used in turn. The overlay options in Specdre applications use this information and allow you to plot with Specdre in the right place on top of, for example, an image displayed in grey scale or colour with KAPPA.
Usually a command whose main task is to produce a plot, has a parameter device which is not
prompted for and which is associated with the global parameter GRAPHICS_DEVICE. The value of this
global parameter will be the name of a graphics device, and the named device will be used for the
display. But you can always specify the device parameter on the command line, thus overriding the
global parameter:
This will also change the global parameter so that next time you use the same device automatically.
There are other Specdre commands that may or may not use a graphics device. These will prompt for
the device parameter and offer the null value as default. This can be accepted to avoid using a
graphics device. When specifying parameters on the command line, device=! may not always work,
but including the accept keyword will have the desired effect.
Some applications not only display graphics, but use a graphics display plus mouse and keyboard to
conduct a dialogue with the user. This is usually optional and controlled by the dialog parameter.
dialog is a character parameter. It is always allowed to be one of the letters {Y,N,T,F,y,n,t,f}.
Sometimes it may also be G or g for graphics dialogue. y,t may or may not mean the same as
g.
If you specify a printer or PostScript device, this may fail for the graphics dialogue case. But otherwise
all plots can be sent to files that you print later. There is one important difference, you have only one
screen, but the printer has many sheets of paper. Your plot may be in a number of printer files and
each printout starts on a new page. If you have done overlay plots on the screen and want the same on
the printer, then you can use as graphics device an Encapsulated PostScript device like
epsf_l. You still get a number of files, but they can be merged into one (without form feed)
using psmerge (cf. SUN/164). Usually the output is a complete PostScript file ready to be
printed.
Specdre uses the NDF routines to access data in Starlink Data Files (SDF). This allows the user to
specify sections of NDFs rather than whole (i.e. base) NDFs. Some applications need one-dimensional
input data, but by themselves offer no means to take a subset of a larger or multi-dimensional data set.
The desired effect can, however, be achieved by NDF. As an example, fitgauss will fit a spectrum
only and rejects two-dimensional input. You still can use a 2-D data set by specifying a section thereof
as input to fitgauss:
(For fitgauss the sub-setting within the row is not necessary, because it does its own masking of
the given 1-D data.) It should be mentioned here that the NDF fed into an application
need not be at the top level of its container file. Once your NDF got a Specdre Extension
(Section 3.2.7) with fit results in it, you can e.g. plot the second fit parameter versus the row
number:
A spectrum may be thought of as a one-dimensional data set. But spectroscopists are also aware of the two-dimensional space of sky position and might use time as an axis in data sets. So the data handling aspects of spectroscopy are more demanding than image processing – Specdre applications may encounter data with any dimensionality. Two-dimensional detectors are often used to take spectra and where observations are not very time-consuming three-dimensional data sets may be quite common.
Specdre can work on data with any dimensionality. The limit is 7-D due to the HDS data access routines. In practice the limit may be 6-D since one structure in the Specdre Extension has one dimension more than the main data array.
However, Specdre is about spectra. An N-D cube is taken as a set of spectra, each spectrum is a row or a column in the cube. A row extends along the first axis of the cube, while a column extends along any axis of the cube. In any cube Specdre assumes that there is exactly one spectroscopic axis, by default this is the first axis. The Specdre Extension specifies which axis is the spectroscopic one.
Specdre’s handling of N-D data falls into three categories.
subset is very similar to KAPPA’s ndfcopy or to taking an “on the fly” section as input to an
application. The differences are that subset also takes subsets of NDFs in the Specdre Extension
(v. 0.7) consistent with the subset of the main NDF, and that it removes “degenerate” axes. Consider
the command
When subset gets the input it is still an image of size 1 by 10. But in the output the degenerate axis has
been removed so that it is also officially 1-D.
Where subset may remove axes, grow deliberately adds new axes – degenerate or genuine ones. So we
could reverse the command above with
The zeros as second vector elements just show that the second axis of image matches the axis in row.
expand shows which of the output axes are and are not in the input. Normally those new axes will of
course not be degenerate, so size might be [5,0]. In that case row could be copied into any of the
output columns, into one of them or repeatedly into a whole range of columns. The main
idea of grow is that you assemble rows into images, images into cubes etc. So when new=n
then you will copy into an existing file. The following puts two spectra and one image
into a common cube. Whatever part of the cube is not copied to, remains filled with bad
values.
If grow creates and expands new axes, xtract collapses existing axes. This is done by averaging
the pixels along each collapsed axis. Note that this is an average and not a sum, which
makes a big difference for the use of input variances and the meaning of output variances.
Basically an average assumes that all values entering the mean are equal and scatter at
random.
grow copies input into output of higher dimensionality, the common dimensions must match.
fillcube is different. It copies input into output of the same dimensionality. Dimensions need not
match, the copy is positioned in output by matching centre coordinates. Indeed the copy may not be
contiguous in output. The output is an existing file, so you can fill it successively from different input
files. This is mosaicing in N-D.
resample, too, plays a rôle in cube manipulation, since it can homogenise and linearise the
coordinates along the spectroscopic axis. When used in mode=Cube it re-samples each row of a
cube individually. Afterwards all rows have the same linear coordinate grid as expressed
by the new vector of centres for the spectroscopic axis. Any spectroscopic values in the
Specdre Extension are thus obsolete and removed. Sometimes it is necessary for other
applications that grids are linear or that there is no array of spectroscopic values in the Specdre
Extension.
Specdre has a number of applications to fit analytical functions to spectral features. Two are line fit routines for Gauss and triangular profiles. These can fit up to six components at a time. The lines can be blended and the line parameters can be free, fixed, or tied to the corresponding (free) parameters of another component. A similar routine fits up to six diluted Planck curves. Finally, a polynomial fit can be performed, the order can be up to 7.
The fit routines can run with a (non-graphic) dialogue or not, they can display data and fit graphically at different stages of the fitting process (masking, guessing, fit residuals).
These fit routines work on one-dimensional data only. But you can pass an NDF section that is (part of) a row or column in an image or cube. For the fit only data inside the union of up to six masking intervals are used. The fit results go first of all to the standard output (the terminal), but can also be recorded in (appended to) an ASCII file. In addition fit results will always be stored in a results’ NDF in the Specdre Extension. Those results can be used to generate a model data set. You could then subtract that from the original data.
Here is an outline of a complex example showing how you might proceed with a long-slit spectrum (but note that the example will not work as presented since some parameters are omitted for brevity). Also, the telluric correction may not be the correct way of doing things. The purpose of the example is to illustrate how you can inter-operate Specdre and KAPPA applications.
image. You would probably run the fit routine with dialogue
and graphics the first time round to play with the mask intervals.
fitgauss. For subsequent image rows you would try without dialogue and just recall the
command line to edit the row number.
sub.
cadd to turn the simple telluric lines into proper telluric spectra with
continuum at 1.0. Then divide the normalised spectra by the telluric spectra.There are four applications to do a wavelength calibration, or at least a calibration into spectroscopic values anyway. You can use frequencies or photon energies if you feel like it. You can also use nanometre, micrometre or Ångström, as you please.
At the heart of this axis calibration is an algorithm written by Mills (1992) to automatically identify
features in an arc spectrum. For this to work, you must have a data base of arc feature identifications
rather than just a simple line list. You can use arcgendb to convert a line list (as distributed with
Figaro) into a “feature data base”. Unfortunately the data base takes some time to build and is also
rather big, 10 to 100 times bigger than the simple list. So there may be a point in taking only the
relevant wavelength range from long line lists like Th-Ar and converting it into a feature data
base.
With such a data base at your disposal, you still cannot run the auto-identifier. This is
because the calibration procedure as performed by Figaro’s arc is broken up into three
steps:
arclocat. Usually this first step will be done in two passes. In the first pass you
run arclocat dialog=f so that it tries on its own to find un-blended narrow emission
features in the arc spectrum. In a second pass you run arclocat dialog=g and use the
graphic dialogue to improve the set of located features. You may add features not found
before, or delete features that are blended or known to be absent from the feature data
base. In general you should locate as many features as possible, you can always leave
some un-identified.
Instead of arclocat you can also use fitgauss or fittri, in case that the simplified line
fit in arclocat is not good enough. arclocat has modes Gauss and triangle, too, but it
fits only one line at a time.
arcident. The result must be regarded with some scepticism, since there
is a small chance that arcident will find a grossly wrong solution or make a slightly
unfavourable selection of features to identify. Any such inadvertencies can be corrected
in the third step.
arcdisp dialog=g will
display a plot of wavelength, frequency etc. versus pixel coordinate. It does not show
the spectrum. Instead vertical lines indicate unidentified (but located) features, horizontal
lines indicate all possible identifications from the feature data base, and crosses indicate
identified features. Finally the would-be dispersion curve is displayed. You can now add
or remove identifications with mouse and keyboard by clicking on the intersection of the
feature location (vertical line) and potential identification (horizontal line).
The improvement of feature identification is one goal of arcdisp. The other is to do
a polynomial fit to the pixel-wavelength relation and to convert pixel coordinates into
wavelengths using that fit. This latter goal can also be achieved without the graphics
dialogue with arcdisp dialog=n. This does not replace the pixel coordinates of the main
NDF, but creates a new array of spectroscopic values in the Specdre Extension.
arclocat, arcident and arcdisp in general work not on a spectrum, but on an image or cube
that has spectra in its rows (rows, not columns). arclocat dialog=g allows you to switch
from one row to another as you please, arclocat dialog=n will scan through all rows in
sequence. arcident will do an independent auto-identification on each spectrum, so it is
suitable for a collapsed échellogram where successive extracted orders are in the rows of an
image.
arcdisp dialog=g will work on each row in sequence, you can work on one row and proceed to the
next in your own time. You can quit at any time; this will leave the file without spectroscopic values,
but any improved feature identifications are kept. If you step through all rows, then the spectroscopic
values will be kept as well. (You can have spectroscopic values in the Specdre Extension either for all
rows or none at all!) You may want to set the label and unit for the spectroscopic values after arcdisp
with KAPPA’s setlabel and setunits.
So far you have only succeeded in producing an array of calibrated spectroscopic values in the
Specdre Extension of the arc spectrum. You will want to copy that array into the celestial observation
you are actually interested in. This can be done with ndfcopy, provided the sky spectrum does have a
Specdre Extension and does not have a SPECVALS component in it. That status can be achieved by two
editext commands. Finally you may want to re-sample each spectrum (row in a cube)
so that all use the same linear grid of spectroscopic values. To that end you use resamp
mode=cube. Here is the whole procedure. (The commands are not complete; parameters are
missing. Also there may be no Th-Ar line lists available for photon energies in the MeV
range.)