SURF
commandsThe SCUBA data reduction commands are summarized briefly in this section. On-line help for these commands can be accessed with scuhelp or by replying with ?? at any prompt.
There are six steps which need to be followed in order to produce the final coadded, but still uncalibrated, photometric result, namely reduce_switch, change_flat, flatfield, extinction, scuphot and scucat. In addition, data taken with the arrays may be corrected for sky noise variations by subtracting off the signal from surrounding bolometers using the remsky command. – see later in this section for a description of these commands.
The data reduction software takes the demodulated data as input and the format is yyyymmdd_dem_xxxx without the .sdf extension, for example, 19970711_dem_0025. Additional files called yyyymmdd_red_xxxx.sdf are produced by the on-line data reduction software and include preliminary signal information.
The SCUBA log command, sculog2, can be run to give a log of all the observations in a directory. All the journal software (sculog, obssum, photsum, etc.) as well as reduce_switch and skydip recognise the concept of a data directory. This means that the data (demodulated or reduced) does not need to be present in the current directory – the tasks will search for data in the directory specified by the datadir environment variable as well as the current directory. Using the unix C-shell this can be achieved by:
which would instruct obssum, say, to search for data in directory /wherever/data/19970706/dem
.
Help on the log commands can be accessed with the -h
option:
The photsum command uses sculog to produce a brief description of each observation (like usum
for
UKT14). If the reduced data files are present then this summary will include signal and skydip tau
values, e.g.,
reduce_switch
Reduces the raw beam-switched data by subtracting the off-position from the on-position. The telescope will nod after each nine-point jiggle (or nine-second stare, etc.) which is thus referred to as a switch. At this stage the resulting signal can also be multiplied by the internal calibrator. At present this is not advised since the extent to which microphonically induced noise affects the signal at the chop frequency has not been fully investigated.
The SPIKE_LEVEL
option can be used to remove spikes at an early stage. Each position
in the map consists of 128 samples which correspond to one second of integration time.
If SPIKE_LEVEL
is given a value in the range 1–128 the one second of integration will be
removed only if this number of spikes is exceeded. The current default SPIKE_LEVEL
is
5.
change_flat
Used, when appropriate, to switch between flatfield files (see §3.3).
flatfield
Photometric data taken with the arrays should always be divided by the flatfield so that sky removal
can be performed at a later stage. Of course, for two and three bolometer chopping the
flatfield command must be used (see §6). The current flatfield file is called photflat1.dat
and should
ideally reside in the data reduction directory unless it is used as default by flatfield. Note
that photflat1.dat
does not yet contain values for the outer ring of SW bolometers. The
change_flat command can be used to switch between flatfield files. Note that it is necessary to do this
before applying the flatfield.
extinction
Applies an extinction correction to the flatfielded data. If more than one sub-instrument (a sub-instrument is defined as one of the arrays or one of the photometric pixels) was used for the observation then extinction will prompt for one of them. For the arrays, the choice is LONG or SHORT and for the photometric pixels (which will be looking at different parts of the sky) the choice is P2000, P1350 and P1100. Each observation will have to be reduced separately from this stage on. For long, coadded integrations it is likely that the transparency of the sky will change during the observation. The actual values of the extinction coefficients will usually be determined by skydipping before, after, and depending on the sky conditions, possibly in between the group of integrations that are to be coadded (note that it is standard practice to split a long integration into smaller chunks). If the first opacity differs from the second then the extinction is linearly interpolated between the relevant times. Note that extinction requires the sidereal time at which each extinction coefficient was determined but 0 can be given in each case if the extinction remained constant over the integration.
remsky
Allows subtraction of the signal from sky bolometers. Tests during the commissioning period showed that, for faint sources, the signal is often dominated by atmospheric variations or sky noise. Furthermore, such variations were found to be correlated across the arrays and can thus be corrected for. At present the sky removal algorithm simply subtracts from the signal bolometer a mean or median signal level from a user specified list of sky bolometers. The mean method allows bolometers that are a specified number of standard deviations from the mean to be dropped. Sky subtraction is done on a jiggle-by-jiggle basis and so the sky point is measured 9 seconds after the source point for the default 9-point jiggle pattern.
Sky removal should be used with caution. Possible pitfalls include subtracting the signal level from a bolometer at the chop position (for chop throws of less than about 90”), using the inner ring of bolometers for a source that may be extended, or selecting bolometers that are microphonic or dominated by 1/f noise. For point sources and the mean sky subtraction method, we recommend using the inner ring of the long-wave array (h6,h8,h13,h14,g15,g16) and the none-noisy bolometers from the second ring out on the short-wave array (d10,e2,d7,c12,c2,b5,b10,c5,c16). For the median method a longer list can be given.
scuphot
Takes the extinction corrected data and averages the nine points together to produce a final signal for each switch. An ASCII summary file is produced by scuphot which contains the basic parameters of the observation such as source name, coordinates, filter name as well as tabulated values of the signal and its variance for each data point. Also included is the value of the coadded result and its variance. See §5.1 for an example of such a file.
scucat
Concatenates the individual photometric observations to produce a final coadded data set. Note that the user specified output file is appended with the bolometer names that were used in the observation (see the section on multiple bolometer chopping for examples (§6)). It is recommended that this command be executed even if the data set consists of one observation – otherwise the nomenclature becomes unwieldy when plotting the data. Specifically, the plotting routine will require an extension of the form .bolometer_peak to be added to the output of scuphot. For example, if the scuphot output file red25_phot.sdf is an observation with the central pixel of the long-wave array then it will be identified by red25_phot.h7_peak if it is not processed by scucat.
Data that have already been concatenated with scucat can be added to scuphot output. This feature is useful if previously reduced data from one night need to be coadded with newly reduced data from another. Note that you will be prompted for a bolometer name after entering the file name of the previously concatenated dataset.