3 The SURF commands

 3.1 reduce_switch
 3.2 change_flat
 3.3 flatfield
 3.4 extinction
 3.5 remsky
 3.6 scuphot
 3.7 scucat

The SCUBA data reduction commands are summarized briefly in this section. On-line help for these commands can be accessed with scuhelp or by replying with ?? at any prompt.

There are six steps which need to be followed in order to produce the final coadded, but still uncalibrated, photometric result, namely reduce_switch, change_flat, flatfield, extinction, scuphot and scucat. In addition, data taken with the arrays may be corrected for sky noise variations by subtracting off the signal from surrounding bolometers using the remsky command. – see later in this section for a description of these commands.

The data reduction software takes the demodulated data as input and the format is yyyymmdd_dem_xxxx without the .sdf extension, for example, 19970711_dem_0025. Additional files called yyyymmdd_red_xxxx.sdf are produced by the on-line data reduction software and include preliminary signal information.

The SCUBA log command, sculog2, can be run to give a log of all the observations in a directory. All the journal software (sculog, obssum, photsum, etc.) as well as reduce_switch and skydip recognise the concept of a data directory. This means that the data (demodulated or reduced) does not need to be present in the current directory – the tasks will search for data in the directory specified by the datadir environment variable as well as the current directory. Using the unix C-shell this can be achieved by:

  % setenv DATADIR /wherever/data/19970706/dem

which would instruct obssum, say, to search for data in directory /wherever/data/19970706/dem. Help on the log commands can be accessed with the -h option:

  % sculog -h
  
  Usage:
    sculog [-h] [-all]
  Options:
    -h[elp]        This message
    -summary       Gives a one line summary of each file
    -all           Catalog all sdf files regardless of numeric range
    -begin nn      First scan number to be considered
    -end nn        Final scan number to be considered
    -demod         Only look at raw demodulated data files (ie _dem_)
    -reduced       Only look at files reduced on-line (ie _red_)
    -mode obs      Select observation modes
  
  also
    --begin=nn     First scan number (note the -- prefix)
    --end=nn       Final scan number
    --mode=obs     Select observation modes
  
    Where nn is an integer and ’obs’ is a comma delimited list of obsmodes.
    Use ’perldoc sculog’ for more information.
  Author:
    Tim Jenness (timj@jach.hawaii.edu)

The photsum command uses sculog to produce a brief description of each observation (like usum for UKT14). If the reduced data files are present then this summary will include signal and skydip tau values, e.g.,

  % photsum -all -reduced
   #    HST    Source   Meas/Int  Am   Filter  SubInst Signal   S/N   Tau  Seeing
  ---  -----   -------  -------- ---- -------- ------- ------  -----  ---  ------
  42   20:17   mars        1/5   1.09 450N:850 LONG   4.80e+00 279.5  0.052 1.376
                                               SHORT  3.10e-01 8.32
  45   20:30   mars        1/5   1.07 450N:850 LONG   4.85e+00 375.7  0.044 1.376
                                               SHORT  1.69e+00 185.
  46   20:33   mars        1/5   1.07 450N:850 LONG   4.86e+00 275.6  0.044 1.376
                                               SHORT  1.71e+00 116.
  47   20:36   mars        1/5   1.06 350N:750 LONG   4.15e+00 518.0  0.045 1.376
                                               SHORT  5.35e-01 242.
  48   20:40   mars        1/5   1.06 850S:PHO P1350  4.95e+00 458.5  0.045 0.487
  49   20:43   mars        1/5   1.05 850S:PHO P2000  4.22e-01 282.9  0.045 0.487
  **************
  50   20:46   SKYDIP     10/20       450N:850 SHORT:  0.936          0.062 0.487
                                               LONG    0.211
  --------------
  **************
  51   20:54   SKYDIP     10/20       350N:750 SHORT:  0.943          0.062 0.487
                                               LONG    0.411
  --------------
  64   22:38   B21122+390  1/20  1.08 850S:PHO P2000  2.74e-06 0.203  0.048 0.17
  65   22:49   B21122+390  1/20  1.09 850S:PHO P2000  2.12e-05 1.786  0.044 0.124
  66   22:59   B21122+390  1/20  1.10 850S:PHO P2000  2.50e-05 2.249  0.045 0.124
  67   23:09   B21122+390  1/20  1.12 850S:PHO P2000  1.79e-05 1.829  0.047 0.124
  68   23:19   B21122+390  1/20  1.13 850S:PHO P1350  1.14e-04 5.434  0.05  0.098
  69   23:30   B21122+390  1/20  1.15 850S:PHO P1350  1.18e-04 5.280  0.047 0.098
  72   23:47   mars        1/5   1.20 850S:PHO P2000  4.19e-01 415.3  0.046 0.096
  73   23:51   mars        1/5   1.21 850S:PHO P1350  4.65e+00 720.6  0.046 0.096
  74   23:55   mars        1/5   1.23 450N:850 LONG   4.93e+00 548.5  0.046 0.096
                                               SHORT  1.69e+00 187.
  76   00:55   mars        1/5   1.53 350N:750 LONG   4.01e+00 633.3  0.051 0.144
                                               SHORT  4.51e-01 514.
  78   01:02   B21122+390  1/20  1.46 350N:750 LONG   1.33e-04 4.697  0.046 0.144
                                               SHORT  4.39e-05 2.08
  79   01:13   B21122+390  1/20  1.52 350N:750 LONG   1.11e-04 4.540  0.048 0.108
                                               SHORT  3.10e-05 1.71

3.1 reduce_switch

Reduces the raw beam-switched data by subtracting the off-position from the on-position. The telescope will nod after each nine-point jiggle (or nine-second stare, etc.) which is thus referred to as a switch. At this stage the resulting signal can also be multiplied by the internal calibrator. At present this is not advised since the extent to which microphonically induced noise affects the signal at the chop frequency has not been fully investigated.

The SPIKE_LEVEL option can be used to remove spikes at an early stage. Each position in the map consists of 128 samples which correspond to one second of integration time. If SPIKE_LEVEL is given a value in the range 1–128 the one second of integration will be removed only if this number of spikes is exceeded. The current default SPIKE_LEVEL is 5.

3.2 change_flat

Used, when appropriate, to switch between flatfield files (see §3.3).

3.3 flatfield

Photometric data taken with the arrays should always be divided by the flatfield so that sky removal can be performed at a later stage. Of course, for two and three bolometer chopping the flatfield command must be used (see §6). The current flatfield file is called photflat1.dat and should ideally reside in the data reduction directory unless it is used as default by flatfield. Note that photflat1.dat does not yet contain values for the outer ring of SW bolometers. The change_flat command can be used to switch between flatfield files. Note that it is necessary to do this before applying the flatfield.

3.4 extinction

Applies an extinction correction to the flatfielded data. If more than one sub-instrument (a sub-instrument is defined as one of the arrays or one of the photometric pixels) was used for the observation then extinction will prompt for one of them. For the arrays, the choice is LONG or SHORT and for the photometric pixels (which will be looking at different parts of the sky) the choice is P2000, P1350 and P1100. Each observation will have to be reduced separately from this stage on. For long, coadded integrations it is likely that the transparency of the sky will change during the observation. The actual values of the extinction coefficients will usually be determined by skydipping before, after, and depending on the sky conditions, possibly in between the group of integrations that are to be coadded (note that it is standard practice to split a long integration into smaller chunks). If the first opacity differs from the second then the extinction is linearly interpolated between the relevant times. Note that extinction requires the sidereal time at which each extinction coefficient was determined but 0 can be given in each case if the extinction remained constant over the integration.

3.5 remsky

Allows subtraction of the signal from sky bolometers. Tests during the commissioning period showed that, for faint sources, the signal is often dominated by atmospheric variations or sky noise. Furthermore, such variations were found to be correlated across the arrays and can thus be corrected for. At present the sky removal algorithm simply subtracts from the signal bolometer a mean or median signal level from a user specified list of sky bolometers. The mean method allows bolometers that are a specified number of standard deviations from the mean to be dropped. Sky subtraction is done on a jiggle-by-jiggle basis and so the sky point is measured 9 seconds after the source point for the default 9-point jiggle pattern.

Sky removal should be used with caution. Possible pitfalls include subtracting the signal level from a bolometer at the chop position (for chop throws of less than about 90”), using the inner ring of bolometers for a source that may be extended, or selecting bolometers that are microphonic or dominated by 1/f noise. For point sources and the mean sky subtraction method, we recommend using the inner ring of the long-wave array (h6,h8,h13,h14,g15,g16) and the none-noisy bolometers from the second ring out on the short-wave array (d10,e2,d7,c12,c2,b5,b10,c5,c16). For the median method a longer list can be given.

3.6 scuphot

Takes the extinction corrected data and averages the nine points together to produce a final signal for each switch. An ASCII summary file is produced by scuphot which contains the basic parameters of the observation such as source name, coordinates, filter name as well as tabulated values of the signal and its variance for each data point. Also included is the value of the coadded result and its variance. See §5.1 for an example of such a file.

3.7 scucat

Concatenates the individual photometric observations to produce a final coadded data set. Note that the user specified output file is appended with the bolometer names that were used in the observation (see the section on multiple bolometer chopping for examples (§6)). It is recommended that this command be executed even if the data set consists of one observation – otherwise the nomenclature becomes unwieldy when plotting the data. Specifically, the plotting routine will require an extension of the form .bolometer_peak to be added to the output of scuphot. For example, if the scuphot output file red25_phot.sdf is an observation with the central pixel of the long-wave array then it will be identified by red25_phot.h7_peak if it is not processed by scucat.

Data that have already been concatenated with scucat can be added to scuphot output. This feature is useful if previously reduced data from one night need to be coadded with newly reduced data from another. Note that you will be prompted for a bolometer name after entering the file name of the previously concatenated dataset.

2Use obssum for one line summaries