Chapter 9
SCUBA-2 Diagnostic Tools

This chapter describes a number of procedures to visualise and assess the quality of raw data files. These steps need not be part of your data reduction process and do not concern the iterative map-maker.

There are reasons, however, that you may wish to examine your raw data in greater depth. The most likely motivation is an unusual result from your reduction such as higher than expected noise, artefacts in your map, or inconsistent noise across multiple observations. This chapter will help you get to the bottom of many of these issues.

9.1 Concatenate & apply a flat-field

Since SCUBA-2 data for a given sub-array are broken into multiple 30-second scans by the data acquisition (DA) system, it is useful to concatenate the data into a single file. The Smurf task sc2concat can be used for this operation. The example below combines all of the files associated with Observation 8 for the s8a array into a single file called s8a20120725_0008_con.

sc2concat will automatically filter out any dark or flat-field observations, so that the concatenated file contains only the science data. Be careful when concatenating a very long observation since the output file may be too large to reasonably handle. Fifteen-minute chunks (30 files) should be sufficient.

sc2concat applies the flat-field by default (although it can be disabled using the noflat option on the command-line).

The flat-field can also be applied manually using the flatfield command.

Here, the output will be a flat-fielded version of each science scan in Observation 8; the file names will be the original input names with _flat appended to them.

As a general rule, you should apply the flat-field to your data before examining it.

9.2 Headers and file structure

There are two Kappa tasks which are extremely useful for examining your data: fitslist and ndftrace, which can be used to view the FITS headers and dimensions of the data.

Full details of these commands can be found in the Kappa manual.

Topcat Example

  % topcat -f tst 20120720_30.tst

Load the tst file generated by jcmtstate2cat into Topcat.

In Topcat select the scatter plot option from the menu bar across the top of the window.

With the scatter plot displayed you can adjust the $X$-axis and $Y$-axis values to DRA and DDEC respectively to display the scan pattern. If you are interested in seeing how any of the variables change over time, select the the $X$ Axis to be either Id or RTS_NUM.

Figure 9.1: Topcat example demonstrating how to display the scan pattern for an observation.

9.3 Displaying scan patterns

The movement of the telescope throughout a scan (as well as other state information) is stored in the MORE.SMURF.JCMTSTATE extension of a data file. The Smurf task jcmtstate2cat converts this information into a simple ASCII tab-separated table.

Multiple files can be supplied to the command using standard shell wild cards. If you have already concatenated your data you can simply input the single concatenated file. It may be useful to view the scan pattern for your observation, particularly for maps taken at high elevations, to ensure the pattern completed successfully.

This catalogue can be loaded into Topcat for plotting, making sure to specify the TST format during loading.

Example of scan patterns displayed with Topcat can be seen in Figure 2.3. Detailed instructions on how to display the scan pattern for your observation are given in Figure 9.1. All of the time-varying header values are available for plotting. Other values include the azimuth and elevation offsets (DAZ & DEL), the WVM and 225 GHz opacity values, and the instrument temperatures (e.g. SC2_FPUTEMP gives the temperature of the focal plane).

Due to extreme accelerations at “turn-around” points of a scan pattern (especially for pongs), the telescope finds it hard to follow the proscribed scan patterns at high elevations. To mitigate this we try to avoid observing any sources above 70${}^{\circ }$ elevation. If the fitslist keywords ELSTART and ELEND indicate that your map was taken at high elevation you may consider checking the success of the scan pattern. If you find your observation has failed to follow the demanded scan pattern don’t worry, the data are likely to still be useful. This is especially true for daisy maps where the high exposure-time central region is usually unaffected.

9.4 Displaying time-series data

Use the Starlink application Gaia to visualise the bolometer time-series data (or indeed any SCUBA-2 data file). This is initiated simply typing gaia into a terminal.

Loading a file in Gaia produces two windows. The main window (see Figure 9.2) shows a map of bolometer values at a given point in time. The time slice displayed may be changed by scrolling through the time axis. This is done in the second window entitled Display image sections of a cube. The Index of plane slider towards the top of this window may be moved to display different time slices in the main window.

Figure 9.2: Initial Gaia windows displayed upon loading a data cube. The main window in the left shows a map of bolometer values at a fixed sample in time. You may have to zoom in multiple times by clicking the Z icon. On the right-hand side, the Display image sections of a cube dialogue enables you to navigate the time axis.

A third window will appear when you click on a bolometer—the Spectral plot (see Figure 9.3). This shows an automatically scaled plot of the raw time stream of data for that given bolometer. It will be overridden when you click on a different bolometer.

Figure 9.3: The Spectral plot window displays the time-varying signal. This window appears automatically when a bolometer is clicked in the main window. The vertical red line indicates the time slice that is currently selected in the Display image sections of a cube dialogue—this can be dragged across the spectrum to scroll through the time-slices.

A second way to scroll through the time axis is to click and drag the vertical red bar on the Spectral plot window. As you do so, the array shown in the main window will automatically update.

To highlight small variations between bolometers you will need to change the auto cut and (depending on your preference) the colour scheme—both are controlled by buttons on the sidebar.

See the Gaia manual for full details.¹

9.5 Regridding data into a map

Any raw time-series data can be quickly regridded into sky frame coordinates using the Smurf makemap task in rebin mode. This involves no further processing of the data. The following command produces a map from the raw concatenated data; unlike the iterative mode of makemap described in the next chapter, no configuration file is required.

The output map here is called crl2688_sky.sdf and is shown in Figure 9.4. The pixel scale is left at the default values of 2 arcsec on a side at 450$\mu$m and 4 arcsec at 850$\mu$m (although this can be changed using the pixsize=$x$ option on the command-line, where $x$ is in arcsec).

Figure 9.4: The regridded map of CRL 2688 with the s8a sub-array displayed with Gaia.

9.6 Notes on cleaning your data

Cleaning raw data is an essential first step towards making a quality final map. The map-maker performs all of these cleaning steps during the pre-processing stage. The commands for manually cleaning your data are given in Appendix A. You can also check out the SMURF SRO Cookbook² which goes into great depth on the data cleaning options.

9.7 Checking the array performance

The on-sky performance of the array can be assessed using the Smurf command calcnoise. Rather than give an absolute measure, calcnoise should be used as an indicator of array performance and stability. calcnoise cleans the data then calculates the white noise on the array (between 2 and 10 Hz by default).

It will prompt for a configuration file to describe the cleaning steps. The default is the file $STARLINK_DIR/share /smurf/dimmconfig_calcnoise.lis that is included in Smurf. Two noise measurements are reported in the terminal: the ‘Effective noise’ and the ‘Effective NEP’.

An output file is created for each sub-array with the NEP map stored in the .MORE.SMURF.NEP extension.

If you have a bright source in the field this will contaminate the signal. In this case you should examine the NOI model from the map-maker instead—see Section 3.3 for a description and Section 9.8 for details on how to examine it.

9.8 Exporting individual models

By default, the final values of the models fitted by the map-maker are not written out. However, this can be changed by setting exportndf in the configuration file to the list of models that you wish to view.

In addition to the models listed in Section 3.3, you can request RES in order to export the RES model (the residual signal remaining after the other models have been removed). If NOI (the estimate of the bolometer noise levels) is exported, it is stored as the VARIANCE component of the RES model; thus, export of RES is implied if NOI is specified.

The exportndf parameter will write out the requested models as NDF files with names based on the first input file that went into the maps for each sub-array. This is first suffixed by con, indicating that several data files may have been concatenated together. The three-letter code for each model is then appended to the filename (such as s8a20120720_00030_0003_con_com.sdf, s8a20120720_00030_0003_con_flt.sdf, s8a20120720_00030_0003_con_res.sdf)³ The variance and quality for the data are stored as the VARIANCE and QUALITY components within the residual file NDF.