Chapter 1
Introduction

1.1 This cookbook

This guide is designed to instruct POL-2 users on the best ways to reduce and visualise their data using Starlink packages: Smurf[6], Kappa[9], Polpack[4], and Gaia[12].

This guide covers the following topics.

• Chapter 1 – Computer resources needed before getting started.
• Chapter 2 – A description of POL-2 and its observing modes.
• Chapter 3 – POL-2 Data Reduction - The Theory
• Chapter 4 – POL-2 Data Reduction - Running pol2map
• Chapter 5 – POL-2 Image Display
• Chapter 6 – POL-2 Advanced Data Reduction

Throughout this document, a percent sign (%) is used to represent the Unix shell prompt. What follows each % will be the text that you should type to initiate the described action.

1.2 Before you start: computing resources

Compared with SCUBA-2 observations, POL-2 observations are far less memory-intensive to reduce. POL-2 time-series data are down-sampled to 2 Hz as a part of the reduction process. Assuming a typical 35-minute POL-2 observation, the reduction requires 35 GB of memory (in comparison with SCUBA-2 maps that may require up to 96 GB of memory).

The main consideration for POL-2 reductions is processing power. PCA calculations in makemap can be lengthy, so fast processors with lots of cores are advised.

1.3 Before you start: software

This manual uses software from Starlink packages: Smurf [6], Kappa [9], Polpack[4], and Gaia [12]. Starlink software must be installed on your system, and Starlink aliases and environment variables must be defined before attempting to reduce any SCUBA-2 data (see Section 1.3.2).

1.3.1 Data formats

Data files for POL-2 are structurally the same as for SCUBA-2, and use the Starlink $N$-dimensional Data Format (NDF; [17]), a hierarchical format which allows additional data and metadata to be stored within a single file. Kappa contains many commands for examining and manipulating NDF structures. The introductory sections of the Kappa document (SUN/95) contain much useful information on the contents of an NDF structure and how to manipulate them.

A single NDF structure describes a single data array with associated meta-data. NDFs are usually stored within files of type .sdf. In most cases (but not all), a single .sdf file will contain just one top-level NDF structure, and the NDF can be referred to simply by giving the name of the file (with or without the .sdf suffix). In many cases, a top-level NDF containing JCMT data will contain other ‘extension’ NDFs buried inside them at a lower level. For instance, raw files contain a number of NDF components, which store observation-specific data necessary for subsequent processing. The contents of these (and other NDF) files may be listed with Hdstrace. Each file holding raw JCMT data on disk is also known as a ‘sub-scan’.

The main components of any NDF structure are:

• an array of numerical data (which may have up to seven dimensions—usually three for JCMT data);
• an array of variance values corresponding to the numerical data values;
• an array holding up to eight Boolean flags (known as ‘quality flags’) for each pixel;
• World Co-ordinate System information;
• history;
• data units; and
• other extensions items. These are defined by particular packages, but usually include a list of FITS-like headers together with provenance information that indicates how the NDF was created. Raw JCMT files also include extensions that define the state of the telescope and instrument at each time slice within the observation.

The Starlink Convert package contains commands fits2ndf and ndf2fits that allow interchange between FITS and NDF format.

1.3.2 Initialising Starlink

The commands and environment variables needed to start up the required Starlink packages (Smurf[6], Kappa, etc.) must first be defined. For C shells (csh, tcsh), the commands are:

before using any Starlink commands. For Bourne shells (sh, bash, zsh), the commands are as follows.

1.3.3 KAPPA and SMURF for data processing

The Starlink Sub-Millimetre User Reduction Facility package, or Smurf, contains the Dynamic Iterative Map-Maker, which will process SCUBA-2 time-series data into images (see SUN/258). Kappa, meanwhile, is an application package comprising general-purpose commands mostly for manipulating and visualising NDF data (see SUN/95). Before starting any data reduction it is necessary to initiate both Smurf and Kappa.

After entering the above commands, the help information for the two packages can be accessed by typing smurfhelp or kaphelp respectively in a terminal, or by using the showme facility to access the hypertext documentation. See Section 1.3.5 for more information.

1.3.4 GAIA for viewing your images and vector maps

Images and vector maps can be displayed and analysed using Gaia (see SUN/214) – an interactive GUI-driven tool that incorporates facilities such as vector selection, vector binning, source detection, photometry and the ability to query and overlay on-line or local catalogue data.

Alternatively, the Kappa package includes many visualisation commands that can be run from the shell command-line or incorporated easily into your own scripts—see Appendix “Classified KAPPA commands” in SUN/95.

1.3.5 How to get help