SUN/258.4

Starlink Project
Starlink User Note 258.4

Edward Chapin, Andrew G. Gibb, Tim Jenness, David S. Berry, Douglas Scott & Remo Tilanus

25th February 2015

Copyright © 2012 University of British Columbia & the Science & Technology Facilities Council

SMURF – the Sub-Millimetre User Reduction Facility

Version 2.0.0

User’s Guide

pdfpict

Abstract

The Sub-Millimetre User Reduction Facility (Smurf) is a software package for initial reduction of data produced by the ACSIS correlator and the SCUBA-2 bolometer array on the James Clerk Maxwell Telescope (JCMT). This document describes the Smurf tasks used to process raw ACSIS data into data cubes, and raw SCUBA-2 data into images. These low-level tasks are used within automated Orac-dr pipelines (documented elsewhere) that produce calibrated data products. This document is intended for users who wish to run the low-level tasks themselves, to explore possibilities not provided by the automated pipelines.

Contents

1 Introduction
 1.1 Document conventions
 1.2 Using Smurf
  1.2.1 Starting Smurf
  1.2.2 Getting help
  1.2.3 Smurf parameters
  1.2.4 Message filter
  1.2.5 Working with data files
 1.3 Data file structure
 1.4 Supported coordinate systems
  1.4.1 Moving sources
 1.5 File sizes and disk space
2 ACSIS Data Reduction
 2.1 The makecube command
 2.2 The timesort command
 2.3 The unmakecube command
 2.4 The fit1d command
  2.4.1 Fitting functions
  2.4.2 Component Parameter files
3 SCUBA-2 Data Reduction
 3.1 Choosing input data files
 3.2 Array calibration and characterisation
  3.2.1 FLATFIELD
  3.2.2 NOISE
  3.2.3 Comparing multiple noise or flatfield observations
 3.3 Dynamic Iterative Map-Maker
 3.4 Speeding up map-making by re-using previously cleaned data
4 Acknowledgments
5 Release Notes
 5.1 2015A
 5.2 2014A
 5.3 Hikianalia
 5.4 Kapuahi
 5.5 Kaulia
 5.6 Namaka
  5.6.1 MAKEMAP (iterative)
  5.6.2 SC2CLEAN
  5.6.3 SC2FFT
  5.6.4 STACKFRAMES
 5.7 Hawaiki
 5.8 Nanahope
A Classified Smurf Commands
B Full Specifications of all Smurf Commands
BADBOLOS – Generate a map of random dead bolometers and add it as an NDF extension to the input file
CALCDARK – Calculate the 2d dark frame from dark observation
CALCFLAT – Calculate a flatfield solution from a flatfield observation
CALCNOISE – Calculate noise image
CALCQU – Calculate Q and U images from a set of time-series data files
CALCRESP – Calculate bolometer responsivity from stored flatfield solution
CHECKCOORDS – Check for discrepancies between AZEL and TRACKING coordinates
COPYFLAT – Copy the flatfield information from a reference file
DREAMSOLVE – Solve DREAM observations and generate 2-D images
DREAMWEIGHTS – Generate DREAM weights matrix
DSUTILS – A collection of utilities for estimating focal plane distortion
EXTINCTION – Extinction correct SCUBA-2 data
FINDSLICES – Find time slices that are centred close to a given sky position
FIT1D – Fit 1-D profiles to a data cube
FITSMERGE – Merge FITS headers
FIXSTEPS – Fix DC steps in a supplied SCUBA-2 time series NDF
FLATFIELD – Flatfield SCUBA-2 data
FTS2DEGLITCH – Removes the glitches from the source
FTS2FLATFIELD – Corrects for the pixel-to-pixel variation in spectral responsivity, due to detector characteristics
FTS2FREQCORR – Off-Axis frequency correction
FTS2INIT – Prepares the input to be processed by the FTS2 Data Reduction tasks
FTS2MASKMAP – Mask time series data for a SCUBA-2 map with FTS-2 in the beam
FTS2OPCORR – Compansates for the effect of the FTS-2 optics on image distortion and field rotation
FTS2PHASECORR – Given a 3D data cube of single-sided interferograms, applies phase correction and outputs the corresponding 3D interferogram cube
FTS2PHASECORRDS – Given a 3D data cube of double-sided interferograms, applies phase correction and outputs the corresponding 3D spectrum cube
FTS2PORTIMBAL – Corrects for atmospheric transmission across the spectral dimension
FTS2REMOVEBSE – Removes the Beam splitter Self Emission (BSE) from the source
FTS2SPECTRUM – Computes the spectrum of the interferograms
FTS2SPLIT – Convert multi scan NDFs for use in FTS-2 data reduction pipeline
FTS2TRANSCORR – Corrects for atmospheric transmission across the spectral dimension
GAU2FIT – Fit a two-component Gausian to a map of an extended source
GSD2ACSIS – Convert a GSD format DAS data file to an ACSIS format NDF
GSDSHOW – Display the contents of headers and arrays for GSD files
IMPAZTEC – Import AzTEC NetCDF files and produce SCUBA-2 ICD-compliant files
JSADICER – Dice an image or cube into JSA tiles
JSAPASTER – Paste several JSA tiles into a single mosaic
JSATILEINFO – Return information about a specified sky tile
JSATILELIST – List the sky tiles that overlap a given set of data files or an AST Region
MAKECUBE – Regrid ACSIS spectra into a data cube
MAKEMAP – Make a map from SCUBA-2 data
NANTEN2ACSIS – Convert NANTEN2 FITS format data files to an ACSIS format NDF
POL2CHECK – Check if specified NDFs probably hold POL-2 data
POL2IPCOR – Create an IP model from a set of POL2 observations of a bright extended source
QUCOVAR – Find co-variance of Q and U from a POL2 observation
RAWPRESS – Compress raw data
RAWRECREATEWCS – Fix broken raw ACSIS files by enabling the spectral WCS to be reconstructed
RAWREWRTSC2WCS – Fix broken WCS in raw SCUBA-2 file by re-writing it
RAWUNPRESS – Uncompress raw data
RAWFIXMETA – Fix metadata associated with a raw data file
REMSKY – Remove sky background from SCUBA-2 data
SC2CLEAN – Clean SCUBA-2 time-series data
SC2CONCAT – Concatenate files into a larger file
SC2EXPANDMODEL – Expand a DIMM model component into a full time-series data cube
SC2FFT – Fourier Transform SCUBA-2 time-series data
SC2FILTERMAP – Filter a 2-d map
SC2MAPFFT – Fourier Transform 2D maps
SC2PCA – Use principal component analysis to identify correlated SCUBA-2 signals
SC2SIM – SCUBA-2 Simulator
SC2THREADTEST – Task for testing speeds of different threading schemes
SKYNOISE – Generate a simulated sky background with spatial noise
SMURFCOPY – Copy a 2d image out of a time series file
SMURFHELP – Gives help about SMURF
STACKFRAMES – Stack 2d processed frames into time series cube
STARECALC – Calculate image for SCUBA-2 STARE observations
SUPERCAM2ACSIS – Convert a Supercam SDFITS format data file to an ACSIS format NDF
TIMESORT – Re-order the time slices in a raw ACSIS data cube into increasing time
UNMAKECUBE – Produce simulated time series data from a regrided ACSIS data cube
UNMAKEMAP – Produce simulated time series data from a SCUBA-2 map

C Python Scripts
CONFIGMELD – Compare two configs using a visual file comparison tool
FTS2GAIA – Display in-band FTS-2 spectrum in gaia
JSAJOIN – Create a single tangent-plane NDF from a set of JSA tiles
JSASPLIT – Split a single NDF up into a set of JSA tiles
JSATILEMOC – Create an image MOC based on a JSA tile
SKYLOOP – Create a map using the " inside-out" algorithm
POL2MAP – Create Q, U and I maps from a group of POL-2 " spin&scan" data files
POL2SIM – Create simulated POL2 data from known I, Q and U maps
POL2IP – Create an Instrumental Polarisation (IP) model from a set of POL2 observations
POL2STACK – Combine multiple Q, U and I images and create a vector catalogue from them
POL2NOISE – Analyse the noise in a POL2 vector catalogue
POL2SCAN – Create Q and U maps from a set of POL-2 " spin&scan" data files

D Other Scripts
ACSIS_INDEX – Provide a listing of ACSIS data files in a directory
DUMPOCSCFG– Retrieve OCS configuration XML from data file.
GETTSYS – Get system temperature information from ACSIS data
JCMTSTATE2CAT – convert JCMT state structure into TST format
MCEHEAD2CAT – Convert SCUBA-2 MCE header information to TST format
SCUBA2_INDEX – Provide a listing of SCUBA-2 data files in a directory
SMAS – Short Map Analysis Script

E Configuration Parameters
F DREAM/STARE data reduction workflow
  F.0.1 DA images
  F.0.2 Applying the flatfield correction
  F.0.3 Removing the atmosphere
  F.0.4 Correcting for atmospheric extinction
  F.0.5 Calculating images from time-series data
  F.0.6 Combining the images
  F.0.7 Using alternative DREAM solutions
 F.1 Moving sources
G Rebinning map-maker
H SCUBA-2 Simulator
 H.1 Simulator workflow
  H.1.1 Astronomical image
  H.1.2 Model atmosphere
  H.1.3 Flatfield simulation
  H.1.4 Running the simulator
  H.1.5 A note on DREAM simulations
 H.2 Processing simulated data

References

[1]   Archibald E. N., et al., 2002, MNRAS, 336, 1

[2]   Buckle J. V., et al., 2009, MNRAS, 399, 1026

[3]   Chapin E. L., et al., 2013, MNRAS, 430, 2545 doi:10.1093/mnras/stt052, arXiv:1301.3652

[4]   Holland W. S., et al., 2006, Proc. SPIE, 6275, 45

[5]   Jenness T., et al., 2002, MNRAS, 336, 14

[6]   Jenness T., Gibb, A., 2007, Data Acquisition/Data Reduction Pipeline Interface Control Document, SCUBA-2 Data Reduction document SC2/SOF/IC210/01

[7]   Le Poole R. S., van Someren Greve H. W., 1998, Proc. SPIE, 3357, 638

[8]   Scott D., 2003, Map-making in different noise regimes, SCUBA-2 Data Reduction document SC2/ANA/S210/001

[9]   Scott D., 2005, Scan Mode Data Reduction Strategies for SCUBA-2, SCUBA-2 Data Reduction document SC2/ANA/S210/006

[10]   Scott D., Van Engelen A., 2005, Scan Mode Strategies for SCUBA-2, SCUBA-2 Data Reduction document SC2/ANA/S210/005

[11]   Van Engelen A., 2005, Analysis of Atmospheric Emission using SHARC-II Data and Implications for the SCUBA-2 Simulator, SCUBA-2 Data Reduction document SC2/ANA/S210/002

[12]   Van Engelen A., Scott D., 2005, A Correlation Study using SCUBA Data, with implications for SCUBA-2 data reduction, SCUBA-2 Data Reduction document SC2/ANA/S210/002