5 Release Notes

5.1 2015A

• This document has been over-hauled, removing or relegating information that is no longer relevant, or is repeated in other documents.
• New commands: JSATILEMOC.
• An experimental scan synchronous noise (SSN) model has been added to MAKEMAP.
• The amount of multi-threading in MAKEMAP is reduced automatically if doing so allows chunking to be avoided.
• The number of chunks used to make a SCUBA-2 map is now recorded in the FITS extension of the map.
• MAKEMAP can now be told to abort if chunking would be used.
• The dimmconfig.lis configuration file has been deprecated. It should no longer be used either directly or as the basis for derived configuration files. Configurations that previously would have inherited from dimmconfig.lis should instead rely on the default parameter values listed in $SMURF_DIR/smurf_makemap.def.

5.2 2014A

• New commands: JSASPLIT, JSAJOIN, JSADICER, JSATILEINFO, JSATILELIST, CHECKCOORDS, FTS2MASKMAP and FITSMERGE.
• New MAKEMAP configuration parameters: AST.SKIP, CHUNKWEIGHT, COM.FREEZE_FLAGS, COM.SIG_LIMIT, FLT.FILT_ORDER, FLT.RING_BOX1, HITSLIMIT, xxx.ZERO_SNR_FFCLEAN, xxx.ZERO_SNR_HIPASS, xxx.ZERO_SNR_LOPASS (“xxx” = AST, FLT, COM).
• New MAKEMAP ADAM parameters: JSATILES, ITERMAPS, INTOPTION.
• New MAKECUBE ADAM parameters: JSATILES, POSERRFATAL.
• New configuration files: dimmconfig_fix_blobs.lis and dimmconfig_fix_convergence.lis.
• Several values have been changed in dimmconfig_bright_extended.lis.

5.3 Hikianalia

• New commands: SKYLOOP and CONFIGMELD.
• If multiple types of mask are requested for any of the AST, FLT and COM models (e.g. ZERO_LOWHITS and ZERO_MASK both specified), they will now be combined into a single mask. Previously, only one of the requested masks would be used.
• Up to three external masks can now be specified using ADAM parameters REF, MASK2 and MASK3. The ZERO_MASK configuration parameter can be set for the AST, FLT or COM model, to indicate which ADAM parameter to use.
• The dimmconfig.lis file now uses maptol to specify the convergence criterion rather than chitol.
• The textttdimmconfig_bright_extended.lis file now inherits from dimmconfig.lis, rather than dimmconfig_bright.lis. This means that the values used for COM.CORR_TOL, COM.GAIN_ABSTOL, COM.GAIN_TOL, DCTHRESH and NOISECLIPHIGH have changed. In addition, dimmconfig_bright_extended.lis now sets the value of FLT.FILT_EDGE_LARGESCALE to 600 arc-seconds for both 450 and 850 um. Previously, the defaults of 600 (at 450 um) and 300 (at 850 um) provided by dimmconfig.lis were accepted. Thus, the filter size has changed from 300 to 600 at 450 um.

5.4 Kapuahi

Many changes - the most significant were.

• New commands: UNMAKEMAP, SC2MAPFFT, SC2FILTERMAP, MCEHEAD2CAT, FIT1D.
• The QLMAKEMAP command has been removed.
• A new, more stable, common-mode estimation has been introduced into MAKEMAP.
• More code has been multi-threaded, making MAKEMAP significantly faster.
• Pointing corrections for the input raw data can now be specified via a text file, using new parameter POINTING.
• CONFIG parameters that specify a number of samples can now also be given in terms of seconds.
• New CONFIG parameter "COM.PERARRAY" allows a separate common mode signal to be estimated and used for each sub-array.
• The COM model can now be masked to omit source samples from the common mode estimation.
• The FLT model can now be masked to omit source samples when estimating the low frequency background of each bolometer time stream.
• SNR masks for the AST, COM and FLT models can now be taken down to much lower levels without introducing isolated noise spikes into the mask (using "xxx.ZERO_SNRLO" parameters).
• A bug has been fixed that caused the memory requirements to grow monotonically when running MAKEMAP repeatedly as a monolith (e.g. from ORAC-DR or ICL).
• Updated Water Vapor Monitor calculation. The WVM samples are now smoothed before being used.
• Setting config parameter "DOWNSAMPSCALE" to -1 will auto-scale to the pixel size. This is now the default.
• New config parameter "MAPTOL" provides a map-based scheme for testing convergence.
• New config parameter "NOI.BOX_SIZE" allows the estimate of the noise within a bolometer time stream to vary with time.
• JCMTSTATE2CAT now calculates telescope speed.
• CALCNOISE now accepts a map-maker config file for pre-cleaning. The default config includes common-mode removal.
• CALCNOISE now calculates the effective noise and NEP.

5.5 Kaulia

Supports compressed SCUBA-2 data files taken since February 2011.

• CALCQU - a new command to create Q and U values from SCUBA-2 fast-spinning polarimeter time-series data files.
• MAKECUBE
  • Now allows the spatial reference pixel in the output cube to be specified by the user.
  • Setting TRIM=YES now modifies any user-supplied LBND/UBND values to remove borders of bad pixels.
• MAKEMAP
  • The loading and flatfielding of raw data on multi-core systems is now faster.
  • The focal plane distortion model has been updated to cover the entire focal plane.
  • A new parameter (TRIM) has been added to allow borders of blank pixels to be removed from the output map.
  • A bug has been fixed so that a request to perform just one iteration (numiter=1) is honoured.

5.6 Namaka

Major updates to Smurf to support SCUBA-2 data processing.

• Polynomial fitting to flatfield data is now the default.
• Fast flatfield ramps are now supported. These are taken at the start of every observation.
• New scripts: smas analyzes "short maps" to investigate high frequency seeing/pointing variations

5.6.1 MAKEMAP (iterative)

• Maps from small chunks of the time series can now be written out. Specify the “shortmap” parameter.
• Typos in config parameters are now trapped and the defaults can be seen in file smurf_makemap.def in $SMURF_DIR. The actual parameters that are used (the merge of the supplied dimmconfig and the defaults) are now stored in the history of the output map.
• Enhanced common-mode removal algorithm which can now break the time series into smaller chunks.
• Enhanced spike removal using a rolling median calculation
• Enhanced step correction.
• Output map now includes QUALITY flags to indicate areas that have been set to 0 by the map-maker.
• The map-maker now reports the details of the flagging every iteration.
• Config files have been re-written to include the standard config with explicit overrides. This makes it easier to see what is being changed.
• Major improvement to the speed of calculating the world coordinates of every bolometer.
• Correctly switch to the CSO tau fits header in the absence of WVM data.

5.6.2 SC2CLEAN

• The parameter DCBOX has been renamed DCFITBOX.
• DCBAD and DCFLAGALL parameters have been removed.

5.6.3 SC2FFT

• Add NGOOD parameter.
• Can now calculate an average power spectrum.

5.6.4 STACKFRAMES

• Fix some bugs associated with maps of different sizes and with missing metadata.
• Propagate QUALITY properly.

5.7 Hawaiki

First official release supporting SCUBA-2 raw data (rather than simulated data).

New applications

• New command CALCNOISE for calculating the noise properties of a SCUBA-2 observation.
• New command COPYFLAT to copy a flatfield from one SCUBA-2 observation to another.
• New command STACKFRAMES for taking a collection of 2-D images (eg noise images or responsivities) and placing them into a time series cube.

Modified applications

• SC2FFT will now concatenate all related files before calculating the FFT.
• MAKECUBE can now avoid generating very thin tiles when tiling is enabled.
• MAKECUBE now uses offset sky co-ordinates in the output cube if, and only if, the input tracking system is (Az,El) or geocentric apparent (RA,Dec).
• JCMTSTATE2CAT now calculates DRA and DDEC columns to indicate arcsecond offsets from base position. Additionally DAZ and DEL are now given as offsets from BASE rather than offsets from the first base position. For SCUBA-2 files JCMTSTATE2CAT will now optionally include MCE header information and will calculate the tau dynamically from the raw WVM data.

5.8 Nanahope

Global changes

• Improve description of SCUBA-2 processing routines for Nanahope Starlink release.
• A summary of input observations is now presented.
• If an output file is derived from multiple observations the output FITS header will now include start and end information (such as date and airmass from the oldest observation and date and airmass from the newest observation).

New applications

• New command CALCRESP for calculating responsivities from flatfield solutions.

Modified applications

• MAKECUBE can now fix up most issues associated with older ACSIS data from 2006 and 2007.
• MAKECUBE output catalogues can now contain additional JCMTSTATE information by using the EXTRACOLS parameter.
• TIMESORT is now significantly faster.
• MAKECUBE, MAKEMAP and QLMAKEMAP are now multi-threaded by default on multi-processor machines.
• MAKECUBE now issues a warning message if WCS information implied by the RECEPPOS and FPLANEX/Y values in an input NDF is inconsistent.
• TIMESORT can now handle single time slice data cubes.
• Many enhancements to SCUBA-2 functionality including support for bad pixel masks and interpolated darks.
• jcmtstate2cat now reports AZ and EL as well as DAZ and DEL.

Bug Fixes

• MAKECUBE formerly mis-interpreted non-zero instrument aperture (INSTAP) values in recent ACSIS data.
• MAKECUBE now ignores input data values that have negative input Tsys values.
• TIMESORT now sets the correct pixel origin in the output NDFs.