H Release Notes—V4.0

The main changes are the addition of support for UIST, ISAAC, and INGRID instruments.

H.1 New recipes

DARK_AND_BPM

The design of the data-handling system means a traditional ARRAY_TESTS is not possible for UIST. Thus there are two recipes to perform the equivalent steps for darks not in the same observation group. This recipe uses a long-exposure DARK frame to locate bad pixels, then creates and files a bad-pixel mask. It then measures and reports the dark current from a long-exposure DARK frame. If this recipe is run at UKIRT, the dark current is stored in an engineering log file.

MAKE_BPM

Creates and files a bad-pixel mask from a long-exposure dark by thresholding $+$/$-$ 5 clipped standard deviations about the clipped mean.

MEASURE_READNOISE

This is the second UIST-specific recipe replacing ARRAY_TESTS. This recipe measures the readnoise for a group of UIST DARK frames and files that measurement with the calibration system. If this recipe is run in a UKIRT environment it will create or append to a log file located at /ukirt_sw/logs/uist_array_tests.log.

NOD_SKY_FLAT_THERMAL

This is like NOD_SELF_FLAT_NO_MASK, but it expects that the nod throws alternate to sky. The sky frames alone make the flat.

H.2 Modified recipes

• ARRAY_TESTS   The log file is now called $<$instrument$>$_array_tests.log where $<$instrument$>$ is the lowercase instrument name.
• NOD_CHOP_FAINT   There is now an option to create and display a quality map of the combined source images.
• NOD_SELF_FLAT_NO_MASK   This recipe and NOD_SELF_FLAT_NO_MASK_APHOT now do not clean deviant pixels from the individual frames contributing to the flat. Bad pixels in the mosaic are not filled. The difference-pair frames (suffix _dp) are no longer removed upon recipe completion.
• REDUCE_DARK   This now supports variance creation and propagation by default. There is a new steering parameter USEVAR to disable this.

H.3 Global changes

The main changes from a user perspective were as follows.

• Updated and expanded documentation, particularly the description of the primitives and lists the main primitives used at each stage, mention of xoracdr, and many more hyperlinks.
• Introduction of environment variable ORAC_KEEP to retain intermediate frames.
• Environment variables ORAC_DATA_IN and ORAC_DATA_OUT can define paths relative to the current working directory.
• xoracdr now knows the default calibrations for all instruments. A bug which prevented xoracdr operating on non-networked computers is fixed.
• Calibration reference offset added for off-centre nod patterns. This and the baseshift offset used array references. Arrays of values in a -calib can be comma separated.
• A bug affecting ORACTIME in calibrations for Michelle fixed; it was previously set to 0 in all cases.
• Cater for early Michelle data which had missing metadata affecting six headers. Allow for missing, undefined, or malformed RA_BASE and DEC_BASE headers in UFTI data.
• Allow for old UFTI data with CTYPE1 header set erroneously to ’Detector Rows’ to create WCS.
• For IRIS2, modify the header translations for filter name and observation mode; correct the airmass calculation; and allow alternate frames to have alternate files, but still be members of the same group by using different headers for group membership checks.
• There is no longer the assumption of a right-handed world co-ordinate system in co-ordinate transformations benefitting UIST and future instruments with an extra reflection in the optical train.
• There are additional checks for significantly negative frame means after dark subtraction. Depending on the severity this may stop the pipeline, or merely issue a warning. This is to ensure that valid darks are observed.
• The broadened mean method has been reinstated for combining frames into a flat. This avoids the artefact in self-flat fielding where in an $n$-point jitter pattern, 1/$n$ pixels divide by themselves giving rise to a false peak at the normalisation factor in the histogram of values in a flat-fielded frame.
• A possible cause of cause of a negative mode in is given in the warning announnced by _NORMALISE_TO_MODE_.
• _GENERATE_OFFSETS_EXTENDED_ now uses the 98 percentile, the same as the other _GENERATE_OFFSETS_ primitives.
• Centroiding now has a default smoothing option to make object location more robust for fainter sources.
• A double-pass median filtering of bad pixels now occurs just prior to the near-infra-red photometry to reduce bias from unmeasured flux. It particulalry helps the flat-fielded images, as the mosaics almost invariably lose their bad pixels in the region of interest during their formation.
• If variance information is available, the aperture photometry uses it to estimate photometric errors.
• Tidying correctly erases NDF components within HDS container files. Allow for _bc files in relevant _TIDY_ primitives.
• Support added to remove electronic ghosting, correct residual bias, and resample for field distortion, all initially for ISAAC.

The main changes from a programmer perspective were as follows.

• Calibration index files permit user header (uhdr) to be in expressions for bias, dark, flat, and sky.
• Infrastructure uses perl modules Astro::FITS::Header for access to FITS headers, and Starlink::HDSPACK for manipulation of HDS components.
• Add the internal headers ORAC_TELESCOPE, ORAC_X_REFERENCE_PIXEL, and ORAC_Y_REFERENCE_PIXEL.
• There is a $Grp->memberindex( $Frm ) method which returns the position the supplied frame has in the group, starting from 0. This is useful in batch-mode processing where the groups are pre-populated.
• _CLIPPED_STATS_ has new NUMBER argument returning the number of good pixels.
• _DIVIDE_BY_FLAT_CHOP_SKY_ and _DIVIDE_BY_FLAT_EXTENDED_ permit flat-fielding of sky frames via a new argument FLATSKY.
• There are some new primitives including _GET_FILTER_PARAMETERS_ to encapsulate filter attributes, and _FORM_SKY_LEVELS_ to determine the modal sky level for the sky frames after flat-fielding.
• Script nongeneric_imaging.csh is moved from the etc/ to the new admin/ directory.
• Lexical handles rather than global variables are used for accessing files.