4 An overview of the reduction

The data reduction for LCOGT depends on the type of data being reduced. Calibration observations (bias, dark, flat) are reduced differently from science observations.

4.1 Bias observations

Bias observations are used to remove the instrumental DC offset applied to the CCD detector to ensure positive values at the ADC.

Masterbiases are created if there are enough ($>2$) raw bias frames in a contiguous group (frames having the same ‘molecule number’, MOLNUM) when the final frame of the group is reached (i.e. when MOLFRNUM == FRMTOTAL). The masterbias is created using the REDUCE_BIAS recipe which in turn calls the _MAKE_BIAS_FROM_GROUP_ primitive. This uses Ccdpack’s makebias command using the method=median combination method without zero offsetting (zero=false) to combine the raw bias frames.

4.2 Dark observations

Dark observations are used to remove the thermal noise from the detector.

Masterdarks are created if there are enough ($>2$) raw dark frames in a contiguous group (frames having the same ‘molecule number’, MOLNUM) when the final frame of the group is reached (i.e. when MOLFRNUM == FRMTOTAL). The masterdark is created using the REDUCE_DARK recipe which in turn calls the _MAKE_DARK_FROM_GROUP_ primitive. This uses Ccdpack’s makecal command using the method=median combination method with the expose= parameter set to a (created temporary) file of the dark exposure time in order to scale the darks before combining the raw dark frames.

4.3 Flatfield observations

Flat observations are used to remove the illumination variation across the field of view and to correct for pixel to pixel sensitivity variations.

Masterflats are created if there are enough ($>3$) raw flat frames in a contiguous group (frames having the same filter and ‘molecule number’, MOLNUM) when the final frame of the group is reached (i.e. when MOLFRNUM == FRMTOTAL). The masterflat is created using the SKY_FLAT recipe which in turn calls the _MAKE_FLAT_FROM_GROUP_ primitive. This uses Ccdpack’s makeflat command to combine the raw flat frames. The combination method and whether cleaning of bad pixels is desired are passed in from the SKY_FLAT recipe in the call to the _MAKE_FLAT_FROM_GROUP_ primitive and is used to set the method= combination method and the clean= parameters, with the normal default being method=broadened and clean=f (as it was found that cleaning introduced clumps of bad pixels, particularly in the sharp edges between Sinistro quadrants). The resulting master flat is normalised to unity.

As masterflats are constructed, the _FLAT_QC_ primitive will check to see if it has found a matching pair of flats (masterflats having the same filter and binning factors, taken from the same night (which is checked via DAYOBS in the header)). Once a suitable pair is found, a difference frame is formed (using Kappa’s sub) and the sigma is computed. If the sigma is within tolerance, the two flats are filed with calibration system for future use (including making copies into the master calibration library location pointed to by $ORAC_DATA_CAL), otherwise the flats are removed from the calibration system flat index.

4.4 Imaging science observations

This section describes the processing that happens for the science frames i.e. the vast majority of the frames. Note that currently ‘standard’ (-s00) frames are treated the same as science (-e00) frames. The standard frames could (and there is some support within other ORAC-DR instrument recipes) be treated differently if desired. This processing is controlled by the OFFLINE_REDUCTION recipe.

4.4.1 Bias subtraction

Bias subtraction and bad pixel masking are actually handled by the generic _PREPARE_SINGLE_FRAME_ primitive which is called from within the recipe-specific _OFFLINE_REDUCTION_HELLO_ primitive. The appropriate bias and bad pixel mask to use are selected based on the normal nearest-in-time suitable calibration frame that matches the frame’s binning.

4.4.2 Dark scaling and subtraction

Dark scaling (the master dark has previously been scaled to an exposure time of unity) to the science frames’ exposure time and subtraction is performed by the _SUBTRACT_DARK_NO_THRESH_ primitive. (The normal _SUBTRACT_DARK_ primitve’s additional thresholding behavior is not appropriate for the LCOGT optical imaging data which is much less suspectible to extra hot pixels). The appropriate dark frame to use is selected based on the normal nearest-in-time suitable calibration frame that matches the frame’s binning.

4.4.3 Flatfield normalization

Flatfield division is performed by the _DIVIDE_BY_FLAT_ primitive and makes use of the Ccdpack flatcor task to perform the actual correction and trimming to the trim limits. The appropriate flatfield frame to use is selected based on the normal nearest-in-time suitable calibration frame that matches the frame’s binning and filter.

4.4.4 Astrometric solution

The current astrometric solver makes use of the AutoAstrom wrapper around the Astrom task and is controlled from within the _ADD_AUTO_ASTROMETRY_ primitive. This handles the detection and extraction of objects on the image using SExtractor and the download of a reference catalog to match against. The reference catalogs are consulted in the order listed in the $ORAC_DATA_CAL/index.skycat_catalogue and the default is UCAC-3 [7] with a fallback of 2MASS [8] (both with two possible server sources at CDS, France and CADC, Canada to add redundancy) and a final fallback of the USNO-B [9] catalog (The Context Cameras (LCOCC) have a different version of $ORAC_DATA_CAL/index.skycat_catalogue which only makes use of the Tycho-2 catalog [10] due to the larger field of view and shallower depth of these cameras).

The Astrom task can fit 4, 6, 7, 8, or 9 coefficients. The 4 coefficient is never used (assumes same scale in both axes & no shear terms), the 6 coefficient is the normal default for most cameras, the 7th term adds fitting for the radial distortion (barrel/pincushion), the 8 coefficient fit adds fitting for the plate center (but not distortion) and finally the 9 coefficient fit adds fit includes fitting for the plate center along with the distortion and is the default for the Context Cameras (LCOCC) due to their large field of view.

After a putative fit, the results are sanity checked on the X and Y rms, the number of stars used in the fit (must be $\ge$6) and the fitted platescale is within 5% of the expected value (which can cause false rejection if the nominal platescale (PIXSCALE in the FITS header) placed by the Instrument Agent hasn’t been properly configured). The cases where the astrometric solution fails or is reset after the sanity check are distinguished by setting WCSERR in the header to ’3’ or ’4’ respectively (with ’0’ indicating success).

4.4.5 Calculation of image statistics

This occurs within the _CALCULATE_SEEING_STATS_ primitive and makes use of SExtractor to perform the extraction and measurement of sources. A relatively high threshold of 5.0 sigma above the background mesh fit with a minimum source size of 12 pixels is used to avoid pushing down too far into the noise and producing a biased result. Objects that have zero size or SExtractor FLAGS!=0 are removed from the calculation and then the mean FWHM, ellipticity and orientation angle are calculated, reported in the log and filed with the calibration system, appearing in index.dqc. These are subsequently added to the header of the products as described in Section 4.4.8.

4.4.6 Production of image catalog

This occurs within the _CREATE_IMAGE_CATALOGUE_ primitive and makes use of SExtractor to perform the source detection and extraction and produce a LCOGT standard source catalog in the format as defined in the LCOGT Pipeline/Science Archive Interface Control Document (LCOGT-SA-ICD Version 0.10.0). This extraction occurs at a lower threshold of 1.5 sigma above the background mesh fit to give a reasonably complete source catalog without overwhelming users or the Science Archive with spurious detections.

The SExtractor config file is in $ORAC_DATA_CAL/extractor_catalogue.sex but some elements are dynamically modified within the primitive as the LCOGT-SA-ICD Version 0.10.0 specification calls for aperture photometry in fixed aperture of 1${}^{\prime \prime }$, 3${}^{\prime \prime }$, 5${}^{\prime \prime }$ and 7${}^{\prime \prime }$ apertures which translates to a variable camera-specific number of pixels, based on the platescale. The output catalog is renamed to $<$root name$>$10_cat.fits or $<$root name$>$90_cat.fits depending on whether QuickLook (indicated by $Frm->uhdr( "QUICK_LOOK" ) == 1) or offline processing is occurring. The output catalog is subsequently modified by the timecorrect.py code as further detailed in Section 4.4.9.

4.4.7 Calculation of zeropoint

This makes use of the catalog of detected sources produced by the previous _CREATE_IMAGE_CATALOGUE_ primitive and performs a cross-match to the combined standard star catalog produced by AJP’s findassm program. The cross match filters on distance, area (object size), ellipticity and quality flags and surviving good matches are used to calculate the mean axial ratio of bright sources,the sky brightness, the limiting magnitude, a transparency estimate and the instrumental zero point. These results are reported in the log and filed with the calibration system, appearing in index.zeropoint. These are subsequently added to the header of the products as described in Section 4.4.8.

4.4.8 Output of derived products

The _DERIVED_PRODUCTS_ primitive is used to create the derived products from the processed frames. The following items are performed:

• the headers are updated with the details of the master calibration frames and correction steps used,
• the saturation and trimmed section are updated with the values used in the pipeline processing,
• A clipped mean, median and sigma are calculated (using _CLIPPED_STATS_MEDIAN_) and added to the header,
• the QC module (the _SET_QC_FLAGS_ primitive) is run to produce the four QC flags in the headers,
• incorrect values of the TAGID and/or PROPID are flagged and corrected and the PROPID is checked against the lists of known proposals,
• access rights and state of the data are set,
• a 512$\times$512 PNG thumbnail of the processed image is created,
• the frame is converted to FITS format,
• the timecorrect.py code (see Section 4.4.9) is called to compute per-star barycentric time correction and airmass and updates the header of the BCD image and source catalog,
• if a final destination is defined by the enviroment variable $FINAL_DATA_OUT, the derived products are moved there.

4.4.9 Airmass and barycentric time correction

In order to provide the most precise absolute timestamp that is free from ambiguities and limited by the properties of the target system, while allowing for improved corrections in the future, we quote the site arrival time (and timesystem) along with the correction to the Barycentric Julian Date in the Barycentric Dynamical Time standard (${BJD}_{TDB}$; see [2] for further details). Although Barycentric Co-ordinate Time, TCB, is formally recommended in the IAU framework (see [3]), it differs from the formerly and widely used time systems such as TDB and TT by $\sim 0.5$ s yr${}^{-1}$ with a current difference of $\sim 17$ s which is likely to lead to confusion for little benefit.

The per-star airmass and barycentric time correction is performed by timecorrect.py which uses a Python-wrapped copy of the SLALIB library to perform the time and reference system transformations. This code calculates a TCORR column which is added to the source catalogs, before being ingested into the archive.

This TCORR is the per-object complete “time correction" to go from UTC start at the observatory to TDB (Barycentric Dynamical Time) at the midpoint of the exposure at the Solar System Barycenter (SSB). This TCORR should be added to the UTC start time to get a proper ${BJD}_{TDB}$. TCORR is a collection of terms:

where $t_{mid}$ is half the exposure time (to get to the midpoint), $t_{clock}$ contains the $\mathrm{UTC}\to \mathrm{TDB}$ correction (which is time-dependent at the few ms level as it involves knowing the Earth’s orientation and the position of the observatory within the Solar System potential), $t_{romer}$ is the Røemer delay (light travel time) from the observatory to the SSB and $t_{einstein}$ is the Einstein delay due to the fact the observatory is moving (with the Earth) and displaced from the geocenter.

The aim of the code was that code should not contribute more than 1% error on the shortest exposure times likely on the LCOGT Network, which was taken to be 30 Hz ($=33.3ms$). The errors due to missing terms or limitations withing the time correction code are believed to be at the few $\times \sim 10\mu s$ level due to:

• Missing from the above equation is the Shapiro delay (due to the light-bending by the Sun’s gravitational potential). This effect is of the order $\sim -20\mu \mathrm{s}$ when observing an object 30 degrees from the Sun’s limb, with the effect decreasing as the Sun$\to$object angle increases.
• The Earth’s barycentric position is calculated using the SLALIB sla_epv routine which in turn uses a simplified version of the VSOP2000 planetary theory [4]. Comparison by TAL with the INPOP10a ephemeris [5] over the range 2005–2031 produced an RMS error of 3.83 km or 12.8 $\mu s$.
• The accuracy on the mean to apparent place transformation (used for airmass) is sub-milliarcsecond, limited by the precession-nutation model (IAU 1976 precession, Shirai & Fukushima 2001 forced nutation and precession corrections [6]), which is also used to transform the Earth’s position to J2000.0 from the true equator and equinox of date.
• The $\mathrm{UTC}\to \mathrm{TT}$ correction needs to know the correct number of leapseconds which relies on the SLALIB sla_dat routine being updated and the code recompiled when a new leap second is announced by the IERS.

The error on the time correction is most likely to be dominated by 1) uncertainty on the NTP synchronization of the instrument control computer ($\sim$1–5 ms), 2) uncertainty in the time-stamping of the exposure start ($\sim$100 $\mu \mathrm{s}$), and 3) uncertainty in the photon-weighted midtime of the exposure.

4.4.10 Photometric calibration

Stefano’s code...