SC/5.3

Starlink Project
Starlink Cookbook 5.3

A.C. Davenhall, G.J. Privett & M.B. Taylor

16th August 2001

Copyright © 2001 Council for the Central Laboratory of the Research Councils

The 2-D CCD Data Reduction Cookbook


Abstract

This cookbook presents simple recipes and scripts for reducing direct images acquired with optical CCD detectors. Using these recipes and scripts you can correct un-processed images obtained from CCDs for various instrumental effects to retrieve an accurate picture of the field of sky observed. The recipes and scripts use standard software available at all Starlink sites.

The topics covered include: creating and applying bias and flat-field corrections, registering frames and creating a stack or mosaic of registered frames. Related auxiliary tasks, such as converting between different data formats, displaying images and calculating image statistics are also presented.

In addition to the recipes and scripts, sufficient background material is presented to explain the procedures and techniques used. The treatment is deliberately practical rather than theoretical, in keeping with the aim of providing advice on the actual reduction of observations. Additional material outlines some of the differences between using conventional optical CCDs and the similar arrays used to observe at infrared wavelengths.

Who Should Read this Cookbook?

This cookbook is aimed firmly at people who are new to reducing CCD observations. Typical readers might have a set of CCD observations to reduce (perhaps for the first time) or be planning to observe with a CCD camera. No prior knowledge of CCD data reduction is assumed.

Contents

1 Introduction
2 Further Reading
3 Typographic Conventions
4 Overview of CCD Detectors
 4.1 Advantages and disadvantages of CCDs
 4.2 Pixel size and field of view
5 Instrumental Effects in CCD Detectors
 5.1 Bad pixels
 5.2 Read-out signal; bias
 5.3 Non-linearity
 5.4 Thermal noise; dark current
 5.5 Pixel sensitivity; flat fielding
  5.5.1 Fringing
 5.6 Cosmic-ray events
 5.7 Photon noise
6 Reducing CCD Data
 6.1 Software available
 6.2 Data formats
  6.2.1 FITS
 6.3 Illustration of data reduction
7 Infrared Arrays
 7.1 Reduction procedure
8 Introduction
 8.1 Getting started
9 Importing Data
10 Displaying Images
11 Calculating Statistics
12 Simple Removal of Instrumental Effects
13 Advanced Removal of Instrumental Effects
 13.1 Additional options
  13.1.1 Specifying bad pixels; ARD files
  13.1.2 De-biassing options
  13.1.3 Dark current subtraction; makecal and calcor
  13.1.4 Large-scale structure in dome flats
14 Combining Target Images
15 Reading FITS Files from Tape
16 Handling Large Images
 16.1 How large is large?
 16.2 Limitation on NDF or HDS file sizes
 16.3 General tips
 16.4 Bottleneck applications in CCDPACK
 16.5 Specific tips
17 Introduction
18 Convert to a New Data Format
19 Clip an Image
20 Process Compressed Files
21 Examine Specified FITS Keywords
22 Automatically Scale Displays
Acknowledgements
References

Revision history

(1)
17th March 1997: Version 1. Original version (GJP).
(2)
8th June 1999: Version 2. Added material for infrared arrays and the section describing the processing of large images (contributed by MBT). Also revised and re-arranged much of the existing material (ACD).
(3)
16th August 2001: Version 3. A few minor changes. Added an additional label to allow external hyper-links to the section on the FITS format. Also revised the references and URLs (ACD).

Acknowledgements

We are grateful to Mike Lawden, Peter Draper, Rodney Warren-Smith and, particularly, Malcolm Currie who all provided many useful comments on the draft version of this cookbook. Additional thanks go to Peter Draper for the examples, text and diagrams borrowed from SUN/139, the CCDPACK manual.

Thanks are due to Mr I. Morgan, Mr Gareth Leyshon and Dr Steve Eales for providing a number of scripts and to Dr Gerry Luppino of the Hawaii IFA for allowing us to use Figure 1. The data from Walter Jaffe’s CD-ROM Astronomical Images are used with the permission of the author and publisher. Karen Moran kindly unearthed the contact details for Twin Press.

Any mistakes, of course, are our own.

References

[1]   D.S. Berry, 1996, SUN/183.3: ARD — A Textual Language for Describing Regions within a Data Array (Starlink).

[2]   C. Buil, 1991, CCD Astronomy – Construction and Use of an Astronomical CCD Camera (Willmann-Bell: Richmond, Virginia). Translated by E. and B. Davoust. Originally published in French as Astronomie CCD, Construction et Utilisation des Cameras CCD en Astronomie Amateur in 1989.

[3]   M.J. Clayton, 1998, SC/7.2: Simple Spectroscopy Reductions (Starlink).

[4]   M.J. Currie, 1998, SC/4.2 C-shell Cookbook (Starlink).

[5]   M.J. Currie, 1998, SUN/102.4: HDSTRACE — Listing HDS Data Files (Starlink).

[6]   M.J. Currie and D.S. Berry, 2000, SUN/95.16: KAPPA — Kernel Application Package (Starlink).

[7]   M.J. Currie, G.J.Privett, A.J.Chipperfield, D.S. Berry and A.C. Davenhall, 2000, SUN/55.14: CONVERT — A Format-conversion Package (Starlink).

[8]   A.C. Davenhall, 1998, Starlink Bulletin, No. 20, pp17-19. In the first instance see your site manager for back issues of the Starlink Bulletin. Alternatively, a version of this article is available by anonymous ftp from Edinburgh. The details are: ftp site ftp.roe.ac.uk, directory /pub/acd/misc, file roadmap.ps. The file is in PostScript format.

[9]   P.W. Draper, 1995, Starlink Bulletin, No. 16, pp6-7. In the first instance see your site manager for back issues of the Starlink Bulletin.

[10]   P.W. Draper, M.B. Taylor and A. Allan, 2000, SUN/139.13: CCDPACK — CCD Data Reduction Package (Starlink).

[11]   P.W. Draper and N. Gray, 2000, SUN/214.8: GAIA — Graphical Astronomy and Image Analysis Tool (Starlink).

[12]   N. Gray, M.B. Taylor and G.J. Privett, 2000, SUN/180.5: ESP — Extended Surface Photometry (Starlink).

[13]   S.B. Howell (ed), 1992, Astronomical CCD Observing and Reduction Techniques, Astronomical Society of the Pacific Conference Series, 23.

[14]   W. Jaffe, 1998, Astronomical Images (Twin Press: Vledder) CD-ROM. To contact Twin Press send an electronic mail message to G. Kiers (gkiers@twinpress.nl).

[15]   D.C. Kay and J.R. Levine, 1995, Graphics File Formats, second edition
(Windcrest/McGraw-Hill: New York). See in particular Chapter 18, pp235-244.

[16]   C.R. Kitchin, 1998, Astrophysical Techniques, third edition (Institute of Physics: Bristol and Philadelphia). First edition published in 1984.

[17]   P. Massey and G.H. Jacoby, 1992, in Howell op. cit. ([13]), pp240-257.

[18]   P. Massey, 1997, A User’s Guide to CCD Reductions with IRAF (National Optical Astronomy Observatories: Tucson). See SG/12 op. cit. ([22]) for details of obtaining IRAF manuals. Retrieve file ccduser3.ps.Z

[19]   I.S. McLean, 1989, Electronic and Computer-Aided Astronomy – From Eyes to Electronic Sensors (Ellis Horwood: Chichester).

[20]   I.S. McLean, 1997, Electronic Imaging in Astronomy – Detectors and Instrumentation (Wiley: Chichester and New York). Published in association with Praxis in the Wiley-Praxis series in Astronomy and Astrophysics.

[21]   R. Morris and G.J.  Privett, 1996, SUN/166.4: SAOIMAGE — Astronomical Image Display (Starlink).

[22]   R. Morris, G.J. Privett and A.C. Davenhall, 1999, SG/12.2: IRAF — Image Reduction Analysis Facility (Starlink).

[23]   M.V. Newberry, 1995, CCD Astronomy, 2, No. 1, pp18-21.

[24]   M.V. Newberry, 1995, CCD Astronomy, 2, No. 3, pp12-14.

[25]   M.V. Newberry, 1996, CCD Astronomy, 3, No. 1, pp18-21.

[26]   J. Palmer and A.C. Davenhall, 2001, SC/6.4: The CCD Photometric Calibration Cookbook (Starlink).

[27]   K.T. Shortridge, H. Meyerdierks, M.J. Currie, M.J. Clayton, J. Lockley, A.C. Charles, A.C. Davenhall, M.B. Taylor, T. Ash, T. Wilkins, D. Axon, J. Palmer and A. Holloway, 2001, SUN/86.18: FIGARO — A General Data Reduction System (Starlink).

[28]   F. Valdes, 1990, User’s Guide to the CCDRED Package (National Optical Astronomy Observatories: Tucson). See SG/12 op. cit. ([22]) for details of obtaining IRAF manuals. Retrieve file ccdguide.ps.Z

[29]   F. Valdes, 1990, The IRAF CCD Reduction Package – CCDRED (National Optical Astronomy Observatories: Tucson). See SG/12 op. cit. ([22]) for details of obtaining IRAF manuals. Retrieve file ccdred.ps.Z

[30]   G. Walker, 1987, Astronomical Observations – An Optical Perspective (Cambridge University Press: Cambridge).

[31]   R.F. Warren-Smith, 2000, SUN/33.7: NDF — Routines for Accessing the Extensible N-Dimensional Data Format (Starlink).

[32]   R.F. Warren-Smith and M.D. Lawden, 1999, SUN/92.11: HDS — Hierarchical Data System (Starlink).

[33]   D.C. Wells, E.W. Greisen and R.H. Harten, 1981, Astron. Astrophys. Suppl, 44, pp363-370.

[34]   D.C. Wells, 2000, in Information Handling in Astronomy, ed. A. Heck (Kluwer: Dordrecht), Astrophysics and Space Science Library, 250, pp65-72.

[35]   L.A. Wells, 1994, Rectifying and Registering Images Using IRAF (National Optical Astronomy Observatories: Tucson). See SG/12 op. cit. ([22]) for details of obtaining IRAF manuals. Retrieve file reg.ps.Z

[36]   L.A. Wells and D.J. Bell, 1994, Cleaning Images of Bad Pixels and Cosmic Rays Using IRAF (National Optical Astronomy Observatories: Tucson). See SG/12 op. cit. ([22]) for details of obtaining IRAF manuals. Retrieve file clean.ps.Z