13 Advanced Removal of Instrumental Effects

This recipe is an example of reducing CCD data by using CCDPACK from the command line. The full functionality of CCDPACK is available when it is used in this way. However, not all the facilities are used in this recipe and you should see the CCDPACK manual, SUN/139[10], for a full description. Figure 18 shows typical data reduction routes for CCDPACK.

Figure 18: Typical data reduction routes for CCDPACK

Using CCDPACK from the command line is less intuitive than using the xreduce GUI introduced in the previous recipe (see Section 12). However, it is more flexible and, perhaps, gives greater insight into the data reduction process. If you have worked through the previous recipe xreduce will have produced the Unix shell-script xreduce.csh which actually performed the reductions. This file contains a sequence of CCDPACK commands. You might find it interesting to print out a copy and ‘compare and contrast’ it with the commands issued interactively in the current recipe.

If you have not already set up for working through the recipes then you should do so now. The procedure is described in Section 8.1. If you have tried the previous recipe using xreduce (Section 12) then you should ensure that the intermediate and reduced files are deleted by running script delete_xreduce_files.csh. Then proceed as follows.

(1)

Move to the directory to which you have copied the example data. If you have not already started CCDPACK then type:

  % ccdpack

to start it . Various on-line help is available. Typing ccdwww or showme sun139 will cause a hypertext version of SUN/139 to be displayed using a Web browser (Netscape by default). Typing ccdhelp invokes an hierarchical help system in the Unix command window.

(2)

The first step in using CCDPACK is to enter the details of the CCD detector that you are using. The sort of details which have to be specified are the extents of any bias strips (and consequently the extent of the image region on the CCD chip), the ADC factor etc. Type:

  % ccdsetup adc=0.75 saturation=61440 rnoise=6.1 bounds=’[11,40]’ \
      extent=’[51,1074,1,1024]’ accept

Here the ADC factor is being set to 0.75, columns 11 to 40 are being used as a bias strip and the image proper comprises columns 51 to 1074 and rows 1 top 1024. Note how the ranges for the bounds and extent are specified inside single quotation marks in order to prevent the square brackets (‘[’ and ‘]’) used to define the ranges from being interpreted by the Unix shell. The use of such ‘special characters’ is described in SC/4: C-shell Cookbook[4]. The accept option causes ccdsetup to accept default values rather than issue additional prompts.

(3)

The next step is to make a single, ‘master’ bias frame from the various individual bias frames that are available. Type:

  % makebias in=’bias/*’ out=master_bias zero=true accept

The points to note here are:

• the input files are specified as ’bias/*’. Here the asterisk (‘*’) is being used as a ‘wild card’ and all the files in subdirectory bias will be read. Again note the use of single quotes to prevent ‘special characters’ from being interpreted by the Unix shell,
• the master bias frame will be written to file master_bias.sdf in the current directory,
• the option zero=true specifies that the mean values of the pixels in the input images are to be adjusted to zero prior to combining them. This option is the preferred method and normal default. However, it can only be used if the CCD chip has bias strips which were specified using ccdsetup. If your data do not have bias strips you will need to set zero=false. See SUN/139 for further details,
• again the accept option is used to suppress additional prompts.

(4)

Once you have created a master bias frame you need to apply it to your flat field and target image frames. First consider the flat field frames. Type:

  % debias in=’flats/sky*’ out=’*_deb’ bias=master_bias accept

Some points to note here are:

• the asterisk is again being used as a ‘wild card’. The input flat fields will be all the files in subdirectory flats with file names beginning in ‘sky’,
• output de-biassed flat fields will be created in the same subdirectory as the input ones and with the same basic file name, but with ‘_deb’ appended. That is, the following files will be created:
    flats/sky_r_6_deb.sdf
    flats/sky_r_7_deb.sdf

Now repeat the procedure to de-bias the target images:

  % debias in=’targets/ngc2336*’ out=’*_deb’ bias=master_bias accept

If you have a non-zeroed master bias frame or you are using only bias strips rather than bias frames then you need to specify different options for debias. Section 13.1 introduces some of the alternatives.

(5)

The next step is to combine the individual de-biassed flat fields to create a single ‘master’ flat field. CCDPACK provides application makeflat for this purpose. The algorithm used by makeflat ensures that the master flat field it creates contains the minimum possible contribution from spurious sources, such as stars still faintly visible in twilight flats, by comparing each flat field with a smoothed version of itself and rejecting pixels that deviate from the local mean by more than a given number of standard deviations. It also ensures that flat field frames of different exposures are combined using an appropriate weight when the ‘median stacking’ option is used.

To invoke makeflat simply type:

  % makeflat in=’flats/*_deb’ out=master_flat accept

Here the de-biassed flat fields in subdirectory flats are being used as input and the master flat field created will be saved as file master_flat.sdf in the current directory.

(6)

The final stage is to use the master flat field to make a flat field correction to the target images. Type:

  % flatcor in=’targets/ngc2336*_deb’ out=’*_flt’ flat=master_flat accept

The input files are the two de-biassed target images in subdirectory targets. The flat fielded images will be created in the same directory and with the same basic file names but with ‘_flt’ appended. That is, the following two files will be created:

  targets/ngc2336_r_1_deb_flt.sdf
  targets/ngc2336_r_2_deb_flt.sdf

These images should be a true representation of the brightness distribution in the region of sky observed (subject to the constraints of atmospheric seeing and instrumental resolution, of course). The images can, for example, be displayed with GAIA. Type:

  % gaia targets/ngc2336_r_2_deb_flt.sdf &

After setting the Auto Cut level, Magnification and colour table (see the recipe in Section 10) the reduced image should appear similar to the ones in the previous recipe (see Figure 17).

(7)

You can delete the intermediate files at this point if you wish. Return to the top level data directory and type:

  % delete_clfiles.csh

Note that the two reduced images are not deleted because they are used in the next recipe.

13.1 Additional options

This section introduces some additional options available in CCDPACK which were not used in the preceding recipe. It merely gives an outline of what is available. For full details see SUN/139[10].

13.1.1 Specifying bad pixels; ARD files

You can use ARD (ASCII Region Description) files to specify the location of any bad pixels, rows or columns in your images. ARD files are simple text files which contain a series of directives defining the defective parts of the CCD. Their syntax is very simple and is fully documented in SUN/183[1].

The observatory where your data were acquired may provide an ARD file for the instrument that you used. However, if one is not available it is straightforward to create one. There are several ways of doing so. Perhaps the simplest is to plot the image with GAIA. The defective region can then be specified with the Image regions… option and saved as an ARD file. Another method is to plot the image using application display in KAPPA and then use ardgen to identify the bad regions. Section 14.1.1, Doing it the ARD Way, of SUN/95 gives an example of the procedure. Alternatively, ARD files can be created with a simple text editor.

Once you have created an ARD file you specify it as a mask for ccdsetup. In this context a mask is simply a set of defective pixels in the image. For example:

where badpix.ard is the ARD file. Once you have specified the ARD file to ccdsetup all subsequent stages in the data reduction will automatically ignore all the image regions identified as containing bad pixels. This approach can save you a lot of time later.

For infrared images a different approach is common; use application thresh in KAPPA to directly identify pixels with aberrant values and mark them as ‘bad’, without the intermediate stage of an ARD file. KAPPA applications stats or histat can be used to find suitable thresholds.

13.1.2 De-biassing options

In the preceding recipe de-biassing was performed using a zeroed master bias frame. Various other options are available.

Non-zeroed MAKEBIAS frame

To subtract a non-zeroed master_bias frame type:

  % debias in=’targets/ngc2336*’ out=’*_deb’ bias=master_bias offset=false

As before, file master_bias.sdf in the current directory is subtracted from all the files with names beginning in ‘ngc2336’ in subdirectory targets. The option offset=false specifies that the master bias frame has not been zeroed.

No bias frames available

If you do not have bias frames, but do have images with bias strips, it is still possible for CCDPACK to approximately de-bias the images. In this case, debias will estimate values of the bias for each pixel of the image on the basis of information taken from the bias strips. For many purposes this approach will work very well. The options for debias are slightly different because no master bias frame is specified:

  % debias in=’targets/ngc2336*’ out=’*_deb’

More about debias

In most cases debias will change the size of the images because it removes the bias strips (compare Figures 10 and 17). Once the frames have been de-biassed they serve no further purpose and merely increase the size of the files.

debias has a number of additional features which are beyond the scope of this cookbook, including:

• error estimation,
• deferred charge correction,
• saturated pixel correction.

See SUN/139[10] for the full details.

13.1.3 Dark current subtraction; makecal and calcor

Though the correction for dark current is usually insignificant, there can be circumstances where it is not and it is necessary to correct for it (see Section 5). In particular, it is often significant for arrays operating at infrared wavelengths. The correction is usually made using dark frames: images taken with the shutter closed but of the same length as your normal exposures. These frames are used to create a master calibration frame.

makecal can be used to combine a number of calibration frames (assumed to be stored in subdirectory ‘darks’):

Correcting the data for the dark current is performed by the application calcor, which subtracts a scaled calibration frame from a list of target frames:

This command will generate a series of dark current subtracted files that have names ending in ‘_dark’.

The correction is slightly more complicated if your calibration frames are not the same exposure length as your normal exposures: the Flash or dark calibration section of SUN/139[10] gives the details.

13.1.4 Large-scale structure in dome flats

Dome flats which are not evenly illuminated may show large-scale structure which must be removed. This correction can be made by modelling the structure using fitsurface in KAPPA and then subtracting the resulting fitted surface from the flat field.