10 THE QUALITY COMPONENT IN MORE DETAIL

 10.1 Purpose of the Quality Component
 10.2 Accessing the Quality Array Directly
 10.3 The Bad-bits Mask
 10.4 Why Ignoring the Quality Component Works
 10.5 Controlling Automatic Quality Masking

10.1 Purpose of the Quality Component

The quality component of an NDF is provided to allow individual pixels to be flagged as having certain properties which applications may wish to take into account when processing them. In general, these properties are expected to be of a binary or logical character, such as membership of a set, rather than being quantitative, although combining several binary values to construct a simple numerical scale is not excluded.

To give a simple practical example, it might be useful to flag all the pixels in an image which are contaminated by defects in the detector system from which it originates. It might also be useful to classify each pixel as lying either in the “sky” background, or in the observed “object”. Armed with this sort of information, it then becomes possible to perform operations like:

An NDF’s quality component allows up to eight such binary conditions to be flagged, so numerous other applications and possibilities obviously exist.

If this appears too complicated for the sort of work you have in mind, then the good news is that the quality component can be almost entirely ignored by most applications and the default action of the NDF_ system will take care of it automatically. However, for applications which wish to exploit the possibilities that the quality component offers, a set of NDF_ routines is provided to access and control it explicitly. Their use is described here.

10.2 Accessing the Quality Array Directly

The main part of the quality component is an array containing an 8-bit (unsigned byte) value for each NDF pixel. Each of the bits in these pixels can be thought of as providing a 1-bit logical mask extending over all pixels and can be used to signify the presence or absence of a particular property at that location.

Thus, if bit 1 (value 1) indicated a contaminated pixel, and bit 2 (value 2) indicated membership of the set of “sky” pixels, then a value of 3 (i.e. 1+2) would indicate a contaminated sky pixel, while a value of 0 would indicate an uncontaminated “object” pixel. With 8 bits available, quality values can range from 0 to 255.

An application may access the quality array directly by means of the routine NDF_MAP. For instance, the following call maps it for read access in exactly the same way that the NDF’s data component might be accessed:

        CALL NDF_MAP( INDF, ’Quality’, ’_UBYTE’, ’READ’, PNTR, EL STATUS )

Note that when accessing the quality array in this way a numeric type of ‘_UBYTE’ must always be used. The outcome of using any other type is unspecified, although a meaning might be assigned to such an operation in future.

‘WRITE’ and ‘UPDATE’ access is also available. When an NDF is first created its quality component is in an undefined state. As with other components, it becomes defined once values have been written to it. Either of the initialisation options ‘/BAD’ or ‘/ZERO’ may also be appended to the mapping mode specification (see §8.6), where ‘/BAD’ causes initialisation to the value 255 (all bits set) and ‘/ZERO’ causes initialisation to zero (all bits clear).

10.3 The Bad-bits Mask

The significance attached to each bit of the quality values is arbitrary, and it is expected that specialised software packages which need to exploit the full capabilities of the quality component will make a particular choice of bit assignments and interpret these to influence the processing they perform, possibly in quite subtle ways. Such applications can never be truly general-purpose, however, because the number of possible bit assignments is unlimited, so agreement between writers of different software packages about how to interpret each bit cannot be achieved.

Nevertheless, some way of taking account of quality values in general-purpose software is very desirable. To allow this, an additional part of the quality component, termed the bad-bits mask is provided. This is a single 8-bit (unsigned byte) value, whose purpose is to allow the 8-bit quality value associated with each pixel to be converted into a single logical value, which can then be interpreted in the same way by all general-purpose software.

The value of the bad-bits mask for an NDF is obtained by calling the routine NDF_BB:

        BYTE BADBIT
  
        ...
  
        CALL NDF_BB( INDF, BADBIT, STATUS )

The returned result BADBIT is an unsigned byte number. If the NDF’s quality component is undefined and/or a bad-bits value has not previously been set, then a value of zero is returned.

A new value for the bad-bits mask may be set with the routine NDF_SBB:

        CALL NDF_SBB( BADBIT, INDF, STATUS )

By definition, the quality value of an individual pixel QUAL is converted into a logical value by forming the bit-wise “AND” with the bad-bits mask BADBIT and testing if the result is zero. In effect, this means that each bit which is set in BADBIT acts as a switch to activate detection of the corresponding bit in QUAL. If detection of a particular bit is activated and that bit is set in the QUAL value, then the corresponding NDF pixel is to be regarded as bad, and should be omitted from processing in much the same way as any pixel which has been explicitly assigned the bad-pixel value (see §9.4).

In Fortran implementations which support bit-wise operations, the logical value might be computed as follows:

        OK = IIAND( IZEXT( QUAL ), IZEXT( BADBIT ) ) .EQ. 0

where a .FALSE. result indicates a bad pixel. However, this operation clearly requires non-standard Fortran functions and is potentially non-portable, so a function NDF_QMASK is provided to perform this task and to hide the implementation details. NDF_QMASK is a statement function and is defined by putting the following include statement into an application, normally immediately after any local variable declarations:8

        INCLUDE ’NDF_FUNC’

Conversion of the quality value into a logical value is then performed by the function as follows:

        OK = NDF_QMASK( QUAL, BADBIT )

The following example shows how a simple loop to find the average value of a pixel in an image array DATA might be written so as to take account of associated quality values, and exclude affected pixels:

        BYTE QUAL( NX, NY )
        INTEGER NGOOD, NX, NY, I, J
        REAL DATA( NX, NY ), SUM, AVERAG
        INCLUDE ’NDF_FUNC’
  
        ...
  
        SUM = 0.0
        NGOOD = 0
        DO 2 J = 1, NY
           DO 1 I = 1, NX
              IF ( NDF_QMASK( QUAL( I, J ), BADBIT ) ) THEN
                 SUM = SUM + DATA( I, J )
                 NGOOD = NGOOD + 1
              END IF
   1       CONTINUE
   2    CONTINUE
  
        IF ( NGOOD .GT. 0 ) AVERAG = SUM / REAL( NGOOD )

As an alternative to the above explicit conversion of quality values into logical values, it is also possible to gain access to the quality array already converted into this form by calling the routine NDF_MAPQL, as follows:

        CALL NDF_MAPQL( INDF, PNTR, EL, BAD, STATUS )

This routine returns a pointer to a mapped array of logical values which are derived from the quality array and the current value of the bad-bits mask following the prescription above. Only read access is obtained by this routine and any modifications made to the mapped values will not result in a permanent change to the quality component. If the quality component is undefined, then an array of .TRUE. values is returned.

NDF_MAPQL also has an additional argument BAD, which indicates whether there are any .FALSE. values in the mapped array it returns. If BAD is returned .FALSE., then all the mapped values will have the value .TRUE., so quality information is either absent or, in combination with the bad-bits mask, has no effect. In such cases it may be possible to omit handling of quality values altogether and affect an improvement in algorithmic efficiency as a result.

Finally, it should be noted that while the use of NDF_MAPQL may appear more convenient than explicitly handling the quality values and bad-bits mask, it nevertheless involves an additional pass through the quality array and will therefore be a less efficient option.

10.4 Why Ignoring the Quality Component Works

Since the purpose of the quality component in general-purpose software is to indicate (in conjunction with the bad-bits mask) whether each pixel is “good” or bad, it is natural to ask whether quality component information can be handled in the same way as bad pixels. The answer is “yes”. In fact the NDF_ routines assume by default that quality information will be handled in this way, and this makes it legitimate simply to ignore the presence of quality information in most cases.

What actually happens to make this possible is that by default all access to an NDF’s data and variance components implicitly takes account of the possible presence of an associated quality component. For instance, if NDF_BAD is called to determine if bad pixels may be present in an NDF’s data array (see §9.4), then an implicit check will also be made (if necessary) to see whether a quality component is also present. A further check on the bad-bits value may also be made, and all this information will be combined to determine whether bad pixels may be present.

When values are read from the data array (by mapping it for read or update access), a similar process takes place. In this case, if quality information is present, it is combined with the bad-bits value and used to insert bad pixels into the mapped data wherever the quality masking function evaluates to .FALSE.. The application accessing the data array need not know that this is happening, and can process the resulting mapped values as normal (but taking due account of the bad-pixel values). Note that no similar process takes place when values are written back to an NDF component (i.e. when any component accessed in update or write mode is unmapped); in this case the array values are transferred without interference from the quality component.

The same automatic quality masking also takes place by default when values are read from the variance component. This indicates why it may be more efficient to map both the data and variance components in a single call to NDF_MAP, e.g:

        INTEGER PNTR( 2 ), EL
  
        ...
  
        CALL NDF_MAP( INDF, ’Data,Variance’, ’_REAL’, ’READ’, PNTR, EL, STATUS )

which returns a pair of pointers in the PNTR array. By doing this, it becomes possible for NDF_MAP to access the quality array (if it exists) only once, and insert bad pixels into both mapped arrays simultaneously.

10.5 Controlling Automatic Quality Masking

The automatic quality masking action described above is controlled by a logical quality masking flag associated with each NDF identifier, and this flag can be manipulated if necessary to modify the default behaviour.

When an NDF identifier is first issued, its quality masking flag is set to .TRUE.. In this state, all access to the NDF’s data and variance components via this identifier will implicitly take account of any quality information present, as described above. So long as no explicit access to the NDF’s quality array is made, the quality masking flag remains set to .TRUE.. Thus, an application which chooses to completely ignore quality values will obtain the desired automatic quality masking action.

However, if explicit access to the quality array is obtained (e.g. by using the routines NDF_MAP or NDF_MAPQL), the flag will simultaneously be reset to .FALSE.. It is then assumed that the application intends to process quality values explicitly, so access to other NDF components will no longer take account of quality values. The quality masking flag returns to .TRUE. once direct access to the quality array is relinquished (by unmapping it using NDF_UNMAP). An application which intends to process quality values explicitly should therefore normally map the quality array first, so that subsequent access to other NDF components does not involve further implicit access to the quality component. The following sequence of operations might be typical:

        CALL NDF_MAP( INDF, ’Quality’, ’_UBYTE’, ’READ’, QPNTR, EL, STATUS )
        CALL NDF_MAP( INDF, ’Data’, ’_REAL’, ’READ’, DPNTR, EL, STATUS )
  
        ...
  
        <process the data and quality information>
  
        ...
  
        CALL NDF_UNMAP( INDF, ’Quality,Data’, STATUS )

Here, the quality masking flag is set to .FALSE. by the first call to NDF_MAP, which accesses the quality component explicitly. The subsequent call to map the data component therefore ignores the quality values. Finally, the quality masking flag is returned to .TRUE. by the final call to NDF_UNMAP which unmaps the quality array.

The value of the quality masking flag associated with an NDF identifier may be determined at any time with the routine NDF_QMF:

        LOGICAL QMF
  
        ...
  
        CALL NDF_QMF( INDF, QMF, STATUS )

and a new value may be set with the routine NDF_SQMF:

        CALL NDF_SQMF( QMF, INDF, STATUS )

This allows the normal behaviour to be over-ridden if necessary. For instance, if direct access to an NDF’s data component is required without automatic quality masking occurring, then masking could be disabled as follows:

        CALL NDF_SQMF( .FALSE., INDF, STATUS )
        CALL NDF_MAP( INDF, ’Data’, ’_REAL’, ’UPDATE’, PNTR, EL, STATUS )

§A.8 contains an example of this technique in use.

8The use of a single include file to declare and define a statement function and its arguments is normally satisfactory. However, it sometimes makes it impossible to satisfy Fortran 77 statement order restrictions. For instance, if several such functions were defined in this way from separate subroutine libraries, it would not be possible to place all the declaration statements in front of all the function definition statements, as Fortran 77 requires. To cope with such problems, the single NDF_FUNC file may be replaced if necessary with the equivalent two separate include files NDF_FUNC_DEC and NDF_FUNC_DEF (in this order) which respectively declare and define the function.