8 ACCESSING ARRAY COMPONENTS

This section explains how to access the array components of an NDF (i.e. its data, quality and variance components) in order to read values from them, or to write new values to them.

8.1 Overview of Mapped Access to Array Components

The NDF_ system provides access to the values in an NDF’s array components by means of a technique known as mapping. In this, an array is created in a region of the computer’s memory and a pointer to it is returned to the calling routine. Access to a selected NDF array component then takes place by means of the application reading and writing values to and from this array. For instance, if the component’s values are to be read, then the array would be filled with the requested values, which the application can then access.

The array of values stored in memory is usually made to appear to an application as if it were a normal Fortran array (with an appropriate numeric type and shape) so that the NDF component’s values may be accessed merely by referring to this mapped array in the normal Fortran way. If appropriate, modifications may also be made to the values in the array and these will be returned to the appropriate NDF component, updating it, when it is unmapped – a process which completes the transfer of values. At this point, the allocated memory is released and no further reference to the mapped values can be made unless the array is mapped again.

8.2 Mapping and Unmapping

The process of mapping one of an NDF’s array components for access may be performed using the routine NDF_MAP. For instance:

will map an NDF’s data component as an array of ‘_REAL’ (i.e. single-precision) values so that the calling routine can ‘READ’ them. NDF_MAP returns an integer pointer to the mapped values via its PNTR argument and a count of the number of array elements which were mapped via its EL argument. (PNTR is actually a 1-dimensional integer array, so the pointer value in this example will be returned in its first element.)

An application can access the mapped values as if they were a Fortran array of real numbers by passing the pointer to a subroutine. On computers which support it, this should be done using the %VAL facility (see below). Afterwards, access is relinquished by unmapping the NDF component using the routine NDF_UNMAP. The following illustrates the process:

Here, the call to NDF_MAP maps the NDF’s data component and returns a pointer to the mapped values. This pointer is passed to the subroutine SUMIT using %VAL, where it appears as a normal Fortran real array with EL elements.⁵ SUMIT can then access the mapped values, in this case summing them, before it returns. Finally NDF_UNMAP is called to unmap the data component, thereby releasing the resources that were used.

In the above example, the array was treated as if it were 1-dimensional, since its actual dimensionality was not important. This is sometimes the case, but if the shape of the array is significant, then a call to a routine such as NDF_DIM would be used to obtain the necessary dimension size information, which would then be passed to the subroutine which accesses the mapped values, for instance:

In this case, the routine SUMIT would declare the array to have the appropriate shape, and access it accordingly, as follows:

The routine NDF_MAP will also accept a list of component names and will map all of them in the same way (i.e. with the same type and mapping mode). Pointers to each mapped array will be returned via the PNTR argument, which must have sufficient elements to accommodate the returned values. The following shows how both the data and variance components of an NDF might be mapped for access, passed to a subroutine DOIT for processing, and then unmapped:

Note how NDF_UNMAP is also provided with a list of components to unmap. An alternative would be to specify a component name of ‘$\ast$’, which would cause NDF_UNMAP to unmap all components which had previously been mapped using the specified NDF identifier.

8.3 Implicit Unmapping

It is always necessary for an array component to be unmapped when access to it is complete. Failure to do this could result in an application running out of some resource, or failing to re-access the component at a later time. In the case where values are being written to an NDF component (described below), failure to unmap that component afterwards may also result in the new values being lost.

However, the need to call NDF_UNMAP explicitly can often be avoided because several other routines will perform this operation implicitly if necessary. For instance, if NDF_ANNUL is used to annul an NDF identifier (see §3.3), then any components which were mapped through that identifier will first be implicitly unmapped. The effect is as if:

had been executed first.

Unmapping is also performed implicitly by the routine NDF_END as part of the “cleaning-up” service it provides (see §3.4). Any NDF identifiers which are implicitly annulled by NDF_END will also have any associated components unmapped at the same time. Careful use of the routines NDF_BEGIN and NDF_END can therefore eliminate many calls both to NDF_UNMAP and NDF_ANNUL.

8.4 Writing and Modifying Array Component Values

The mapping modes ‘WRITE’ and ‘UPDATE’ may be specified in calls to NDF_MAP in place of ‘READ’ to indicate that new values are to be written to an NDF’s array component, or that its existing values are to be updated (i.e. modified), respectively. The mapped values can then be accessed in exactly the same was as for read access, except that any modifications made to the mapped values will be reflected in the actual values stored in the data structure. The transfer of these new values back to the NDF is completed when the array is unmapped, for instance:

Here, an NDF’s variance component is mapped for ‘WRITE’ access and passed to a routine SETVAR which assigns values to it. When NDF_UNMAP is called, these new values are transferred to the NDF. If the variance component was previously in an undefined state, it now becomes defined.

8.5 More About Mapping Modes

The mapping mode argument of NDF_MAP is used to indicate how the transfer of mapped values should be handled when an array component is accessed. The three basic mapping modes ‘READ’, ‘UPDATE’ and ‘WRITE’ each has a separate use, and the following describes how they operate in detail:

‘READ’ –

When an array component is mapped for ‘READ’ access, its pre-existing values are read into memory and made available to the calling routine (if the component’s state is undefined, then an error will result since there will be no values to read). When the component is unmapped, the mapped values are discarded. If any have been modified they do not result in a permanent change to the data structure (in fact the values should not be modified, because the region of memory used may sometimes be write-protected and this will result in a run-time error).

‘UPDATE’ –

When an array component is mapped for ‘UPDATE’ access, its pre-existing values are read into memory exactly as for ‘READ’ access (an error will similarly be produced if the component is in an undefined state). When the component is unmapped, any modifications made to the mapped values are written back to the NDF, which is therefore permanently modified.

‘WRITE’ –

When an array component is mapped for ‘WRITE’ access, an array is created to receive new values, but no pre-existing values are read in. Since the mapped array is un-initialised it may contain garbage at this point. All elements of the array must therefore be assigned values by the application before the component is unmapped, at which point these values will be written to the data structure to become permanent. ‘WRITE’ access will succeed even if the component is initially in an undefined state, and the component’s state will become defined once the unmapping operation has completed successfully.

8.6 Initialisation Options

Variations on the above basic mapping modes may be obtained by appending either of the two initialisation options ‘/BAD’ or ‘/ZERO’ to the mapping mode argument passed to NDF_MAP. These options specify the value to be supplied as “initialisation” in the absence of defined values. For instance, if ‘READ/ZERO’ is specified, thus:

then NDF_MAP will read the component’s values if available, but instead of producing an error if its state is undefined, it will supply an array full of zeros instead. In similar circumstances, the access mode ‘READ/BAD’ would supply an array full of the bad-pixel value appropriate to the numeric type in use.⁶ The same technique applied to update access (e.g. a mapping mode of ‘UPDATE/BAD’) has a similar effect, except that in this case the values, with possible subsequent modifications, will be written back to the array component when it is unmapped, causing its state to become defined.

The effect of initialisation options on write access (e.g. a mapping mode of ‘WRITE/ZERO’) is slightly different. In this case the initial state of the component is irrelevant, and any initialisation option is always used to initialise the array’s values when it is first mapped. These values, with subsequent modifications, are then written back to the NDF component when it is unmapped. Note that there is generally no point in using an initialisation option with write access if an application will subsequently assign values to all of the array’s pixels, but it may be useful if only certain pixels will subsequently be assigned values as it can save having to explicitly assign default values to all the others.

8.7 Implicit Type Conversion

An application seeking access to an NDF array component will not usually be concerned with the actual numeric type used to store the values and will request a type which suits whatever processing is to be performed. Single-precision (i.e. ‘_REAL’) values are normally most appropriate and have been specified in previous examples.

Nevertheless the third (TYPE) argument to NDF_MAP may be used to specify any of the seven numeric types supported by the NDF_ system (see §7.1) and a mapped array of values having the requested type will be made available. For instance:

will map an array of ‘_WORD’ (i.e. Fortran INTEGER$\ast$2) values, performing type conversion automatically if required. If the component’s values are being written or modified, then type conversion will also be performed, in reverse, when it is unmapped. The resulting “back-converted” values are used to update the NDF.

Unfortunately, type conversion takes time, so it may sometimes be worth avoiding if possible. This can be done, although only at the expense of some additional programming effort, by calling NDF_TYPE to determine the numeric type of a component which is to be mapped and using the same type for access, so that type conversion becomes unnecessary, for instance:

The application must then be prepared to explicitly process whatever numeric type is supplied, which will normally mean duplicating the main processing algorithm for each possible type. Such steps are not justified for normal use, but may sometimes be adopted for general-purpose software which will be heavily used. The GENERIC compiler (see SUN/7) can be useful for generating the necessary multiple copies of suitably-written algorithms if this approach is to be used.

8.8 Accessing Complex Values

Array components which hold complex values may be accessed using the routine NDF_MAPZ, which is identical to NDF_MAP except that it returns a pair of pointers for each component mapped; one for each of the real and imaginary parts. For instance:

will return pointers to the real (i.e. non-imaginary) and imaginary parts of an NDF’s data component, as double-precision values, via the arguments RPNTR and IPNTR. As with the routine NDF_MAP, a list of components may also be supplied and the RPNTR and IPNTR arguments must then have sufficient elements to accommodate the returned pointers. Unmapping of components mapped using NDF_MAPZ is performed in exactly the same manner as if NDF_MAP had been used.

Implicit type conversion can be performed between non-complex and complex types (and vice versa) if required, so the use of NDF_MAPZ is not restricted to array components which hold complex values. Equivalent comments also apply to NDF_MAP. A non-complex component accessed via NDF_MAPZ will be converted to have an imaginary part of zero, while a complex component accessed using NDF_MAP will have its imaginary part discarded.

8.9 Mapping the Variance Component as Standard Deviations

The values held in an NDF’s variance component are estimates of the mean square error expected on the corresponding values stored in its data component. This particular method of storage is chosen because it allows more efficient computation in the common situation where errors from two or more sources are to be combined. For instance, errors are usually combined by adding the contribution from each source in quadrature, i.e. if ${\sigma }_A$ and ${\sigma }_B$ are the errors associated with two independent values, then the error $\sigma$ in their sum is given by:

The corresponding variance values therefore combine by simple addition:

and this is a considerably faster computation to perform.

Nevertheless, the variance values stored in an NDF are unsuitable for direct use for such purposes as plotting error bars on graphs. In this case the square root of each variance estimate must be taken, in order to obtain an estimate of the standard deviation, which is the value normally employed. This can be done explicitly, but it is also possible to instruct the NDF_MAP routine to provide an array of standard deviation values directly by specifying a special component name of ‘Error’. For example:

would return a pointer to a mapped array of single-precision standard deviation (or “error”) values, obtained by taking the square root of the values stored in the NDF’s variance component. These values should later be unmapped in the usual way, either implicitly, or by an explicit call to NDF_UNMAP, thus:

If a mapping mode of ‘UPDATE’ or ‘WRITE’ had been specified, then the mapped values would be squared before being written back to the NDF’s variance component. (Note a component name of ‘Variance’ should be specified when unmapping an array of standard deviation values; it is only during the mapping operation that the name ‘Error’ may be used.)

The same procedure may also be used when obtaining mapped access to complex values using the routine NDF_MAPZ (§8.8). In this case, the process of taking the square root (or of squaring during the subsequent unmapping operation) will be applied separately to the real (i.e. non-imaginary) and imaginary parts.

8.10 Restrictions on Mapped Access

Certain restrictions are imposed on mapped access to NDF components, the most important being that each array component may only be mapped for access once through any NDF identifier. Thus, two successive calls to NDF_MAP (or NDF_MAPZ) specifying the same identifier and array component will result in an error. Of course, the component may later be re-mapped if it has first been unmapped, either by an intervening call to NDF_UNMAP, or implicitly.

One way of avoiding this restriction is to clone a duplicate identifier using NDF_CLONE so that a component can be mapped twice, once through each identifier. However, a more fundamental restriction exists if a possible access conflict may occur. Update and write access to NDF components is always exclusive, so the NDF_ system will reject any attempt to access a component twice if one of the attempts involves update or write access, although the same component may be accessed any number of times (through different identifiers) for read access alone.

In addition, arrays stored in scaled or delta form (rather than primitive or simple form), are currently read-only. An error will be reported if an attempt is made to map a scaled or delta array component for update or write access. If you need to modify the values in such an array, then you should first make a copy of the NDF. Copying an NDF automatically converts all scaled amd delta arrays into simple arrays, and so the copied array can be mapped for any type of access.