New output NDFs may also be generated by a process termed propagation, in which a new structure is created based on an existing template NDF. This is the most common method of creating an NDF to contain output from a processing algorithm, and is typically used whenever an application draws input from one or more NDFs and produces a new output NDF as a result.
As far as the user of such applications is concerned, the output dataset would typically be based upon one of the input datasets; i.e. it might inherit its shape and component type, storage form and possibly values from an input dataset.13 Of course, the output data structure would also incorporate whatever changes the processing algorithm is designed to perform.
Seen from within such an application, the purpose of propagation is to create a “skeleton” output NDF based on an input structure, but containing “blank” (i.e. undefined) components into which calculated results can be inserted. Usually, there will also be “non-blank” (i.e. defined) components in the newly-created NDF, which derive their values directly, without change, from one of the input datasets. Such components are said to have been propagated.
The way in which components (and extensions) are selected for propagation is central to the philosophy of the NDF and it is important to understand the principles if you are to write applications which process NDFs consistently. There are two sets of propagation rules which apply separately to standard NDF components and to extensions. The distinction between them is explained in the following two sections, after which the way in which these ideas are implemented in practice using NDF_ routines is described.
Because the meaning and interpretation of the standard NDF components (data, variance, quality, etc.) is well-defined, it is always possible to decide into which of three categories each of these components falls when writing an application. This then dictates the action which should be taken, as follows:
For instance, if an operation is to be performed on the data component, then an appropriate operation must usually also be performed on the variance component (if defined) so that it continues to represent the variance of the data in the output NDF. If the application cannot perform the necessary operation, then any component which would become invalid as a result falls into category 3 below (and is simply ignored).
For instance, the special case of adding a constant to the data component would not render the variance component invalid. It may therefore simply be propagated (i.e. copied) to the output NDF unchanged. Most applications which perform pixel-to-pixel processing and do not change the shape of an NDF can also propagate the axis and quality components in the same way.
It may not be possible to ensure that some NDF components will retain their validity after processing. This may simply be because an application (or perhaps an entire software package) chooses not to support certain NDF components; this is quite acceptable behaviour. Alternatively, the meaning of certain components may be destroyed by certain types of processing; for instance the validity of the quality component cannot possibly survive a Fourier transform operation applied to the data component no matter how sophisticated the software. In either case, the affected component(s) must be ignored and not propagated. This means that they will be lost from the output NDF.
The purpose of these rules is to ensure that the validity of all the standard NDF components is retained throughout all stages of processing, and that all defined components in an NDF always have valid values. If an application cannot guarantee this for any component, then it must ignore that component so that it remains undefined in the output data structure.
The propagation of extensions is necessarily different from the propagation of standard NDF components, because only certain pieces of software may recognise any particular extension. There is therefore no way in which other applications can judge whether the processing they are performing will render the information in the extension invalid. This is an unavoidable consequence of extensibility.
Rather than taking a pessimistic view and automatically deleting all unrecognised extensions, propagation of extensions proceeds along more optimistic lines, as follows:
For instance, an application in a software package which recognised an ‘IRAS’ extension should always check for the existence of such an extension and ensure that its contents were not invalidated by the processing it performs, making changes to the extension to achieve this if necessary.
For instance, an application may recognise an extension but be unable to ensure its continued validity after processing, either due to a limited implementation or more fundamental causes. In either event, the application should suppress propagation of that extension so that it is lost from the output data structure.
There may be any number of extensions present in an NDF which a particular application does not recognise. These should be ignored, and the default action of the NDF_ system will then be to propagate (i.e. copy) them to the output NDF.
The purpose of these rules is to ensure that extension information is always retained unless it is certain that it will no longer be valid. This contrasts with the rules for processing standard NDF components (§14.2) which are retained only if it is certain that they will still be valid.
Of course, the rules for propagating extensions carry the risk that an application which does not recognise an extension will inadvertently render it invalid. However, if all applications within a software package recognise the same extension(s), then this can only happen if software from several packages is intermixed. It then becomes the user’s responsibility to check the validity of information held in extensions.
Fortunately, the rules above are far easier to apply in practice than they might appear, and typically amount simply to deciding which NDF components (or extensions) will not need any processing performed on them. These components are propagated and any that remain (and which the application chooses to support) are then processed. All others are ignored.
Propagation is performed by the routine NDF_PROP, which creates a new NDF structure via a parameter, based on a template NDF which already exists. At the same time, it will selectively copy components and extensions present in the template structure and use them to initialise the corresponding components (and extensions) in the new NDF. For example:
will create a new output NDF, associate it with the parameter ‘OUT’ and return an identifier for it via the INDF2 argument. The new NDF is based on the template structure with identifier INDF1, and inherits its shape and the type and storage form of its components. Subsequent changes may be applied to any of these inherited attributes if required, e.g. by calling NDF_STYPE to change the numeric type of any of the new NDF’s array components (see §7.4).
The second (CLIST) argument to NDF_PROP specifies a list of components and extensions which are to have their values propagated (i.e. copied) to initialise the new NDF. The default (blank) value specified above causes all extensions, together with the title, label and history components to be propagated, if present. These three standard components are considered “safe”, in that they are likely to retain their validity through most common types of processing which takes place on NDFs.
Note, however, that no axis, data, variance, quality or units information will be propagated by default. This allows an application to explicitly process these components to generate new versions for the output structure if possible, or simply to ignore them and have them omitted from the output NDF if the necessary processing cannot be performed.
If you are certain that a component will not be rendered invalid by the processing which an application performs (for instance the addition of a constant to the data component would leave the axis, quality and variance components valid), then propagation of these components can be specified by listing them in the CLIST argument to NDF_PROP, thus:
These components would then be copied to the output NDF and would not need to be explicitly considered again by the application.
Conversely, if any of the standard NDF components which are propagated by default would be rendered invalid by the processing an application performs, then propagation may be inhibited by specifying the component name with the prefix ‘NO’ in the CLIST argument to NDF_PROP. For instance:
would force propagation of the quality component, but inhibit the default propagation of the label component.
Propagation of specific extensions may also be inhibited by specifying ‘NOEXTENSION()’ in the CLIST argument to NDF_PROP and listing the extensions to be omitted between the parentheses. For instance:
would force propagation of the axis components, but inhibit the propagation of any ‘IRAS’ or ‘ASTERIX’ extension which may be present. Note that ‘NOEXTENSION’ may be abbreviated but extension names must appear in full, although mixed case is permitted.
This mechanism can be useful if an extension may contain a large amount of information but is not required in the output data structure. The alternative (of propagating the extension, then deleting it) is less efficient and may lead to an unnecessarily large output file.
An asterisk may be used to indicate “all extensions”. Thus
would inhibit the propagation of all extensions. Likewise,
would propagate just the IRAS extension.
13Where there is more than one input NDF, one of them should be designated the primary input dataset and be used as the template for the output dataset. By convention, this should be the first input NDF acquired by the application and the first to be described in documentation.