The history component of an NDF provides a way of recording the processing operations which are performed on the dataset. It consists of a series of history records (with some ancillary information) each of which contains information about a particular processing operation and when it occurred. Normally, each of these operations will correspond with the execution of a single application which has either modified or created the NDF.
It is intended that the user of NDF applications should have considerable control over the recording of history information, but that writers of applications should not generally need to concern themselves with how this is achieved. Indeed, if a user has access to basic utilities for manipulating the history component of NDFs, then new applications will automatically provide history recording facilities without their authors making any specific provision for it. Most programmers may therefore need to read very little of this section except in special cases where explicit control over history recording is required.
As with other NDF components, the history component has a state, which is represented by one of the two values .TRUE. or .FALSE.. If the state is .FALSE., the history component is not defined, and the NDF contains no history information and no reference can be made to its history component. However, if the state is .TRUE., then the history component is defined and enquiries can be made about it. In this case, history records may also be present which describe the past processing history of the NDF. If it is a new NDF, however, it is possible that no such records may yet have been written to the history component.
The state of an NDF’s history component can be determined using the NDF_STATE routine by specifying a component name of ‘History’, thus:
The state information is returned via the LOGICAL argument STATE.
Unlike other NDF components, the history component does not become defined simply by writing to it (i.e. by the act of recording new history information). This is because recording of history information is not always required. It can, for instance, lead to the use of large amounts of file space or of additional processing time which is not justified, so resetting the history component (setting its state to .FALSE. – see §22.4) provides the user of an application with one way of disabling history recording when it is not required. In general, one does not want this action negated by the next application which attempts to record further history information.
Before history information can be recorded in an NDF, it is therefore necessary to explicitly define a history component to receive it. The NDF_HCRE routine will perform this, as follows:
NDF_HCRE will return without error if a history component is already defined. Otherwise, it will initialise a new one, making the NDF receptive to new history records (although there will initially be none of these present).
In addition, if the “NDF_AUTO_HISTORY” environment variable or “AUTO_HISTORY” tuning parameter is set to a non-zero integer (see NDF_TUNE), then a History component will be added automatically to NDFs created using either NDF_CREAT or NDF_NEW.
The opposite process, of resetting the history component to an undefined state, may be performed using the NDF_RESET routine with a component name of ‘History’, thus:
This has the effect of rendering the history component undefined and of deleting all the information it contains. Because this operation is irreversible and destroys all previous history information, it is performed only if explicitly requested. In particular, the history component remains unaffected when a “wild-card” component name of ‘’ is given to NDF_RESET, or when an existing NDF is opened for ‘WRITE’ access (circumstances in which other NDF components are normally automatically reset).
A less drastic method of disabling history recording while retaining earlier information is controlled by the history update mode which is described in §22.8.
As its name suggests, default history recording provides a means of recording information in the history component of an NDF in cases where the programmer has not taken any explicit action to generate this information within an application. This is expected to be the norm.
For default history recording to occur, the NDF’s history component must be in a defined state (as explained above). Control over this is normally in the hands of the user of the application, who can run a separate utility to modify the history component of specified NDFs and hence nominate which of them are to accumulate history information.
In addition, the NDF must have been opened either for ‘WRITE’ or ‘UPDATE’ access, so that the current application will have modified (or created) it. Some further conditions are also necessary, as discussed later, but no explicit action is required on the part of the programmer when writing the application. Thus, default history recording is available to even the most rudimentary of NDF applications.
Default history information is normally written to the history component of an NDF when it is released from the NDF_ system, either by a call to NDF_ANNUL which annuls the last valid identifier for the NDF, or implicitly via the cleaning-up action of the NDF_END routine (see §3.4). In any event, the information written will be the same, consisting of the following items:
| APPLICATION – | Name of the application which wrote the information |
| DATE – | Date and time the information was written |
| USER – | User ID for the person who ran the application |
| HOST – | Name of the machine on which the application was running |
| REFERENCE – | Reference name, which fully identifies the NDF dataset |
| TEXT – | Free-format text containing any additional information |
The first five of these items contain “fixed” information which is always automatically associated with any new history record and which may be read back at a later date and acted upon.23
The final TEXT item is “free format” and there is no restriction on what this may contain. It is available for applications to write their own history information, either to augment or to replace that written by default. When written by default, this history text will typically record the command line used to invoke the application (or, depending on the programming environment in use, the values of its parameters) together with the name of the file which was executed. The precise content and format of the default history text may depend on a number of factors,24 so you should not write software which depends upon such details.
Not all NDF data structures are created from scratch and then successively modified in situ. More commonly, they are produced by copying a certain amount of information from related input datasets while applying the required modifications in the process. In this situation, one of the input datasets is normally regarded as primary, and it is from this that the output NDF inherits most of its ancillary information by the process termed propagation (see §14).
To be complete, a description of the processing history of an NDF generated in this way would need to contain the processing histories of all the input datasets which have contributed to it. However, it is generally regarded as unnecessary to retain this wealth of history information as it would lead to exponential growth in the amount of information to be processed and stored. A more practical proposition, and the one supported by the NDF_ library, is to propagate the past history from only the primary input dataset to the output, and then to update this by appending a new record to reflect the action of the current application.
As a consequence, a complete record of all previous events will not be present in the new NDF. However, if the history information recorded by each application includes the full names of its input datasets, then these can be inspected separately to recover any further information. This keeps the amount of history information within reasonable bounds. Because the NDF_ library stores the name of the NDF in which a history component resides whenever it creates a new history record,25 an audit trail is automatically produced which allows the “ancestors” of any dataset (i.e. those datasets from which the history component has previously been propagated) to be identified.
Propagation of history component information takes place in a similar way to the propagation of any other NDF component (see §14) and is typically performed by the NDF_PROP (or NDF_SCOPY) routine. In this context, the history component is considered “safe” in the sense that its validity is not affected by the processing performed by most applications. It is therefore propagated by default, and you must explicitly specify if you do not want it to be propagated. This means that in practice most applications need take no action to ensure that history information is kept, since it will be maintained automatically via the propagation and default history recording mechanisms.
Some applications create new NDFs from scratch using (for instance) NDF_NEW or NDF_CREAT, rather than by propagation via NDF_PROP. In many cases, such applications will still want to propagate history information to the new NDF from some specified input NDF. This can be achieved using NDF_HCOPY.
When writing an application, you have the option of accepting the default history recording provided by the NDF_ library, or you may want to add further information of which the NDF_ system is unaware (a record of the results of a calculation, for instance). Not only is it possible to add such information, either prefixing or appending it to the default history text, but it is also possible to replace the default history text entirely if required.
The routine which allows this is NDF_HPUT, which writes new textual information to a history record, thus:
In this example, NDF_HPUT is being used to append two lines of text, stored in the TEXT array, to the text of the current history record for an NDF – remember, a new history record is normally created for each application which executes. (The HMODE, APPN, TRANS, WRAP and RJUST arguments can be ignored for the moment; their use is described in subsequent sections.)
If this is the first history information to be written to this particular NDF by the current application, then there will not yet be a history record to contain it. In this case, NDF_HPUT will create a new record and initialise it, recording the date and time and other standard information. The text provided will then form the first two lines of the text associated with the history record. If a current record already exists, NDF_HPUT will instead simply append the text to whatever may have been written earlier.
Normally, default history information is not written by the NDF_ system until an NDF is released, so information written by NDF_HPUT in the manner above will be prefixed to this default information. If a different order is required, then the routine NDF_HDEF may be used to force the default history information to be written prematurely, thus:
It will then not be written when the NDF is released. This makes it possible to interleave explicit and default history text in any order.
It is also possible to prevent default information from being written at all by setting the REPL argument of NDF_HPUT to .TRUE., indicating that the new text is to replace that provided by default. If the default information has not yet been written (normally the case), it will be completely suppressed by this action. A single such call to NDF_HPUT is sufficient and default history recording cannot then be re-established until the NDF has been released (or a new application started – see §22.18).
Each history record has a date and time associated with it. By default, this will be the UTC date and time at which the history record was created. However, an alternative date and time can be specified by calling the NDF_HSDAT routine before the history record is created.
As already indicated, there may be circumstances where finer control over the recording of history information is required than is provided simply by the state of the history component. For instance, you might want to disable further recording of history information for an NDF without losing the record of its past processing history. There may also be cases in which rather brief history information is required in order to save file space, or other circumstances where the fullest possible information is required (perhaps if the processed data were intended for archiving).
These requirements are met by the history update mode associated with an NDF’s history component. It may have any of the following states, with the associated meanings:
| ‘DISABLED’ | No history recording is to take place |
| ‘QUIET’ | Only brief history information is to be recorded |
| ‘NORMAL’ | Normal history recording is required |
| ‘VERBOSE’ | The fullest possible history information is required |
When a history component is first defined, its update mode defaults to ‘NORMAL’. This may subsequently be altered using the NDF_HSMOD routine, so long as ‘WRITE’ access is available for the NDF, as follows:
The update mode string may be abbreviated (to not less than 3 characters).
The update mode can be retrieved from an NDF using NDF_HGMOD.
The history update mode is a permanent attribute of the history component and remains with it to affect subsequent recording of history information, although it may be given a new value at any time. It affects both default history recording and (potentially) history information written explicitly by applications, according to its value at the time the information is written.
As might be expected, the HMODE argument of NDF_HPUT (set to ‘NORMAL’ in the example in §22.7) corresponds with the history update mode argument of NDF_HSMOD. In this case, it specifies the “priority” which the history information should have and takes one of the following values:
| ‘QUIET’ | Highest priority |
| ‘NORMAL’ | Normal priority |
| ‘VERBOSE’ | Lowest priority |
The text supplied to NDF_HPUT will actually be written to the NDF only if:
Otherwise, NDF_HPUT will simply return without action (or error).
The purpose of this is to allow applications to tailor their explicit recording of history information to the setting of the history update mode attribute of the NDFs they are processing. Thus, most history text might be classed as ‘NORMAL’ but additional information classed as ‘VERBOSE’ could also be provided and would only be available to users who request it (by setting the history update mode of their NDFs appropriately). Really vital information should be classed as ‘QUIET’, and will then always be delivered unless history recording has been completely disabled.
If the HMODE argument of NDF_HPUT is left blank, it defaults to ‘NORMAL’.
The NDF_HPUT routine facilitates the inclusion of variable values in history records by allowing message tokens to be optionally embedded within the lines of text supplied to it. If this facility is required, then the TRANS argument should be set to .TRUE. to request message token translation, and the tokens used should be defined beforehand – for instance, by calling the appropriate MSG_ routines (see SUN/104 which includes a full description of message tokens). In the following example, the results of an earlier calculation are inserted into a history record in this way:
The text stored in history records may have any width (or line length) ranging from 1 to NDF__SZHMX26 characters, but all the lines of text associated with a given record have the same width (i.e. the same upper limit on the number of characters they may contain). This width is determined when the first line of text is written to the record.
If the first line of text in a history record is written by the NDF_ system itself (when writing default history information, for instance), then the text width for that record will be initialised to NDF__SZHIS27 characters. This is the recommended width for history text. However, if the NDF_HPUT routine writes the first line, then the text width will be determined by the length of each element of the TEXT array supplied to it, as returned by the Fortran intrinsic LEN function. An error will result if this is too long (i.e. exceeds NDF__SZHMX).
Because the text width associated with a history record is not fixed, there can sometimes be difficulty in knowing how to format history text correctly, especially when appending to an existing record. In principle, it would be possible to use the enquiry routine NDF_HINFO to determine the text width and to act accordingly, but in practice this is cumbersome and rarely worth the effort. In addition, if message tokens are present within the text, it becomes difficult to predict the length of each line after token substitution has taken place.
To address this problem, NDF_HPUT provides several formatting options aimed at reconciling the possibly different widths of the text supplied (after message token substitution if appropriate) and the history record into which it must be written. The guiding principle is that no history text should ever be discarded as a result of limited line length, and that as far as possible it should remain legible.
The simplest formatting option is selected by setting both the WRAP and RJUST arguments of NDF_HPUT to .FALSE., in which case input text lines will simply be broken if they exceed the width available and continued on a new line (line breaking will take place at a suitable space if possible). This option is normally most suitable for “fixed-format” output of results.
If its WRAP argument is set to .TRUE., NDF_HPUT will instead perform “paragraph wrapping” on the text supplied. In this case, the text will be re-formatted so that the maximum number of words appears on each line, with words separated by single spaces. Re-formatting will not occur across “paragraph boundaries” marked by blank input lines. This option is most suitable for prose-style continuous text, particularly if it contains elements of unknown length such as message tokens. It also has the advantage of minimising the space needed to store the text.
Finally, if the RJUST argument of NDF_HPUT is set to .TRUE., it will “right justify” the text wherever a line is broken for continuation on the next line (i.e. after first performing either of the formatting operations described earlier if necessary). This results in lines of the final text being padded with blanks so as to produce a uniform right margin occupying the full text width (normally the right margin would remain ragged). When wrapping text, the final line of each paragraph, which may naturally be shorter than other lines, is not padded, and neither are unbroken lines if wrapping is disabled. This option is largely cosmetic but may improve the readability of some types of text. It does not affect the amount of space used.
One event which it is usually very important to know about is the failure of an application to complete properly. This could be because something unrecoverable went wrong and the application decided to terminate prematurely, or it could simply be that the user requested that it should abort. In either case, it is important that any datasets modified by that application should contain some record that it did not complete normally, so that they can later be identified and their contents regarded with appropriate suspicion.
The NDF_ library provides this service automatically via its history recording mechanism. To do this, it inspects the value of the STATUS argument passed to any routine which can potentially cause NDF datasets to be released from the NDF_ system (NDF_ANNUL and NDF_END). If the STATUS value is not equal to SAI__OK,28 this is taken to indicate that an error has occurred and that any NDFs released at that point may contain erroneous data. To record this fact, the value of STATUS and the text of any pending error messages (as previously reported through the ERR_ and EMS_ routines – see SUN/104 and SSN/4) will be appended to the current history record before each NDF is released. Rather similar action is also taken by the special routine NDF_HEND (although it does not itself cause NDF datasets to be released – see §22.18).
This error recording facility operates somewhat like NDF_HPUT with an effective priority of ‘QUIET’, so recording of errors will take place regardless of the current history update mode setting. However, it will not occur if history recording is completely disabled.
After several applications have processed an NDF and recorded history information, the NDF’s history component will contain a series of records, each identified by a date and time, which document its processing history.
You can determine how many such records exist, either with the general history enquiry routine NDF_HINFO, or more directly using the NDF_HNREC routine, thus:
The number of records is returned via the INTEGER argument NREC. Note that this value may (or may not) include a history record for the current application, depending on whether any history information has yet been written by it. The value may also be zero if no history information has ever been written.
Information about a particular record may be obtained by calling NDF_HINFO and supplying the record number.29 For example, if history recording has been in operation, you could obtain the name of the application which most recently modified an NDF as follows (the result is returned in the APPN variable):
Other information about individual records may also be obtained using NDF_HINFO, such as the text width, the number of text lines written and, of course, the date and time associated with the record (see Appendix D for full details of the information items available).
In addition to the record number, the date and time form a convenient means of addressing history records which are, necessarily, stored in chronological order. To facilitate this, the NDF_HFIND routine provides a search facility which allows records to be identified by date and time:
Given a date and time (with the years, months, days, hours and minutes fields stored in the YMDHM array and the seconds field held in SEC), this routine will return the number of the first record written after the specified date and time. A record number of zero is returned if no such record exists.
In this example, the EQ argument is set .FALSE., which instructs the routine not to report an exact date/time match. By setting it to .TRUE., an exact match would be permitted. This makes it possible to select records according to any constraint on the date and time of creation, using combinations of the following techniques:
The history text is perhaps the item of most interest associated with a history record. No routine is provided for accessing this directly, but NDF_HOUT may be used to pass it to a service routine, which may then handle it in any way required. Most commonly, this will involve displaying the text, for which a default service routine NDF_HECHO is provided. NDF_HECHO is a very simple routine and is described in Appendix D, but an outline of how it works is given here:
To invoke this as a service routine, it is passed as an argument to NDF_HOUT, as follows:
Note that it must be declared in a Fortran EXTERNAL statement.
NDF_HECHO provides a template if you wish to write your own service routine. By doing so, it is possible to perform other types of processing on history text. For instance, the following skeleton routine sets a logical value indicating whether the text of a history record contains a given sub-string (stored in the variable KEY), thus allowing a simple keyword search through history records to be performed:
Note that additional data must be passed to and from such a service routine through global variables held in common blocks.
The NDF_ library does not allow you to re-write history. However, it does allow past history records to be deleted so that irrelevant information can be removed, usually to save space. The routine NDF_HPURG will perform this, as follows:
Here, the history record numbers IREC1 and IREC2 specify a range of existing records to be deleted. Once these have been removed, any remaining records are re-numbered starting from 1.
It is recommended that users of NDF applications should normally invoke this routine via a utility application which records its own history information in the NDF. Thus, while the processing history may subsequently be incomplete, a record showing that deletion has occurred would still be present.
Whenever a new history record is written by the NDF_ library, the name of the currently executing application is stored with it. The intention is that this should subsequently allow the software which was used to be identified as fully as possible. In practice, there are several possible sources from which the name of an application may be obtained, so a system of defaulting is used which operates as follows.
The first place in which the NDF_ library looks for an application name is the APPN argument of the NDF_HPUT routine. If this is not blank (and a new history record is being created by this routine), then the value supplied is used directly. This means that the writer of an application which explicitly writes history text via NDF_HPUT can also explicitly supply the name of the application as it is to appear in any new history record.
Whether it is a good idea to supply a name in this manner depends on the circumstances. The advantage is that the name given can be very specific (for example it might contain a version number for the application) and can easily be updated if the application is changed. Conversely, it makes it more difficult to re-use that software in another application and users may be confused if they invoke the application via a command whose name differs from that which is “hard-wired” into the software.
If the APPN argument to NDF_HPUT is blank, or if a new history record is being written by some other means (by the default history recording mechanism, for instance), the NDF_ library next looks to see if a “current” application name has previously been defined and, if so, it uses it. A current name may be declared at any time by calling the routine NDF_HAPPN, for example:
and may be revoked by calling the same routine with a blank first argument.
This method is best used in situations where several applications may be invoked by a single executable program. For instance, some software packages have a structure similar to the following, where a command is repeatedly obtained and the appropriate routine (application) is then invoked:
In such a situation, the NDF_ library has no way of telling when one application has finished and a new one is beginning, so it cannot generate separate names for them. This problem is solved by including a call to NDF_HAPPN before invoking each application, as above.
The final method of obtaining an application name is used only if both of the above methods have failed to produce a (non-blank) name. In this case, the NDF_ library will use a default method, whose precise details may depend on the software environment or operating system in use. For instance, the name of the currently executing file might be used.
The main disadvantage of allowing the NDF_ library to generate the application name itself is that it will not contain precise details (such as a software version number) and may not always be able to distinguish between separate applications invoked from within a single executable program, as in the earlier example. Nevertheless, it is recommended that this method should be adopted first, and either of the above methods substituted if it proves inadequate.
One final consideration may apply if multiple applications are to be invoked from a single program (as in the example in §22.17) but some “global” NDF data structures are required to remain in use between applications. This is not a common requirement, but since default history recording is normally initiated only by the action of releasing an NDF, it will not occur in such a situation, so the global datasets will not automatically receive history information. To overcome this problem, a call to NDF_HEND may be used at the end of each application, as follows:
This notifies the NDF_ system that the current application has finished. It causes default history information to be written, if required, to all NDFs currently in use and ensures that new history records will be created to contain any history information written subsequently (i.e. by the next application). NDF_HEND then revokes any “current application” name established via the NDF_HAPPN routine (see §22.17).
If its STATUS argument indicates an error condition, NDF_HEND also records the STATUS value and pending error message information in any suitable history components belonging to NDFs currently in use, as described in §22.12.
23Except that the USER, HOST and REFERENCE items (marked by open circles) may not always be present in history records written by applications which do not use the NDF_ library. In this case, enquiries about them will simply result in a blank value being returned. Also, DATE can optionally be set to an alternative, user-supplied value using the NDF_HSDAT routine.
24Such as the software environment or operating system in use, the version of the NDF_ library, and the history update mode – see §22.8.
25This information is stored in the REFERENCE item – see §22.5.
26The NDF__SZHMX constant is defined in the include file NDF_PAR and currently has the value 200.
27Also defined in the NDF_PAR include file, currently to be 72 characters.
28As defined in the include file SAE_PAR.
29History record numbers start at 1.