6 Data formats in, and data migration to, the UNIX version

UNIX SPECX introduced yet another version of data formats (4.1 and 4.2) for spectra and for maps. It can read and write disk-FITS. From VERSION 6.6 onwards it can also read GSD files.

Version 4.1 spectral format is based on Starlink’s Hierarchical Data Structures (HDS), a binary format that is portable between different machine architectures. Thus you can take VERSION 4.1 spectra from a Sun workstation to an Alpha workstation and back, without noticing it.

Versions 4.1 and 4.2 map formats, too, are based on HDS. Thus maps are portable as well.

Data currently held on a VMS file system can be carried across to UNIX as detailed below. Note that the way back may not be possible, or at least difficult.

• GSD files: Data in GSD format can be copied binary. One way is the UNIX cp command if both disks are mounted by the UNIX machine. Another way is ftp in binary mode. Then use READ-GSD-DATA in the usual way.

  For going back from UNIX to VMS similar problems may arise as with disk-FITS files (see below).

• Disk-FITS files: These must be written with IBM byte order (swap order on Digital machines). They can then be copied binary. One way is the UNIX cp command if both disks are mounted by the UNIX machine. Another way is ftp in binary mode.

  This works between UNIX machines and from VMS to UNIX. If disk-FITS files are moved from UNIX to VMS, it may be necessary to make them files of fixed record length 2880 byte. cp is thus unsuitable. ftp in binary mode will create the right sort of file with the wrong record length of 512 byte. You can either rectify this with some VMS command, or use a more tolerant disk-FITS reader under VMS.

• Spectral data VERSION 2 and 3: These are VMS binary formats. Take a binary copy to the destination machine. The VMS Versions SPECX 6.2 and 6.3 can use it straight away, of course. For SPECX 6.6 the file has to be imported (converted into a VERSION 4.1 file) with the command CONVERT-VAX-FILE. This keeps the existing file (say file.dat) and creates an new file (file.sdf).
• Spectral data Version 4.0 and 4.1: These are portable binary formats based on HDS. Just take a binary copy to the destination machine (UNIX cp or ftp in binary mode). The file can be used straight away. Optionally you can convert it to the native flavour of the HDS format on the destination machine. Use the command native in the KAPPA package. This should speed up data access somewhat.

  Version 4.0 or 4.1 spectral data cannot be imported back into SPECX 6.2 or 6.3.

• Map files Version 2 and 3: These are VMS binary formats. Take a binary copy to the destination machine. The VMS Versions SPECX 6.2 and 6.3 can use it straight away, of course. For SPECX 6.6 the file has to be imported (converted into a Version-4.1 file) with the command CONVERT-VAX-MAP. This keeps the existing file (say file.map) and creates an new file (file_map.sdf).
• Map files Version 4.0: These are “local” binary formats written only by the beta-test version of SPECX 6.4. “Local” means they can only be read by the same type of machine that wrote them, either a Sun4 workstation or an Alpha workstation. These files can be converted to format Version 4.1 with CONVERT-VAX-MAP. In spite of the name of the command, the file will be assumed to have been written by the same UNIX machine that the command is run on. The existing file (say file.map) is kept and a new file (file_map.sdf) created.

  Version-4.0 map files cannot be imported back into SPECX 6.2 or 6.3.

• Map files Version 4.1: This is a portable binary format based on HDS. Just take a binary copy to the destination machine (UNIX cp or ftp in binary mode). The file can be used straight away. Optionally you can convert it to the native flavour of the HDS format on the destination machine. Use the command native in the KAPPA package. This should speed up data access somewhat.

  Version 4.1 map files cannot be imported back into SPECX 6.2 or 6.3.

• Map files Version 4.2: This is also a portable binary format based on HDS. The structures have been rearranged somewhat from V4.1 to improve storage efficiency and to try to prevent corruption of the file if specx crashes. Version-4.1 map files are automatically converted to V4.2.

Dump files and the mapplane.tmp files are not portable. Care has to be taken when the same file space is used with different machines, because SPECX may not run properly on one machine while there is a SPECX dump in the working directory that was written by another machine.