Differences between revisions 38 and 39
Deletions are marked like this. Additions are marked like this.
Line 20: Line 20:
 * All commands now have a Parameter FMT to select the format in which the output object should be written, if it is written to a text file. Allowed values are AST, XML, STCS, or any FitsChan encoding such as FITS-WCS.  * All commands now have a Parameter FMT to select the format in which the output object should be written, if it is written to a text file. Allowed values are AST, XML, STCS, or any !FitsChan encoding such as FITS-WCS.
Line 34: Line 34:
 * FINDCLUMPS GaussClumps method now excludes peaks below the specified threshold from the output catalogue and NDF.
 * Improvements to the FINDCLUMPS GaussClumps method has improved its ability to decompose merged clumps.
 * FINDCLUMPS !GaussClumps method now excludes peaks below the specified threshold from the output catalogue and NDF.
 * Improvements to the FINDCLUMPS !GaussClumps method has improved its ability to decompose merged clumps.
Line 255: Line 255:
 * Extended support for a variety of distorted projections within the FitsChan class, including SIP, TPV, TNX, ZPX, and SCAMP.
 * The PolyMap class can now use an iterative Newton-Raphson method to evaluate the inverse the inverse transformation if no inverse transformation is defined when the PolyMap is created.
 * All classes of Channel can now read to and write from specified text files, without the need to provide source and sink functions when the Channel is created. The files to use are specified by the new attributes SourceFile and SinkFile.
 * The FitsChan class has a new read-only attribute called "Nkey", which holds the number of keywords for which values are held in a FitsChan.
 * Frames now interpret the unit string "A" as meaning "Ampere" rather than "Angstrom", as specified by FITS-WCS paper I.
 * Extended support for a variety of distorted projections within the !FitsChan class, including SIP, TPV, TNX, ZPX, and SCAMP.
 * The !PolyMap class can now use an iterative Newton-Raphson method to evaluate the inverse the inverse transformation if no inverse transformation is defined when the !PolyMap is created.
 * All classes of Channel can now read to and write from specified text files, without the need to provide source and sink functions when the Channel is created. The files to use are specified by the new attributes !SourceFile and !SinkFile.
 * The !FitsChan class has a new read-only attribute called "Nkey", which holds the number of keywords for which values are held in a !FitsChan.
 * Frames now interpret the unit string "A" as meaning "Ampere" rather than "Angstrom", as specified by FITS-WCS Paper I.
Line 263: Line 263:
  * Attributes for axes of a CmpFrame with permuted axes could be indexed incorrectly
  * Inappropriate equinox values could be used when aligning SkyFrames if the AlignSystem attribute was set.
  * The astFindFrame method allowed a template Frame of a more specialised class to match a target frame of a less specialised class. For example, this bug would allow a template SkyFrame to match a target Frame.
  * Attributes for axes of a !CmpFrame with permuted axes could be indexed incorrectly
  * Inappropriate equinox values could be used when aligning !SkyFrames if the !AlignSystem attribute was set.
  * The astFindFrame method allowed a template Frame of a more specialised class to match a target frame of a less specialised class. For example, this bug would allow a template !SkyFrame to match a target Frame.
Line 272: Line 272:
 * Add function atlTableLutMap to create a LutMap from a Table column  * Add function atlTableLutMap to create a !LutMap from a Table column

Starlink Software Collection - Kapuahi Release Notes

Highlights

General

  • Kapuahi is required in order to reduce SCUBA-2 data.
  • New PAL library replaces SLA in all C applications.
  • CNF now used to provide mutex when entering C interfaces to Fortran routines. This allows multi-threaded C applications to call Fortran routines.
  • ADAM applications now respect the MSG_SZOUT environment variable to control the width of output messages.

Changes to Applications

ATOOLS

  • All commands now have a Parameter FMT to select the format in which the output object should be written, if it is written to a text file. Allowed values are AST, XML, STCS, or any FitsChan encoding such as FITS-WCS.

CCDPACK

  • Add NORM parameter to MAKEFLAT.
  • Correct provenance propagation in CALCOR.

CONVERT

  • SPECX2NDF has a new SYSTEM parameter that specifies the NDF's co-ordinate system.
  • NDF2FITS now avoids scaling when the range of data values lies within the scaled type's range, such as double to single precision.

CUPID

  • FINDCLUMPS GaussClumps method now excludes peaks below the specified threshold from the output catalogue and NDF.

  • Improvements to the FINDCLUMPS GaussClumps method has improved its ability to decompose merged clumps.

  • FINDBACK now uses mutiple processors or cores (if available) to speed up the the procesing of independent slices.
  • FINDBACK can now find and use an independent default noise value for each slice of supplied NDF.

DATACUBE

  • pvslice
    • New -p option to choose a plane in which to select the slice.
    • Fix bugs that failed to use the rotated slice, and prevented a slice whose direction was to lower pixel indices.

DIPSO

  • Fix reporting of unexpected input characters.

ESP

  • Pixel size calculation has been improved.

FIGARO

  • Fix workspace handling in line fitting on 64-bit systems.

FLUXES

  • Extend Mars model to 2103 (see astro-ph/0703640v1)
  • Use updated brightness temperatures for Uranus and Neptune based on Herschel calibration data.

GAIA

  • A snapshot can now be taken of the whole image, including the off-screen parts, so it is now possible to capture a graphic of the whole image at full or higher resolution when it is larger than the main window.
  • The "Intensity Map:" selector in the main panel has been replaced by a "Color Scale:" selector. This still provides linear and log-like intensity mapping of the displayed image, so no features have been removed. Color scaling gives a more careful transformation of intensity into log-like space, reducing the false contouring seen in low intensities. The square root colour scale is now also available from this control, as well as more negative mappings options. As part of these changes the colour scale algorithms themselves have been tweaked to give better log and square root mappings.
  • Errors in the default intensity and colormaps have been fixed to remove a discontinous change that could show up as a false contour (this has been present for many years).
  • When connecting the coordinate system of a catalogue to that of the displayed image the current coordinate system will now always be used in preference to any other available system. Previously if you had another celestial coordinate system in addition to the current one, that would be used in preference to the current one. This avoids problems with secondary coordinate systems that are incorrect or not as accurate as the current one.
  • FITS files with more than one period in their names will now be handled by the analysis toolboxes. Previously names like "a.b.c.d.fits" would give an error when a conversion to the NDF format was attempted. This uses enhancements to the CONVERT package.
  • Image slices will now use start and end points anywhere within the region of the displayed pixel. Previously the region of the end points started at the middle of the previous pixel and extended in the middle of the selected pixel.
  • Additional collapse estimators clipped mean and standard deviation added to cube analysis.
  • The Coords menus in the cube and spectral display now also offers a selection of standards of rest.
  • Tabulated photometric results have better precision and may use exponential format.
  • Several issues with the axes plotting in the three-dimensional visualisation toolboxes have been fixed. There is a new free-form entry area for AST attributes so that the axes can be configured.

KAPPA

  • General changes:
    • Colourmaps added from MRAO: Dave Green's cubehelix (in two forms), John Richer's colour spiral, and the SPECX bl2yl.
    • The default for the AXES parameter in HISTOGRAM, NORMALIZE, PSF, and SCATTER has changed such that by default axes are only drawn if the picture is cleared.
  • ARDMASK -- The VARIANCE and DATA arrays in the NDF can now be masked simultaneously by setting the COMP parameter to "All", which has become the default value for COMP.
  • FITSMOD -- new READONLY parameter permits read-only edits such as "Print" and "Exist" on write-protected NDFs, although FITSVAL and FITSEXIST tasks are recommended for these operations.
  • HISTOGRAM
    • Now permits plotting of the histogram for data-value limits that exceed the single-precision data range. Plot-bounds parameters have therefore changed type to _DOUBLE.
    • New Parameter WEIGHTS allows different weights to be associated with each value in the input NDF.
    • New Parameter WEIGHTSTEP defines the increment in weight value corresponding to a single integer histogram count.
  • LOOK
    • New format "wglist" has been added, which lists the WC co-ordinates and pixel values in an image, omitting bad pixels.
    • New mode "all" has been added, which lists the entire supplied NDF.
  • MFITTREND
    • Permits processing of very large datasets by dividing into manageable sections.
  • NORMALIZE -- The new ZEROFF parameter can be set TRUE to force the fit to use an offset of zero.
  • PASTE -- There is a new SHIFT parameter that applies constant shifts of origin between successive pasted data.
  • REGRID -- The new parameter AXES allows regridding of selected pixel axes.
  • SCATTER -- The Pearson coefficient of the data in the scatter plot is now displayed, and written to the CORR output parameter
  • SETAXIS
    • In WCS mode it tries to find the best WCS axis to use when the WCS axes are crossed or not parallel to the pixel axes.
    • "Linear_WCS" is a new MODE option. It is a variant of "WCS" where the axis centres are set to the least-squares linear fit to the values of the selected axis. This is useful for exporting co-ordinates to software with limited FITS WCS capabilities, such as no -TAB support, and when the non-linearity is small.
  • SHOWQUAL -- Add new QNAMES output parameter to list the attribute name associated with each QUALITY bit.
  • TRANDAT -- When AUTO=FALSE, the new POFFSET parameter allows a shift to be specified between the data co-ordinates in the input text file, and the pixel co-ordinates in the output NDF.
  • WCSALIGN -- If an input NDF has more pixel axes than the reference NDF, an attempt will now be made to align the input using the reference axes, whilst retaining the other axes. Thus, if a three-dimensional spectral cube is aligned with a two-dimensional image, the resulting output NDF would be a three-dimensional cube, aligned spatially with the image.
  • Bug fixes:
    • DISPLAY failed to plot and report an error when supplied a three-dimensional NDF that had a leading degenerate pixel axis and no WCS component.
    • Prevent crash when using co-ordinates in MATHS expressions.
    • Right ascensions above 12h used to cause WCSSLIDE to shift in the wrong direction.

OBSERVE

  • Now uses PGPLOT for plotting instead of GKS.

ORAC-DR

See http://www.oracdr.org/oracdr/ChangeLog

PAMELA

  • SEGV fixed in REGPIC.

PHOTOM

  • Improve formatting of results.

SMURF

  • New command UNMAKEMAP creates artificial SCUBA_2 data from a trial sky image and a set of real SCUBA-2 raw data files.
  • New command SC2MAPFFT.
  • New command SC2FILTERMAP.
  • New command MCEHEAD2CAT reads the MCE configuration from SCUBA-2 data files.
  • QLMAKEMAP has been removed. Use MAKEMAP instead.
  • MAKEMAP
    • Offers a new more-stable common-mode estimation. See [http://pipelinesandarchives.blogspot.co.uk/2012/05/changes-to-common-mode-estimation.html] for an illustrated explanation.

    • More code has been multi-threaded resulting in significantly shorter run-times on multi-core machines. See [http://pipelinesandarchives.blogspot.co.uk/2012/05/better-core-utilisation.html] for some numbers.

    • Pointing corrections for the input raw data can now be specified via a text file, using new parameter POINTING.
    • CONFIG parameters that specify a number of samples can now also be given in terms of seconds.
    • New CONFIG parameter "SAMPCUBES" creates an output cube that contains all the data values that are combined to make each output map pixel value.
    • New CONFIG parameter "COM.PERARRAY" allows a separate common mode signal to be estimated and used for each sub-array.
    • The COM model can now be masked to omit source samples from the common mode estimation.
    • The FLT model can now be masked to omit source samples when estimating the low frequency background of each bolometer time stream.
    • SNR masks for the AST, COM and FLT models can now be taken down to much lower levels without introducing isolated noise spikes into the mask.
    • A bug has been fixed that caused the memory requirements to grow monotonically when running MAKEMAP repeatedly as a monolith (e.g. from ORAC-DR or ICL).
    • New distortion model for the SCUBA-2 focal plane.
    • The REF parameter can now be used to specify and external mask.
    • The FLT model is now more robust.
    • Updated Water Vapor Monitor calculation. The WVM samples are now smoothed before being used.
    • New config parameter "GROUPSUBARRAY" to enable subarrays to be processed independently.
    • New config parameters "NOISECLIPHIGH" and "NOISECLIPLOW" control clipping of outlier noisy bolometers.
    • New config parameter "COM.NOFLAG" can turn off flagging of bolometers from common-mode calculation.
    • The ITERMAP config parameter can now be set to -1 to write out the final iteration of each chunk.
    • Setting config parameter DOWNSAMPSCALE to -1 will auto-scale to the pixel size. This is now the default.
    • New config parameter "AST.GAUSSBG" can be used to Gaussian smooth or background remove the map during iterations.
    • Added a new template matching model (TMP).
    • Updated extinction parameters.
    • A bug has been fixed that led to incorrect weighting and exposure times whem multiple chunks were being reduced simultaneously.
    • The variance calculation in the output map is much more robust.
    • New config parameter "MAPTOL" provides a map-based scheme for testing convergence.
    • The step-finder has been improved.
    • The option to use disk files during iteration (memiter=0) has been removed.
    • New config parameter "NOI.BOX_SIZE" can be used to weight subsections of time-series independently. This can correct for a bolometer suddenl$
    • A bad flatfield ramp now disables the associated science data rather than be ignored.
    • A new ADAM parameter NBOLOEFF contains the effective bolometer count from MAKEMAP.
  • JCMTSTATE2CAT
    • Calculate precipitable water vapor values.
    • Calculate telescope speed.
    • Bad values are now written out as NaN to provide better compatibility with TOPCAT.
  • CALCNOISE
    • CALCNOISE now accepts a map-maker config file for pre-cleaning. The default config includes common-mode removal.
    • Calculate the effective noise and NEP. This is the noise you would get if all bolometers were co-incident.
  • TIMESORT
    • Now works on an NDF section along the time axis.
  • SMAS
    • Now handles Moon data.
    • APERADD is now available using the -a option.
    • Output catalogues can be used to correct pointing in MAKEMAP.
  • SMURFCOPY
    • Output WCS now includes a focal plane coordinate frame (FPLANE).
  • MAKECUBE
    • Empty output tiles are now ignored, and any remaining tiles created as normal.
    • A bug has been fixed that caused the memory requirements to grow monotonically when running MAKECUBE repeatedly as a monolith (e.g. from ORAC-DR or ICL).
  • FIXSTEPS
    • Has two new output parameters. NFIXED returns the number of steps fixed, and NREJECTED stores the number of bolometers rejected.

SPLAT

  • A bug in the "remove" & "remove selected" options in the cutter toolbox has been fixed. After the last change the spectrum created did not have a correct coordinate value assigned to the removed region. This resulted in a non-monotonic spectrum.

  • A bug in the flip toolbox that always reported the radial velocity as zero in the log file has been fixed. The log now also includes whether the values are redshifts.
  • A bug that meant some NDF spectra opened in SPLAT using the "splatdisp" command could not be saved to disk. This has been fixed. Note this includes spectra sent from GAIA.

SURF

  • Addition of SKYDIP2 command to enable SCUBA-2 skydips to be reduced using SCUBA's atmospheric model.

TOPCAT

New Libraries

PAL

The PAL library is a re-implementation of the Fortran SLA library in C. It uses the SOFA library wherever possible and has a GPL licence. We now use the PAL library in SMURF, AST and most of the perl interfaces. Fortran SLA is used by Fortran applications. The PAL library's associated document is SUN/267.

THR

THR is a library to support multi-threaded applications in C. It was extracted from the SMURF application to provide a general infrastructure for other Starlink applications. THR provides a facility for creating thread queues, submitting jobs to the queue and waiting for jobs to complete. THR is documented in SUN/266.

SOFA

We now ship a copy of the thirdparty SOFA astrometry library. Kapuahi comes with version 2012-03-01.

Changes to Libraries

ARD

  • Use CNF to add mutex for C access
  • Minor fix to ARD AST plotting support.
  • Issues with discontinuities at 360 degrees longitude have been fixed.

ARY

  • Now has extra checks for inconsistent compressed data arrays.

AST

  • The AST distribution now includes a copy of the IAU SOFA library as a replacement for the obfuscated subset of the C SLALIB library used previously.
  • The number of libraries installed by AST has been reduced.
  • The Spitzer "-SIP" distortion code is now recognised within FITS headers that describe non-celestial axes, as well as celestial axes.
  • A Python wrapper for AST is now available.
  • Added support for the 2012 leap second.
  • Extended support for a variety of distorted projections within the FitsChan class, including SIP, TPV, TNX, ZPX, and SCAMP.

  • The PolyMap class can now use an iterative Newton-Raphson method to evaluate the inverse the inverse transformation if no inverse transformation is defined when the PolyMap is created.

  • All classes of Channel can now read to and write from specified text files, without the need to provide source and sink functions when the Channel is created. The files to use are specified by the new attributes SourceFile and SinkFile.

  • The FitsChan class has a new read-only attribute called "Nkey", which holds the number of keywords for which values are held in a FitsChan.

  • Frames now interpret the unit string "A" as meaning "Ampere" rather than "Angstrom", as specified by FITS-WCS Paper I.
  • Plotting of disjoint regions within a compound region is faster.
  • Bug fixes:
    • Attributes for axes of a CmpFrame with permuted axes could be indexed incorrectly

    • Inappropriate equinox values could be used when aligning SkyFrames if the AlignSystem attribute was set.

    • The astFindFrame method allowed a template Frame of a more specialised class to match a target frame of a less specialised class. For example, this bug would allow a template SkyFrame to match a target Frame.

    • Reading Plot3D objects through basic channels.
    • Correct text strings used to represent the "Labelling" attribute within dumps of a Plot. Previously they were reversed.

ATL

  • New function atlReadTable to create an AST Table from a disk file.
  • Add function atlTableLutMap to create a LutMap from a Table column

  • Add F77 wrapper for astChrSplitRE

CNF

  • Remove support for old-style CNF API (13 years is long enough to migrate over). cnf_XXX should now be written as cnfXxx.
  • Use a mutex to synchronise all Fortran library calls from C. Use the new F77_LOCK macro.

EMS

  • Minor fixes for a memory leak and malloc handling.

GRP

  • Use CNF to add mutex for C access.
  • Correct conversion of Fortran identifiers to C pointers.
  • grpValid returns zero without error if the supploed Grp pointer is invalid.
  • Add new routine GRP_SLAVE.
  • Add grp_own grp_sown and grp_head to C interface.

HDS

  • Use CNF to add mutex for C access.
  • Add missing datGet0UW prototype to header file.
  • The datErmsg API has been changed so that an external buffer is supplied.
  • Minor internal fixes and compiler warning cleanups.

IMG

  • Use CNF to add mutex for C access.

IRQ

  • Use CNF to add mutex for C access.

KAPLIBS

  • KPG1_ASGDP now ensures that the returned point is at least 10% of the box width away from the origin of both input and output Frame, as the origin has unusual properties.
  • KPG1_ASGET uses the appropriate precision when testing AXIS monotonicity.
  • FTS1_EDFEX and FTS1_EDKEY use read access if none of the editing operations change anything.
  • New routine KPG1_CORRx calculates the sample Pearson correlation coefficient between two arrays.
  • KPG1_FRACx generates a new error code KPG1_ _SPARS when the data are too sparse for its algorithm.
  • KPG1_GRAPH and KPG1_GRPHW have a change API. An additional two-element array scales the X, Y values.
  • KPG1_GRPHW now uses _DOUBLE parameters the plot limits, andsets a dynamic default for Parameter AXES so that by default axes are only drawn if the existing picture is cleared.
  • KPG1_GHSTx has two new arguments that control the weighting of values into the histogram bins.
  • Use CNF to add mutex for C access.

MERS

  • Fix bug such that a value of 0 for MSG_SZOUT and MSG_STREAM environment variables is accepted.
  • Use CNF to add mutex for C access.

NDF

  • Allow foreign format files to have dots in the base name.
  • Fix line breaking of long parameters in history.
  • Fix problem using WCS sections with NDFs that have swapped WCS axes.
  • Use CNF to add mutex for C access.

NDG

  • Add convenience function for storing multiple ancestors in an NDF.
  • Allow foreign format files to have dots in the base name.
  • Use CNF to add mutex for C access.
  • Many fixes to provenance handling.

ONE

  • Add one_strtod for converting strings to doubles.

PAR

  • Use CNF to add mutex for C access.

PCS

  • Allow the user to control messages through environment parameters. msgTune is now called in the ADAM fixed part.
  • Use CNF to add mutex for C access.

PDA

  • Use CNF to add mutex for C access.

SHL

  • Use CNF to add mutex for C access.

SLA

  • Added the 2012 leap second.
  • Removed the C slaEuler code and now use the Fortran as before.
  • Use CNF to add mutex for C access
  • Add C wrappers for slaPreces, slaPvobs and slaMapq.

Starlink: Kapuahi (last edited 2020-09-23 19:17:29 by MalcolmCurrie)