Computes ordered statistics for an NDF’s pixels using an histogram
the pixel sum,
the pixel mean,
the pixel median,
the pixel mode,
the pixel value at selected percentiles,
the value and position of the minimum- and maximum-valued pixels,
the total number of pixels in the NDF,
the number of pixels used in the statistics, and
the number of pixels omitted.
The mode may be obtained in different ways (see Parameter METHOD).
"Data"
, "Error"
,
"Quality"
or "Variance"
(note that "Error"
is the alternative to "Variance"
and
causes the square root of the variance values to be taken before computing the
statistics). If "Quality"
is specified, then the quality values are treated
as numerical values (in the range 0 to 255). ["Data"]
[!]
"Histogram"
–- This finds the peak of an optimally binned histogram, the mode being the
central value of that bin. The number of bins may be altered given through Parameter
NUMBIN, however it is recommended to use the optimal binsize derived from the
prescription of Freedman & Diatonis.
"Moments"
–- As "Histogram"
but the mode is the weighted centroid from the moments of
the peak bin and its neighbours. The neighbours are those bins either side of the peak
in a continuous sequence whose membership exceeds the peak value less three times the
Poisson error of the peak bin. Thus it gives an interpolated mode and does reduce the
effect of noise.
"Pearson"
–- This uses the 3 * median - 2 * mean formula devised by Pearson. See the
first two References. This assumes that the median is bracketed by the mode and mean
and only a mildly skew unimodal distribution. This often applies to an image of the
sky.
["Pearson"]
"Moments"
. This must lie in the range 10 to 10000. The suggested default
is calculated dynamically depending on the data spread and number of values
(using the prescription of Freedman & Diaconis). For integer data it is advisble
to use the dynamic default or an integer multiple thereof to avoid creating
non-integer wide bins. []
!
). The
percentiles must be in the range 0.0 to 100.0 [!]
stats.dat
. The mode is derived using the Pearson formula. Where the histogram contains a few extreme outliers, the histogram limits are adjusted to reduce greatly the bias upon the statistics, even if a chosen percentile corresponds to an extreme outlier. The outliers are still accounted in the median and percentiles. The histogram normally uses 10000 bins. For small arrays the number of bins is at most a half of the number of array elements. Integer arrays have a minimum bin width of one; this can also reduce the number of bins. The goal is to avoid most histogram bins being empty artificially, since the sparseness of the histogram is the main criterion for detecting outliers. Outliers can also be removed (flagged) via application THRESH prior to using this application.
There is quantisation bias in the statistics, but for non-pathological distributions this should be insignificant. Accuracy to better than 0.01 of a percentile is normal. Linear interpolation within a bin is used, so the largest errors arise near the median.
Moroney, M.J., 1957, Facts from Figures (Pelican)
Goad, L.E. 1980, Statistical Filtering of Cosmic-Ray Events from Astronomical CCD Images in Applications of Digital Image Processing to Astronomy, SPIE 264, 136.
Freedman, D. & Diaconis, P. 1981, On the histogram as a density estimator: L2 theory, Zeitschrift f"ur Wahrscheinlichkeitstheorie und verwandte Gebiete 57, 453.
This routine correctly processes the AXIS, DATA, VARIANCE, QUALITY, TITLE, and HISTORY components of the NDF.
Processing of bad pixels and automatic quality masking are supported.
All non-complex numeric data types can be handled. Arithmetic is performed using single- or double-precision floating point, as appropriate.
Any number of NDF dimensions is supported.