Map-based Despiking
First, a new method for despiking the data has been added. Instead of searching throughout the time series for individual samples that are outliers compared to their neighbours (which can accidentally flag large astronomical sources as spikes in some cases), this new method looks at the weighted scatter of samples that land in each map pixel, and removed outliers. This feature is now turned on by default in dimmconfig.lis:
ast.mapspike = 10
which flags all samples that are > 10 standard deviations away from the other samples that contribute to the flux in the same map pixel, after each iteration. In fact, it works so well that we are using it instead of the time-domain de-spiking in most cases.
We first reduce the Orion data using the default dimmconfig_bright_extended.lis,and the ERROR component of the map looks like this:
we then re-run it, but turning off the default map-based despiking, and turning on time-based despiking:
ast.mapspike = 0
spikethresh = 10
spikebox = 50
and the ERROR component of the map now looks like this:
This latter image clearly shows the locations of large spikes that made it through the time-based despiking. Also note that their locations are not correlated particularly with the brightest regions of emission that are obvious at the lower-right (the large scatter in the ERROR component is a reflection of other systematic errors, such as in the flatfield, that become more prominent in regions of strong astronomical signals).
Flagmaps and Shortmaps
We have added two new types of map extensions to help identify problems while map-making. "flagmaps" are images that count the number of samples with QUALITY bits that match a given quality mask, as a function of their position on the sky. This is useful to see, for example, whether DC step correction and/or de-spiking is affected by bright sources. For example, setting
flagmap = (BADBOL,SPIKE,DCJUMP)
will produce a set of .more.smurf.flagmaps extensions corresponding to each continuous chunk of data indicating where all spikes and DC jumps were located. This is the resulting flagmap for Orion:
The "U" shaped patterns primarily indicate the locations of DC steps that were correlated across the array, and a number of the other spots are simply locations of spikes identified by the map-based despiker. To gain further insight, maps can be re-run, setting flagmap to a single quality bit each time.
There is also a modified form of the "shortmaps" extensions. Instead of specifying the amount of time for each map explicitly, you can now set
shortmap = -1
which will produce a new .more.smurf.shortmaps extension each time the telescope completes a full, cross-linked pass through the scan pattern. In our Orion example, we show the full map, followed by the four sub-maps (.more.smurf.shortmaps.ch00sh00000 through .more.smurf.shortmaps.ch00sh00003) clearly showing how the position angle was rotated for each fully-crosslinked pass across the field:
Choosing frequency-domain filters based on angular scale
Finally, we have implemented an alternative method for specifying frequency edges. In the past, the frequency edge would be given explicitly, e.g.
filt_edgehigh = 0.3
flt.filt_edgehigh = 0.3
to remove all power in the bolometer time-series below 0.3 Hz using the pre-processing and iterative filters respectively (this is the approximate 1/f knee). However, in order to evaluate the impact of such filters on real astronomical structure, one would have to divide the scan rate by this number to convert it to an angular scale. The requested scan rate is recorded in the "SCAN_VEL" FITS header, which for this Orion map was 250 arcsec/sec. Therefore a high-pass filter explicitly removes any information in the map on scales > (250/0.3) = 833 arcsec.
This calculation is cumbersome, and also relies on the requested "SCAN_VEL" being correct. We have therefore added an alternative method for specifying filters based on angular scale, e.g.
filt_edge_largescale = 600.
450.filt_edge_smallscale = 2.
850.filt_edge_smallscale = 4.
In this case a high-pass filter will be selected such as to suppress angular scales larger that 600 arcsec, and low-pass filtered will be used to remove scales smaller than roughly the FWHM/4 of the PSF in each band. The calculation is based on the actual mean slew velocity is measured from the pointing data rather than relying on "SCAN_VEL". For example, in the above Orion maps, the default iterative filter is set to
850.flt.filt_edge_largescale=300
and if the map-maker is run with msg_filter=verbose, when the FLT model is evaluated you will see
flt
smf_scale2freq: Based on a slew speed of 243.1 arcsec/sec, setting:
smf_scale2freq: FILT_EDGEHIGH = 0.810 Hz (less than 300.0 arcsec scales)
Even though the 1/f noise varies from observation to observation (and is at a higher frequency at 850um for SRO data), it may be useful to specify the same angular scale irrespective of scan rate (such as ensuring that the attenuation of the filtering is the same for calibrators as for science data, even if they were scanned at different rates).
Related to all of this, we have replaced the "flagstat" parameter, for flagging data when the telescope was slewing too slowly, with
flagslow = 30
450.flagfast = 600
850.flagfast = 980
These default values include only data where the telescope was slewing at least 30 arcsec/sec, and no more than 600 arcsec/sec at 450um, and 980 arcsec/sec at 850um. The lower limit is set by an approximate 1/f knee of 1 Hz to ensure that at scales of order the 850um PSF will be useful in the data. Similarly, the upper speed limits are chosen to avoid smearing given the PSF sizes in each band and the 200 Hz sample rate.
No comments:
Post a Comment