2014-07-23

Spotting chunking caused by insufficient memory

It has been shown that better maps are created if all available time stream data are processed in a single chunk to create a single map. If the available memory is too small to allow this to happen, the data will be split into multiple chunks and a separate map made from each chunk, which are all co-added at the end to make the final map.

Thus it can be important to spot whether or not your map was "chunked" or not. Until recently the only option was to trawl through the makemap screen output looking for the warning about insufficient memory. Now, however, you also have an option to tell makemap to abort immediately with an informative error message if the data would be chunked due to insufficient memory (no output map is created in this case). To do this, add "memcheck=1" to your config.

Note, this only affects what happens if the data is processed in chunks due to lack of memory - if chunking is caused by the time-series data being intrinsically discontiguous then a map is still created even if memcheck is set to 1.

It will be necessary to rsync the stardev system from Hilo to use this feature. See starlink.jach.hawaii.edu/starlink/rsyncStarlink

No comments: