Large scale structure divergence

First, my apologies for the lousy image layout! Next post will be better...

As many of you have noticed, large-scale noise structures in the maps appear to grow with the number of iterations (clearly not the desired behaviour!). This blog entry describes the problem in more detail with an example, and discusses a partial fix.

The map on the left was produced from about 6 min. of scans across a blank-field at 450um using the dimmonfig_faint.lis configuration file, but turning up the number of iterations to 100. As you can see there is a strong gradient across the map -- the more iterations I added, the more the gradient grew.

In the next plot I show the common-mode signal from this map solution (the average signal in all the detectors as a function of time). Mostly we see the fridge temperature oscillations (about every 30s), plus a more slowly varying component (maybe sky?)

Next, I set ast.zero_lowhits = 0.5 in the config file. What this option does is identify all of the pixels in the output map with less than 50% of the mean number of hits, and then sets those pixels to zero after each iteration (i.e. a zero boundary condition). In the next two plots I show the map, and the difference between the common-mode estimated from the two solutions:








What we can can see is that the gradient has been drastically reduced. So what's happening? The common-mode difference plot shows that the map without edge constraints has a large additional periodic signal (about 10% of the main common mode signal) with a frequency that is a factor of a few larger than the 30s fridge oscillations. In fact, this signal is nicely correlated with the higher-frequency scan pattern.

Basically this shows that, left to its own devices, the solution is perfectly happy to pump large-scale structure into the map. It then still manages a flat residual because it simply compensates for this gradient by putting negative signal of the correct amplitude into the common-mode. In other words, the largest scales and the common-mode are degenerate model parameters.

However, I have noticed that even with this edge constraint turned on, there is other strange structure in the map, like this convex curvature. This fact, combined with the substantially worse noise at 850um, shows us that the model is insufficient for accurately describing the data. The edge constraint helps, but that curvature indicates some kind of tension (like the model trying to fit some sort of gradient across the array and failing). I suspect that, lacking any other conditions on the model, the large-scale noise is just growing without bound.

We are presently looking at ways to improve the model for the data and/or introduce other reasonable constraints to help convergence to something sensible! For now, it is worth turning ast.zero_lowhits on if you don't believe you have any strong emission features around the edge of the map (tricky for maps of structure in the galactic plane).