Fixing instabilities in the WVM

The JCMT's Water Vapour Monitor (WVM) was taken off the telescope between the 20130326 and 20130408 to improve instabilities with its performance. The WVM is now back on the telescope as of 20130409 and initial results are looking good.

Tau estimates during this time have been taken from the CSO's WVM. The CSO values are used by the map maker automatically when reducing SCUBA-2 data. If you have any concerns or questions regarding data collected during this time please contact your Friend of Project. Your Friend of Project can be identified using the following link:

http://www.jach.hawaii.edu/JCMT/allocations/

New OMP features for projects.

The OMP has recently been updated to display several additional pieces of information on the project pages.

Tau graph

The first thing you will notice is a new graph of tau over the course of the night. Previously, the pages had such a plot that was generated each time the page was loaded. This took a while, and the plot had automatically generated limits on the y-axis which made comparing graphs between two night effectively impossible.

The new graph is made with consistent x- and y-limits that allows for easy comparison between nights. The various weather grades are also delineated, and the value of tau from the CSO WVM is plotted as well.

The new plot of tau vs. time. Click an any picture for a larger view.

The thicker gray horizontal lines running across the graph mark the weather grade boundaries, and the shaded area shows the night hours in Hawaii. The time in UTC is plotted along the bottom axis, with the time in HST along the top.

Pie chart

The pie chart of time spent in each grade.

There is also a pie chart that sums up the amount of time spent in each weather grade throughout the night. The grades are color-coded from green to red to give a quick visual assessment of how good a given night was (green good; red bad). Usually a given night will fall predominately within just one or two grades.


ACSIS standards table

For nights on which data was taken using ACSIS, there will be an HTML table in text form like the example below with some information relating to the ACSIS standards observed.

Obs #	Time	Integ. Int.	Peak Int.
16	19:36:32	251.33	8.82
19	19:50:31	4775.56	6.88
27	20:59:30	4157.76	5.96
36	22:22:36	3520.79	5.08

SCUBA-2 calibrations table

Similarly, for nights where SCUBA-2 was used there will be an HTML table like the example below listing the FCFs for each observation of a calibration object.

20120822			FCFasec				FCFpeak
Obs #	Time (UT)	Source	850µm	err	450µm	err	850µm	err	450µm	err
8	05:41:27	CRL2688	2.41	0.01	4.64	0.03	572.2	1.5	542.9	5.7
37	10:24:23	CRL2688	2.30	0.01	4.66	0.02	513.5	1.2	466.0	3.7
68	17:26:48	CRL618	2.23	0.01	4.28	0.04	487.8	1.4	436.8	4.8

Along with the table on SCUBA-2 nights there will three additional graphs, which will be detailed below. Each of these graphs has two sub-plots, the top one being the 450 micron data and the bottom one being the 850 micron.

NEPs vs observation number graph

This graph shows the min, max, and mean for each of the eight subarrays that make up SCUBA-2, plotted by observation number. The mean is the colored line, and the shaded areas are the filled-in areas between the min and max. By following the lines you can tell which arrays changed significantly, and by watching the shaded areas you can get a feel for the spread in the NEPs. The vertical scale is fixed, and is the same as the scale for the following plot.

NEPs vs time graph

This graph, like the previous one, shows the NEPs, but plots them against time rather than observation number. Each of the eight subarrays is present once again (with the same colors as in the preceding plot). The time (in UTC) is marked along the bottom of the plot, and the observations are marked by number along the top, with vertical lines that descend to help mark when each particular observation began.

NEFDs vs time graph

The final plot is a plot of the NEFDs vs. time. The number of points depends on the night, this particular night only had a few. Like the two previous plots, the vertical axis is fixed to make comparisons between nights easier.

Hopefully these new features will prove useful to users of the OMP, and additional updates or improvements may be forthcoming in the future. Feedback on the new features is welcome.

A new feature in CONFIGMELD

The CONFIGMELD tool in SMURF can be used to compare two different sets of MAKEMAP configuration parameters. But sometimes you are only interested in a single parameter value, and wading through the entire configuration is bit off-putting. So now you can specify a single parameter name when running CONFIGMELD, and it will just display the value of that one parameter as plain text in your terminal window (that is, no visual comparison tool is used). Also, if you are only interested in the parameter value in a single config, then you can just leave CONFIG2 unspecified.


% configmeld ^/star/share/smurf/dimmconfig_bright_extended.lis  param=flagslow
   WAVEBAND - The waveband to display - 450 or 850 /850/ > 
   Showing 850 um values

   flagslow = 30

% configmeld map1 map2 param=flt.filt_edge_largescale

  Showing 450 um values

  flt.filt_edge_largescale = 600 (config1)

  flt.filt_edge_largescale = 400 (config2)

As always, to use this feature you will need to update your starlink to the current development version. Linux users can simple rsync Starlink from JAC following the instructions at http://starlink.jach.hawaii.edu/starlink/rsyncStarlink

Straight lines in ZERO_SNRLO masks

Today I've fixed a bug that affected masks created using the "xxx.ZERO_SNRLO" configuration parameter ("xxx" = AST, FLT or COM). The effect of this bug was to reduce the size of the source area in the mask by "slicing" sections off the mask. Very often (but not always), this slicing would leave an unnaturally straight edge in the final mask.  As an example, here are two masked maps of G34 - the first map is affected by the bug, and the second is not. To get the bug fix, you will need to rsync Starlink from JACH.

Note, my initial fix for this bug was itself buggy. So if you rsynced prior to 11:20 GMT on  15th March, you will have only a partial fix for this problem, and you should rsync again.

In addition, the amount of smoothing used to create ZERO_SNRLO masks has been reduced slightly, so expect masks that are little more "crinkly" round the edges than before.

Getting Pipelines and Archives info by email

You should now be able to see a "Subscribe via email" link at the top right of the "Pipelines and Archives" blog page. Click on this and follow the instructions if you want to get email alerts of any new blog posts - you'll only get at most one email per day (and no emails at all on days when no posts are made).

Checking SCUBA-2 calibrator data

A new PICARD recipe has been added called SCUBA2_CHECK_CAL which can be used to assess the quality of the calibration by calculating flux conversion factors (FCFs) and estimating the beam size to compare with standard values.

It can be run on any processed observations of standard sources (except, obviously, focus data). The input files must be uncalibrated, either the output from running makemap by hand or from the pipeline and then processed with the PICARD recipe UNCALIBRATE_SCUBA2_DATA.

Run it as any other PICARD recipe:

% picard -log sf -recpars myparams.ini SCUBA2_CHECK_CAL calfiles*.sdf

The recipe trims the image, optionally removing a background before it fits the source to estimate the beam.

The next step is calculating the FCFs which are reported to the screen. The map is then calibrated using either a standard FCF or one of those just calculated (controlled by a recipe parameter). The noise in the map is calculated from this calibrated image.

The recipe writes a log file called log.checkcal which contains various parameters. The values of interest include the total flux and uncertainty derived from the uncalibrated (input) map, the noise in the calibrated version of that map, the FCFs and associated uncertainties and the beam FWHM. In the future there will also be an estimate of the error beam. The log file can be read into Topcat so values can be plotted as a function of time, elevation or any other parameter (if there are enough of them).

The total flux is measured using aperture photometry with an aperture radius of 30 arcsec. The aperture size may be adjusted using a recipe parameter, but be aware that comparison with the standard ARCSEC FCF will require correcting for the different aperture. See the SCUBA-2 calibration paper for further details.

By default the recipe can only estimate FCFs from known calibrators. However, it also be used to estimate the FCF from non-standard sources if the flux is known to the user. Recipe parameters exist to allow this to be specified (on a per-source basis if desired). Note that the sources should still be unresolved, point sources.

The FCF calculations can be compared with the standard values (given in the calibration paper) to determine whether or not there may be particular issues with the calibration on a given night. The BEAM FCF is more sensitive to focus than the ARCSEC value, so if the former is out of spec, while the latter is not, it could indicate that the telescope was not as well focussed as it could have been. If both are far out of spec (indicated by red text in the recipe output) then it may suggest that a non-standard FCF is in order. It is also important to look at the trend in the FCFs over the night, and not just a single value.

The main recipe parameters of interest are:

• APERTURE_RADIUS - radius of photometry aperture in arcsec. Default is 30.
• USEFCF - a flag to denote whether the derived FCF should be used to calibrate the data. Default is 0 (use the standard FCF).
• FLUX_850 - total flux of a point source at 850 um. The corresponding 450 um parameter is FLUX_450. Fluxes may be specified for multiple sources by adding the source name (upper case with spaces removed) to the parameter - e.g., FLUX_850.MYSOURCE1, FLUX_850.MYSOURCE2 etc.
• REMOVE_BACKGROUND - a flag to denote whether or not a background should be removed. Default is 0 (do not remove a background). If true, then there are other parameters available to control the background fitting and removal process.

The full list of recipe parameters is described in SUN/265 and will be available in the upcoming Starlink release.

Comparing MAKEMAP configurations

You have two SCUBA-2 maps, created by makemap from the same observation at different times, and they look different. Not unnaturally you want to know why. The first thing you may suspect is that they were made with different configuration parameter values. You can do this by using the hislist command in kappa to look at the history information in each map, but it's a little painful. The good news is that the new configmeld command in smurf now makes this a lot easier. It will read the configuration parameter values from the History information of the two maps and use a visual comparison tool such as "meld" to display them side-by-side, sorted alphabetically, and with any differences highlighted. For instance

% configmeld map1.sdf map2.sdf

may produce a display similar to the one below...

Instead of reading the configurations to be displayed from the history of an NDF, either (or both) configuration may instead be specified "as normal" - that is, as you would do when running makemap, so if you have a set of configuration parameter values in text file myconfig.lis, you could do something like:


% configmeld ^myconfig.lis map2.sdf

to compare the configuration specified by myconfig.lis with the configuration used to create the map in map2.sdf. As for all Starlink commands, you can use the findme command to see the full list of options provided by configmeld. Type:

% findme configmeld

which should open a web page, and click on the link for configmeld.


Notes:

1. If your system does not have the meld command, the first available command in the following list will be used instead: opendiff, diffmerge, kdiff3, tkdiff, diffuse.
2. Configuration values that have "no value" by default (usually shown as "") are not included in the history information stored in the map created by makemap, and so will not appear in the list shown by configmeld. The sharp ones among you may therefore notice a small lie above - the screenshot shown earlier was actually made by comparing a "normal" config file with the configuration from an NDF, not from two NDFs as suggested by the example command. Sorry! That's why one side (the normal config file) has "" values and the other side (read from an NDF) does not. 
3. configmeld is in fact a Python script designed to run with python 2.7 or later. If you have problems using it, it may be because your Python installation is too old.
4. The configmeld script just uses the configecho command from KAPPA to list the two sets of configuration parameters to two separate temporary text file and then uses meld (or your selected tool) to display the two files side-by-side.