The problem stems from what happens to questionable data which remains in some form of limbo until its status is deemed to be GOOD or BAD. Setting data to BAD is an issue in itself as such data are usually BAD because of a fault. However, the Legacy nature of JLS means that some data will be deemed to be unacceptable for the surveys and so should not be processed into Advanced Data Products. These data are not intrinsically BAD and so the plan is for them to be immediately released to the public.
We resolve this issue by having a new quality parameter in obslog - REJECT. Definitions:
GOOD: data is good and is processed by the pipeline
QUESTIONABLE: data may have problems with it - a human needs to look and make a decision on its quality
BAD: data is not good, do not process
REJECT: data does not meet agreed standards for survey
We don’t expect to be using the REJECT flag during normal PI observations. Furthermore, it is expected that with working QA in the survey pipelines the number of REJECT and QUESTIONABLEs will be small (but there will be enough, especially at the beginning as we're bedding the system, that we need to deal with them appropriately).
The following table summarises what happens to data with these flags:
_cube _reduced group proprietary
GOOD Y Y Y Y
BAD Y N N N
QUEST. Y Y N Y
REJECT Y Y N N
ADP charged VO/master product
GOOD Y Y Y
BAD N N N
QUEST. N Y N
REJECT N N Y
(N.B: QUESTIONABLE data should not be combined into the public VO product as those data are in an undefined quantum state and until their wave functions have collapsed into either of GOOD, BAD or REJECT you don’t know what to do with them)
No comments:
Post a Comment