The QCRAD VAP has been developed to assess the data quality, and to enhance data continuity, for the ARM radiation data collected at all ARM central and extended facilities. The QCRAD methodology uses climatological analyses of the surface radiation measurements to define reasonable limits for testing the data for unusual data values. The main assumption is that the majority of the measurements are “good” data, which for field sites operated with care, such as those of the ARM facility, is a reasonable assumption. Data that fall outside the normal range of occurrences are labeled either “indeterminate” or “bad”, depending on how far outside the normal range the particular data reside. The methodology not only sets fairly standard maximum and minimum value limits, but includes many cross-comparisons based on what we have learned about how these instruments behave in the field in developing other VAPs such as the Diffuse IR Loss Correction VAP (Younkin and Long 2004) and the Best Estimate Flux VAP (Shi and Long 2002).
The QCRAD VAP produces two daily files containing 1-minute radiation measurement fields and their QC values. The *.c1-level file includes auxiliary information and detailed aqc and bit-packed qc flags. The *.s1-level summary file includes a simplified version of the qc flags and fewer auxiliary data values.
During operational processing, the QCRAD VAP applies a generic correction to the shortwave (SW) downwelling hemispheric flux to correct for infrared loss within the radiometer. On a yearly basis, a more detailed correction that uses fits derived for the specific instruments (rather than the generic correction) is applied. This correction will affect the output of the downwelling SW values for those occasions for which the sum of the direct plus diffuse SW is not available as the “best estimate” for downwelling SW. So that the user may easily tell whether this full correction has been applied, files that have had the full correction implemented are labeled as *.c2 and *.s2. Once the *.c2 and *.s2 files are available, they will replace the *.c1 and *.s1 files in the ARM Data Center.
In September 2011, an end-to-end reprocessing of the QCRad datastreams was undertaken to:
- Include the newest data from a previous end-to-end reprocessing of the MFRSR datastreams
- Remove a qc test on the longwave upwelling and downwelling fluxes that was found to be too restrictive and was throwing out good data
- Correct inconsistencies between the original ‘aqc’ flags and the ARM-standard ‘qc’ flags
- Correct the coding of the logic for calculating the BestEstimate_down_short_hemisp; this correction only affects some cases where the BestEstimate was calculated from morning or afternoon fit.
The new version of the code is now being run in operational processing. All of the historical QCRad data at all sites/facilities is being reprocessed, reviewed, and sent to the Data Center. The status of the reprocessing effort is available here.
Additional Information on the QCRAD VAP
ARM Technical Report: The QCRad Value-Added Product: Surface Radiation Measurement Quality Control Testing, Including Climatologically Configurable Limits. Atmospheric Radiation Measurement Technical Report, ARM TR-074, 69 pp. Long, CN, and Y. Shi. 2006. Available via https://www.arm.gov/publications/tech_reports/doe-sc-arm-tr-074.pdf.
Long, CN, and Y Shi. 2008. “An automated quality assessment and control algorithm for surface radiation measurements.” TOASJ 2: 23-37, doi:10.2174/1874282300802010023.
Shi, Y, and CN Long. 2006. Surface Radiation Measurement Data Quality Assessment at the ARM TWP and NSA Sites. 16th ARM Science Team Meeting Proceedings, Albuquerque, New Mexico, March 27-31, 2006. Available online at http://www.arm.gov/publications/proceedings/conf16/extended_abs/shi_y.pdf.
Shi, Y, and CN Long. 2005. Examples of Detecting Measurement Errors with the QCRad VAP. 15th ARM Science Team Meeting Proceedings, Daytona Beach, Florida, March 14-18, 2005. Available online at http://www.arm.gov/publications/proceedings/conf15/extended_abs/shi_y.pdf.
Shi, Y, and CN Long. 2004. Techniques and Methods Used to Determine the Best Estimate of Total Downwelling Shortwave Radiation. 14th ARM Science Team Meeting Proceedings, Albuquerque, New Mexico, March 22-26, 2004. Available online at http://www.arm.gov/publications/proceedings/conf14/extended_abs/shi-y.pdf.
Younkin, K, and CN Long. 2004. Improved Correction of IR Loss in Diffuse Shortwave Measurements: An ARM Value Added Product. Atmospheric Radiation Measurement Program Technical Report, ARM TR-009, Available online at http://www.arm.gov/publications/tech_reports/arm-tr-009.pdf.
Shi, Y, and CN Long. 2003. Preliminary Analysis of Surface Radiation Measurement Data Quality at the SGP Extended Facilities. 13th ARM Science Team Meeting Proceedings, Broomfield, Colorado, March 31-April 4, 2003. Available online at http://www.arm.gov/publications/proceedings/conf13/extended_abs/shi-y.pdf.
Shi, Y, and CN Long. 2002. Best Estimate Radiation Flux Value-Added Product: Algorithm Operational Details and Explanations, Atmospheric Radiation Measurement Program Technical Report. ARM TR-008, Available online at http://www.arm.gov/publications/tech_reports/arm-tr-008.pdf.