qcrad: Data Quality Assessment for ARM Radiation Data

qcradThe QCRAD VAP has been developed to assess the data quality, and to enhance data continuity, for the ARM radiation data collected at all ARM central and extended facilities. The QCRAD methodology uses climatological analyses of the surface radiation measurements to define reasonable limits for testing the data for unusual data values. The main assumption is that the majority of the measurements are “good” data, which for field sites operated with care, such as those of the Atmospheric Radiation Measurement (ARM) Facility, is a reasonable assumption. Data that fall outside the normal range of occurrences are labeled either “indeterminate” or “bad”, depending on how far outside the normal range the particular datum reside. The methodology not only sets fairly standard maximum and minimum value limits, but includes many cross-comparisons based on what we have learned about how these instruments behave in the field in developing other VAPs such as the Diffuse IR Loss Correction VAP (Younkin and Long 2004) and the Best Estimate Flux VAP (Shi and Long 2002).

The QCRAD VAP produces two daily files containing 1-minute radiation measurement fields and their QC values. The *.c1-level file includes auxiliary information and detailed aqc and bit-packed qc flags. The *.s1-level summary file includes a simplified version of the qc flags and fewer auxiliary data values.

During operational processing, the QCRAD VAP applies a generic correction to the shortwave (SW) downwelling hemispheric flux to correct for infrared loss within the radiometer. On a yearly basis, a more detailed correction that uses fits derived for the specific instruments (rather than the generic correction) is applied. This correction will affect the output of the downwelling SW values for those occasions for which the sum of the direct plus diffuse SW is not available as the “best estimate” for downwelling SW. So that the user may easily tell whether this full correction has been applied, files that have had the full correction implemented are labeled as *.c2 and *.s2. Once the *.c2 and *.s2 files are available, they will replace the *.c1 and *.s1 files in the archive.

In September 2011, an end-to-end reprocessing of the QCRad datastreams was undertaken to:

  • Include the newest data from a previous end-to-end reprocessing of the MFRSR datastreams
  • Remove a qc test on the longwave upwelling and downwelling fluxes that was found to be too restrictive and was throwing out good data
  • Correct inconsistencies between the original ‘aqc’ flags and the ARM-standard ‘qc’ flags
  • Correct the coding of the logic for calculating the BestEstimate_down_short_hemisp; this correction only affects some cases where the BestEstimate was calculated from morning or afternoon fit.

The new version of the code has been released and is now being run in operational processing. All of the historical QCRad data at all sites/facilities is being reprocessed, reviewed, and sent to the Archive. The status of the reprocessing effort is available here.

Additional Information on the QCRAD VAP

ARM Technical Report: The QCRad Value-Added Product: Surface Radiation Measurement Quality Control Testing, Including Climatologically Configurable Limits. Atmospheric Radiation Measurement Program Technical Report, ARM TR-074, 69 pp. Long, CN, and Y. Shi. 2006. Available via https://www.arm.gov/publications/tech_reports/doe-sc-arm-tr-074.pdf.

Long, CN, and Y Shi. 2008. “An automated quality assessment and control algorithm for surface radiation measurements.” TOASJ 2: 23-37, doi:10.2174/1874282300802010023.

Shi, Y, and CN Long. 2006. Surface Radiation Measurement Data Quality Assessment at the ARM TWP and NSA Sites. 16th ARM Science Team Meeting Proceedings, Albuquerque, New Mexico, March 27-31, 2006. Available online at http://www.arm.gov/publications/proceedings/conf16/extended_abs/shi_y.pdf.

Shi, Y, and CN Long. 2005. Examples of Detecting Measurement Errors with the QCRad VAP. 15th ARM Science Team Meeting Proceedings, Daytona Beach, Florida, March 14-18, 2005. Available online at http://www.arm.gov/publications/proceedings/conf15/extended_abs/shi_y.pdf.

Shi, Y, and CN Long. 2004. Techniques and Methods Used to Determine the Best Estimate of Total Downwelling Shortwave Radiation. 14th ARM Science Team Meeting Proceedings, Albuquerque, New Mexico, March 22-26, 2004. Available online at http://www.arm.gov/publications/proceedings/conf14/extended_abs/shi-y.pdf.

Younkin, K, and CN Long. 2004. Improved Correction of IR Loss in Diffuse Shortwave Measurements: An ARM Value Added Product. Atmospheric Radiation Measurement Program Technical Report, ARM TR-009, Available online at http://www.arm.gov/publications/tech_reports/arm-tr-009.pdf.

Shi, Y, and CN Long. 2003. Preliminary Analysis of Surface Radiation Measurement Data Quality at the SGP Extended Facilities. 13th ARM Science Team Meeting Proceedings, Broomfield, Colorado, March 31-April 4, 2003. Available online at http://www.arm.gov/publications/proceedings/conf13/extended_abs/shi-y.pdf.

Shi, Y, and CN Long. 2002. Best Estimate Radiation Flux Value-Added Product: Algorithm Operational Details and Explanations, Atmospheric Radiation Measurement Program Technical Report. ARM TR-008, Available online at http://www.arm.gov/publications/tech_reports/arm-tr-008.pdf.

Locations

  • Fixed
  • AMF1
  • AMF2
  • AMF3

Components

gndrad: Ground Radiometers on Stand for Upwelling Radiation

mfrsr: Multifilter Rotating Shadowband Radiometer

siros: Solar and Infrared Radiation Observation Station Instruments

sirs: Solar and Infrared Radiation Station

skyrad: Sky Radiometers on Stand for Downwelling Radiation

Related Publications

2018

Zhang C, S Xie, S Klein, H Ma, S Tang, K Van Weverberg, C Morcrette, and J Petch. 2018. "CAUSES: Diagnosis of the Summertime Warm Bias in CMIP5 Climate Models at the ARM Southern Great Plains Site." Journal of Geophysical Research: Atmospheres, 123(6), doi:10.1002/2017JD027200.

2015

Van Weverberg K, CJ Morcrette, H Ma, SA Klein, and JC Petch. 2015. "Using regime analysis to identify the contribution of clouds to surface temperature errors in weather and climate models." Quarterly Journal of the Royal Meteorological Society, 141(693), 10.1002/qj.2603.

Shupe MD, DD Turner, A Zwink, MM Thieman, EJ Mlawer, and T Shippert. 2015. "Deriving Arctic Cloud Microphysics at Barrow, Alaska: Algorithms, Results, and Radiative Closure." Journal of Applied Meteorology and Climatology, 54(7), 10.1175/jamc-d-15-0054.1.

2014

Reda I, J Grobner, and S Wacker. 2014. Results of second outdoor comparison between Absolute Cavity Pyrgeometer (ACP) and Infrared Integrating Sphere (IRIS) Radiometer at PMOD. Presented at 2014th Atmospheric System Research (ASR) Science Team Meeting. Potomac, MD.

2013

Yoneyama K, C Zhang, and CN Long. 2013. "Tracking Pulses of the Madden–Julian Oscillation." Bulletin of the American Meteorological Society, 94(12), 10.1175/bams-d-12-00157.1.

McFarlane SA, CN Long, and J Flaherty. 2013. "A Climatology of Surface Cloud Radiative Effects at the ARM Tropical Western Pacific Sites." Journal of Applied Meteorology and Climatology, 52(4), 10.1175/jamc-d-12-0189.1.

Mather JH and JW Voyles. 2013. "The Arm Climate Research Facility: A Review of Structure and Capabilities." Bulletin of the American Meteorological Society, 94(3), 10.1175/bams-d-11-00218.1.

2012

May PT, CN Long, and A Protat. 2012. "The diurnal cycle of the boundary layer, convection, clouds, and surface radiation in a coastal monsoon environment (Darwin, Australia)." Journal of Climate, 25(15), 10.1175/jcli-d-11-00538.1.

Long CN and SA McFarlane. 2012. "Quantification of the impact of Nauru Island on ARM measurements." Journal of Applied Meteorology and Climatology, 51(3), 10.1175/jamc-d-11-0174.1.

Reda I, J Zeng, J Scheuch, L Hanssen, B Wilthan, D Myers, and T Stoffel. 2012. "An absolute cavity pyrgeometer to measure the absolute outdoor longwave irradiance with traceability to international system of units, SI." Journal of Atmospheric and Solar-Terrestrial Physics, 77(March 2012), 10.1016/j.jastp.2011.12.011.


View All Related Publications