Data Gathering and Delivery
Collecting the Data
The ARM Climate Research Facility instruments automatically collect data on surface and atmospheric properties, and the information is sent to the site data systems. In addition to the instruments, ARM data is also gathered through aircraft, forecast models, satellites, field campaigns, and value-added processing.
Scientists will often use aircraft to gather airborne data during field campaigns at an ARM Climate Research Facility (ARM) site. The ARM Aerial Facility (AAF) provides airborne measurements required to answer science questions proposed by the ARM Science Team and the external research community and is an integral measurement capability of ARM. Aircraft choice is dictated by science requirements—such as the required measurements and desired flight profile—and aircraft availability. Data obtained from the aircraft are documented, checked for quality, integrated into the ARM Archive, and made available in a timely and consistent manner for use by the scientific community.
The External Data Center regularly collects analysis and forecast model products produced by the National Centers for Environmental Prediction (NCEP) and the European Center for Medium-Range Weather Forecasting (ECMWF). These products are used to augment the instrument data collected at the ARM sites. Forecast models are available for each specific site on the ARM External Datastream Descriptions page.
High-quality satellite observations are needed to measure the top-of-the-atmosphere radiation. These data augment the data being generated within the Program and are usually converted from native format to either netCDF or HDF formats. Again, the External Data Center gathers these kinds of external data sets and sends them to the Data Archive. The types of satellite data are different at each site and are available on the ARM External Datastream Descriptions page.
Field Campaigns, aka Intensive Operational Periods
ARM sites frequently hold field campaigns at the request of the scientific community to augment routine data acquisition. Typically, a scientist will request to bring their instruments to an ARM locale for the purpose of calibrating and/or testing the instrument or to run an instrument. At the other end of the scale, so is a major field experiment that might include ships or aircraft activities at or near a research site, requiring extensive planning of a year or more. The data from these field campaign efforts are then sent to the IOP Archive for permanent storage. For more information regarding submitting data, see Steps for Sending IOP Data to the External Data Center. To learn more about the process, please refer to the IOP Request, Approval and Implementation Document and Flowchart (pdf).
Many of the scientific needs of the ARM Climate Research Facility are met through the analysis and processing of existing data products into "value-added" products or VAPs. A VAP creates a "second generation" data stream by using existing ARM data streams as inputs and applying algorithms or models to them. A VAP is run continuously in the Data Management Facility (DMF) and the output generated is treated as a new ARM data stream.
Delivery Through Data Systems
ARM data systems are composed of three primary groups developed to collect, process, and transfer instrument data for scientific analyses. The ARM data systems collect, analyze, process, and transfer data streams of known and reasonable quality to the Data Archive for long-term storage and delivery to users. The files are generally available from the Archive within 48 hours. These data are collected from ARM field measurement sites, the Data Management Facility (DMF), or the External Data Center. Two of the ARM data system sites (Southern Great Plains and North Slope of Alaska) are linked by high-speed communications directly to the Data Archive at Oak Ridge National Laboratory. Data gathered by the field measurement sites in the Tropical Western Pacific are sent by tape to the DMF via mail. In addition, the External Data Center, located at Brookhaven National Laboratory, acquires additional data from other sources, such as National Weather Service satellite and surface data. Most of the ARM data are distributed in netCDF format.