Standards for ARM Data Assure Consistency, Sustainability

 
Published: 2 May 2014
Establishing standards for ARM data assures consistency and sustainability in the ever-increasing data sets available to the scientific community.

As the ARM Facility has enhanced its in situ and remote-sensing observation capabilities over the years, its data systems are handling massive amounts of data—up to 30 terabytes per month! To sustain and enable consistent use and analysis of the multiyear data, the newly published ARM Data Standards Version 1.0 details the required and recommended standards for new data to be published in the ARM Data Archive.

Developed by the ARM Standards Committee, the new standards will benefit users with consistency across datastreams, as well as simple and consistent software that can read all standardized netCDF files, and will allow for data code reuse. These standards also allow for development of automated monitoring and data health status tools, and will aid in the development of future capabilities for delivering data.

“Leading the effort to create standards we can all work with was the hardest thing I’ve done working for the ARM program. But I also feel it is one of my greatest accomplishments. This group knocked it out of the park,” said Ken Kehoe, a scientist for the University of Oklahoma and assistant manager of the ARM Data Quality Office. “Standardizing our data files opens so many doors to new exciting possibilities. I can’t wait to see what amazing things this will lead to,” he added.

A separate committee will review any exceptions to these standards prior to acceptance and publication of the data, and all archived ARM data will be reprocessed to comply with the new standards.

This is the first report of its kind for ARM data and plans are to update the report as standards evolve. The ARM Standards Committee was appointed by the ARM Technical Director and is comprised of members from multiple organizations, including private corporations and national labs.

Read the report online, or request a hard copy version through pubs@arm.gov.