Improving earth system models is at the heart of a pair of new studies
Hot streaks? Not only athletes have them. Institutions do too.
A pair of recent studies published as early online releases by the Bulletin of the American Meteorological Society (BAMS) adds to a hot streak of papers in the journal. At least five BAMS studies so far in 2020 showcase data from the U.S. Department of Energy’s Atmospheric Radiation Measurement (ARM) user facility.
Since 1992, ARM’s fixed and mobile atmospheric observatories have collected terabytes of long-term, continuous data on the atmospheric processes that create weather and influence earth systems. These include clouds, precipitation, turbulent fluxes, radiation, aerosols, and land-surface influences. These data, quality-checked and archived, are widely used to calibrate, develop, and improve numerical weather prediction and climate models.
ARM data are at the core of one June 2020 BAMS study on optimizing retrieval algorithms. The lead author is remote sensing and retrieval expert Maximilian Maahn. On July 1, 2020, he moved to Leipzig University in Germany from the Cooperative Institute for Research in Environmental Sciences (CIRES) at the University of Colorado, Boulder.
Data from ARM are also the key to a July 2020 BAMS paper on metrics and diagnostics for improving climate model evaluations. The lead author is Jill Chengzhu Zhang, a staff scientist at Lawrence Livermore National Laboratory in California.
Optimal Estimation Retrievals
Maahn joined co-authors in California, Washington state, Utah, Colorado, and his native Germany to write the June 2020 retrieval-algorithms paper.
It begins with what every atmospheric scientist knows: that remote sensing instruments (such as cloud radars and many other instruments operated by ARM) do not provide direct observations of variables within Earth’s atmosphere.
“We are not actually observing what we want to know,” state the authors.
Instead, the instruments take indirect measurements, such as shifts in voltage or changes in backscatter intensity. Unable to be measured directly from the ground are the specific atmospheric variables of interest: the ice water content at a specific altitude, for instance, or the number of cloud droplets.
That means algorithms are needed to use the indirect measurements to estimate more specific information about an atmospheric variable, a process called “retrieval.” Such algorithms are informed by prior known information, such as a previous data set. Using a classic 1994 source to illustrate a retrieval, the authors compare the process to describing a dragon by observing only its footprints in the sand but backed up by what was previously known about dragons.
The authors present a series of examples, using the Optimal Estimation (OE) method of retrieval.
“We guide the reader through the development of such a retrieval, highlighting common pitfalls and challenges,” says Maahn. “The reader can play with it and explore it on his or her own.”
OE code uses an iterative process to search for an optimal solution based on an observation and prior information. The retrieval process is complicated by data “noise” and other factors, the authors caution, and prompts uncertainties in models. (The authors go into detail.)
Maahn describes the paper as “educational,” because the code is already available online.
To help readers experiment with their own OE retrieval projects, Maahn and his co-authors include links to supplemental Jupyter Notebooks, along with a pyOptimalEstimation Python library. The Jupyter Notebooks provide two examples of OE retrievals implemented on ARM instruments that can be run in a web browser; the pyOptimalEstimation library provides code implementing the OE methods.
ARM-Oriented Metrics and Diagnostics
“Combining ARM’s high-frequency ground measurements with a tool like ARM-DIAGS provides a complementary test for evaluating models.”
In their July 2020 paper, Zhang and her co-authors explicitly acknowledge ARM’s “comprehensive data sets” and their wide use in developing and improving earth system models.
From there, the study introduces a metrics and diagnostics package called ARM-DIAGS, which was developed for enabling the use of ARM data in earth system model evaluations. The package is housed on the GitHub development platform, and its analysis codes are publicly available. ARM-DIAGS includes ARM data sets, compiled from multiple ARM data products, as well as a Python-based analysis toolkit for computation and visualization.
ARM-DIAGS also includes simulation data from models in the Coupled Model Intercomparison Project (CMIP), which makes it easier to compare new models with those already in CMIP.
Earth system model developers use a set of standard metrics and diagnostics as a way of routinely assessing model performance and judging the performance of new parameterizations.
Now ARM data can be used more easily to address a range of issues in models, including diagnosing summertime warm bias (the tendency of models to overestimate surface temperatures), the metrics of convective onset, precipitation distribution, and the diurnal cycles of both cloud fraction and precipitation.
Historically, climate model developers have used satellite remote sensing to calibrate and tune models. There are also many examples of modelers using ARM data to assess models, but generally these data have not been part of routine model development workflows.
“Given the growing interest in improving parameterizations with process-oriented metrics and diagnostics, ARM observations should play a more important role,” says Zhang. “Combining ARM’s high-frequency ground measurements with a tool like ARM-DIAGS provides a complementary test for evaluating models.”
For now, ARM-DIAGS is based on data from ARM’s Southern Great Plains and North Slope of Alaska observatories, as well as from historical data from three observatories within the now-closed Tropical Western Pacific ocean site.
The hope, Zhang and her co-authors write, is “that (ARM-DIAGS) can serve as an easy entry point for climate modelers to compare their models with ARM data and supplemented CMIP data sets.”
ARM is a DOE Office of Science user facility operated by nine DOE national laboratories.