Editor’s note: ARM Associate Director for Research Jennifer Comstock provided the following blog post.
Among the feedback received from its 2020 Triennial Review, the ARM user facility was encouraged to develop a plan for engaging with the climate modeling community to understand how ARM data are used for model development and evaluation, as well as to develop metrics that illustrate ARM’s impact on the improvement of earth system models.
Since the review, ARM has identified several avenues for addressing these recommendations.
Better Understanding ARM’s Impact on Models
During the 2020 review, ARM presented metrics such as total publications and citations, papers published in high-impact journals, highly cited papers, three-year h-index, and impact factor, a measure of the average impact per publication. In addition to these aggregate metrics, ARM also tracks publication trends across research areas such as clouds, aerosols, and modeling. While these metrics provide insight into ARM’s impact on the atmospheric science community, they do not highlight its impact on model improvement, which is a core part of ARM’s mission.
To help with its metrics, ARM has reviewed the process for tagging publications that are entered into its publications database. The metadata tags now include different modeling identifiers, such as earth system models, single-column models, and cloud-resolving models. New publications are assessed as they are entered using the online form, and assessment of historical publications is underway. The goal of this assessment is to gain a better understanding of how ARM data are used in modeling studies.
Additional metrics may include tracking use of ARM data by the broader user community, including the weather, climate, and satellite communities; and identifying specific science areas in which ARM data have made a significant impact on model improvement.
Supporting Model Development and Evaluation Efforts
ARM staff have begun dedicating additional effort to modeling community outreach to understand how model developers use ARM data and to identify areas in which ARM can further collaborate and expand its resources for model development and evaluation. A natural starting point for this effort is the U.S. Department of Energy’s high-resolution Energy Exascale Earth System Model (E3SM) project.
In April, ARM Modeling Translator Shaocheng Xie, who also leads the development of the lower-resolution E3SM Atmosphere Model (EAM), and I met with E3SM developers Peter Caldwell and Peter Bogenschutz. Caldwell leads the high-resolution Simple Cloud-Resolving E3SM Atmosphere Model (SCREAM) and the next-generation EAMxx, and Bogenschutz is the developer of both the E3SM Single-Column Model (SCM) and the doubly periodic cloud-resolving model DP-SCREAM. Shaocheng and I learned that the E3SM team had just completed a major development effort for SCREAM and DP-SCREAM and was starting to evaluate these models.
During this engaging and productive meeting, the four of us identified several cloud regimes and ARM field campaigns of interest as important for the SCREAM and DP-SCREAM evaluation efforts. Instrument simulators and data uncertainty information are also needed to improve model-ARM data comparison, and developing an ARM case library specifically for DP-SCREAM is highly desired.
ARM will keep working with the E3SM evaluation team to identify specific ARM data sets and time periods of interest to focus future development efforts.
Lastly, ARM continues to engage with its constituent groups, including the User Executive Committee (UEC), which has a subgroup focused on facilitating communication with the modeling community, including E3SM. This summer, Shaocheng and I will meet with the UEC modeling subgroup, led by Susannah Burrows, to learn more about its efforts and look for ways we can collaborate to broaden ARM’s impact.