A joint ARM and ASR workshop turned basic instrument output into more valuable data
While many of their neighbors were thinking of the sun, beaches, and waters of Long Island, a group of Atmospheric System Research scientists and ARM Climate Research Facility developers gathered at Stony Brook University (SBU) from June 23 to July 1 for an intensive workshop. Together, they built new ARM data products using observations from Scanning ARM Cloud Radars (SACRs).
Pavlos Kollias, SBU, and Eugene Clothiaux, Pennsylvania State University (PSU), spearheaded the organization of the workshop, which included scientists, graduate students, postdocs, and developers.
Cleaning Up Code that Cleans Up Radar Data
By the end of the week, the group had developed three new beta-level SACR data products:
- Quality-controlled three-dimensional gridded radar moments and cloud fraction
- Quasi-vertical profiles of polarimetric variables
- Profiles of the horizontal wind in the column over the radar.
Without these new ARM data products, the SACR produces raw data that are not immediately useful to most scientists. For example, the radar scans in a way that provides slices of information, but modeling uses a grid system. Basic radar measurements must also be transformed into atmospheric quantities used in research and modeling. Scientists need to translate the data using algorithms.
In the workshop—or “code sprint”—ARM data developers took algorithms that had been previously created and implemented them to run in a production environment. Instead of running the algorithms themselves, scientists will now be able to pull usable data straight from the ARM Data Center.
“We took algorithms and codes from scientists and integrated them with ARM,” Karen Johnson, an ARM Facility developer at Brookhaven National Laboratory (BNL), said. “The original codes were sometimes not quite bulletproof and written in MATLAB or FORTRAN. We rewrote them in Python and brought them up to ARM standards.”
Working Together Without Distractions
The code developers were joined by the scientists who originally created the algorithms, and ARM Data Integrator staff at Pacific Northwest National Laboratory were available remotely to help with integrating the new programs.
Having all the right people together was important, but so was being able to focus solely on the work at hand. The participants were able to leave their day-to-day distractions behind and really dig in, Johnson said.
Tangible Products to Come Soon
Some work remains to make the ARM data products fully operational, but preliminary results should be available soon. Once up and running, the data products will make available years of radar data that were difficult for non-experts to use.
This code sprint was just the most recent example in ARM’s history of working with scientists to take specialized information and return data more useful to the broader scientific community. The workshop was a highly concentrated effort that packed a punch, and produced great results.
Because of the workshop’s success, the group is considering holding similar workshops in the future, focusing on developing new ARM data products.
Despite the long hours put in over the 10-day sprint, both scientists and developers left the workshop energized by their collaborative effort, thankful for the opportunity to focus their work on a single project and to produce results that exceeded their expectations.
# # #
The ARM Climate Research Facility is a national scientific user facility funded through the U.S. Department of Energy’s Office of Science. The ARM Facility is operated by nine Department of Energy national laboratories.