Journal cover Journal topic
Geoscientific Model Development An interactive open-access journal of the European Geosciences Union
Journal topic

Journal metrics

Journal metrics

  • IF value: 4.252 IF 4.252
  • IF 5-year value: 4.890 IF 5-year 4.890
  • CiteScore value: 4.49 CiteScore 4.49
  • SNIP value: 1.539 SNIP 1.539
  • SJR value: 2.404 SJR 2.404
  • IPP value: 4.28 IPP 4.28
  • h5-index value: 40 h5-index 40
  • Scimago H index value: 51 Scimago H index 51
Volume 10, issue 2 | Copyright
Geosci. Model Dev., 10, 537-552, 2017
© Author(s) 2017. This work is distributed under
the Creative Commons Attribution 3.0 License.

Methods for assessment of models 03 Feb 2017

Methods for assessment of models | 03 Feb 2017

A new and inexpensive non-bit-for-bit solution reproducibility test based on time step convergence (TSC1.0)

Hui Wan1, Kai Zhang1, Philip J. Rasch1, Balwinder Singh1, Xingyuan Chen1, and Jim Edwards2 Hui Wan et al.
  • 1Atmospheric Sciences and Global Change Division, Pacific Northwest National Laboratory, Richland, WA, USA
  • 2Climate and Global Dynamics Laboratory, National Center for Atmospheric Research, Boulder, CO, USA

Abstract. A test procedure is proposed for identifying numerically significant solution changes in evolution equations used in atmospheric models. The test issues a fail signal when any code modifications or computing environment changes lead to solution differences that exceed the known time step sensitivity of the reference model. Initial evidence is provided using the Community Atmosphere Model (CAM) version 5.3 that the proposed procedure can be used to distinguish rounding-level solution changes from impacts of compiler optimization or parameter perturbation, which are known to cause substantial differences in the simulated climate. The test is not exhaustive since it does not detect issues associated with diagnostic calculations that do not feedback to the model state variables. Nevertheless, it provides a practical and objective way to assess the significance of solution changes. The short simulation length implies low computational cost. The independence between ensemble members allows for parallel execution of all simulations, thus facilitating fast turnaround. The new method is simple to implement since it does not require any code modifications. We expect that the same methodology can be used for any geophysical model to which the concept of time step  convergence is applicable.

Publications Copernicus
Short summary
Solution reproductibility testing is an important task for assuring the software quality of a climate model. A new method is developed using the concept of numerical convergence with respect to temporal resolution. The method is objective, easy to implement, and computationally efficient. This paper describes the new test and demonstrates its utility in the Community Atmosphere Model version 5 (CAM5).
Solution reproductibility testing is an important task for assuring the software quality of a...