Journal cover Journal topic
Geoscientific Model Development An interactive open-access journal of the European Geosciences Union
Geosci. Model Dev., 10, 19-34, 2017
© Author(s) 2017. This work is distributed
under the Creative Commons Attribution 3.0 License.
Methods for assessment of models
02 Jan 2017
CPMIP: measurements of real computational performance of Earth system models in CMIP6
Venkatramani Balaji1,4, Eric Maisonnave2, Niki Zadeh3,4, Bryan N. Lawrence5,6, Joachim Biercamp7, Uwe Fladrich8, Giovanni Aloisio9,10, Rusty Benson4, Arnaud Caubel11, Jeffrey Durachta3,4, Marie-Alice Foujols12, Grenville Lister6, Silvia Mocavero9, Seth Underwood3,4, and Garrett Wright3,4 1Princeton University, Cooperative Institute of Climate Science, Princeton, NJ, USA
2Centre Européen de Recherche Avancée en Calcul Scientifique (CERFACS), Toulouse, France
3Engility Inc., Dover, NJ, USA
4NOAA/Geophysical Fluid Dynamics Laboratory, Princeton, NJ, USA
5National Centre for Atmospheric Science and University of Reading, Reading, UK
6Science and Technology Facilities Council, Abingdon, UK
7Deutsches Klimarechenzentrum GmbH, Hamburg, Germany
8Swedish Meteorological and Hydrological Institute, Norrköping, Sweden
9Centro Euro-Mediterraneo sui Cambiamenti Climatici (CMCC) Foundation, Lecce, Italy
10University of Salento, Lecce, Italy
11Laboratoire des Sciences du Climat et de l'Environnement LSCE/IPSL, CEA-CNRS-UVSQ, Université Paris-Saclay, 91191 Gif-sur-Yvette CEDEX, France
12Institut Pierre-Simon Laplace, CNRS/UPMC, Paris, France
Abstract. A climate model represents a multitude of processes on a variety of timescales and space scales: a canonical example of multi-physics multi-scale modeling. The underlying climate system is physically characterized by sensitive dependence on initial conditions, and natural stochastic variability, so very long integrations are needed to extract signals of climate change. Algorithms generally possess weak scaling and can be I/O and/or memory-bound. Such weak-scaling, I/O, and memory-bound multi-physics codes present particular challenges to computational performance.

Traditional metrics of computational efficiency such as performance counters and scaling curves do not tell us enough about real sustained performance from climate models on different machines. They also do not provide a satisfactory basis for comparative information across models. codes present particular challenges to computational performance.

We introduce a set of metrics that can be used for the study of computational performance of climate (and Earth system) models. These measures do not require specialized software or specific hardware counters, and should be accessible to anyone. They are independent of platform and underlying parallel programming models. We show how these metrics can be used to measure actually attained performance of Earth system models on different machines, and identify the most fruitful areas of research and development for performance engineering. codes present particular challenges to computational performance.

We present results for these measures for a diverse suite of models from several modeling centers, and propose to use these measures as a basis for a CPMIP, a computational performance model intercomparison project (MIP).

Citation: Balaji, V., Maisonnave, E., Zadeh, N., Lawrence, B. N., Biercamp, J., Fladrich, U., Aloisio, G., Benson, R., Caubel, A., Durachta, J., Foujols, M.-A., Lister, G., Mocavero, S., Underwood, S., and Wright, G.: CPMIP: measurements of real computational performance of Earth system models in CMIP6, Geosci. Model Dev., 10, 19-34, doi:10.5194/gmd-10-19-2017, 2017.
Publications Copernicus
Short summary
Climate models are among the most computationally expensive scientific applications in the world. We present a set of measures of computational performance that can be used to compare models that are independent of underlying hardware and the model formulation. They are easy to collect and reflect performance actually achieved in practice. We are preparing a systematic effort to collect these metrics for the world's climate models during CMIP6, the next Climate Model Intercomparison Project.
Climate models are among the most computationally expensive scientific applications in the...