Journal cover Journal topic
Geoscientific Model Development An interactive open-access journal of the European Geosciences Union
Geosci. Model Dev., 10, 1403-1422, 2017
http://www.geosci-model-dev.net/10/1403/2017/
doi:10.5194/gmd-10-1403-2017
© Author(s) 2017. This work is distributed
under the Creative Commons Attribution 3.0 License.
Model evaluation paper
04 Apr 2017
Global gridded crop model evaluation: benchmarking, skills, deficiencies and implications
Christoph Müller1, Joshua Elliott2,3, James Chryssanthacopoulos2,3, Almut Arneth4, Juraj Balkovic5,6, Philippe Ciais7, Delphine Deryng2,3, Christian Folberth5,8, Michael Glotter9, Steven Hoek10, Toshichika Iizumi11, Roberto C. Izaurralde12,13, Curtis Jones12, Nikolay Khabarov5, Peter Lawrence14, Wenfeng Liu15, Stefan Olin16, Thomas A. M. Pugh4,17, Deepak K. Ray18, Ashwan Reddy12, Cynthia Rosenzweig19,3, Alex C. Ruane19,3, Gen Sakurai11, Erwin Schmid20, Rastislav Skalsky5, Carol X. Song21, Xuhui Wang7,22, Allard de Wit10, and Hong Yang15,23 1Potsdam Institute for Climate Impact Research, 14473 Potsdam, Germany
2University of Chicago and ANL Computation Institute, Chicago, IL 60637, USA
3Columbia University Center for Climate Systems Research, New York, NY 10025, USA
4Karlsruhe Institute of Technology, IMK-IFU, 82467 Garmisch-Partenkirchen, Germany
5International Institute for Applied Systems Analysis, Ecosystem Services and Management Program, 2361 Laxenburg, Austria
6Department of Soil Science, Comenius University in Bratislava, 842 15 Bratislava, Slovak Republic
7Laboratoire des Sciences du Climat et de l'Environnement. CEA CNRS UVSQ Orme des Merisiers, 91191 Gif-sur-Yvette, France
8Department of Geography, Ludwig Maximilian University, 80333 Munich, Germany
9Department of the Geophysical Sciences, University of Chicago, Chicago, IL 60637, USA
10Earth Observation and Environmental Informatics, Alterra Wageningen University and Research Centre, 6708PB Wageningen, the Netherlands
11Institute for Agro-Environmental Sciences, National Agriculture and Food Research Organization, Tsukuba, 305-8604, Japan
12Department of Geographical Sciences, University of Maryland, College Park, MD 20742, USA
13Texas AgriLife Research and Extension, Texas A&M University, Temple, TX 76502, USA
14Earth System Laboratory, National Center for Atmospheric Research, Boulder, CO 80307, USA
15Eawag, Swiss Federal Institute of Aquatic Science and Technology, 8600 Duebendorf, Switzerland
16Department of Physical Geography and Ecosystem Science, Lund University, 223 62 Lund, Sweden
17School of Geography, Earth & Environmental Sciences and Birmingham Institute of Forest Research, University of Birmingham, Edgbaston, Birmingham, B15 2TT, UK
18Institute on the Environment, University of Minnesota, Saint Paul, MN, USA
19National Aeronautics and Space Administration Goddard Institute for Space Studies, New York, NY 10025, USA
20Institute for Sustainable Economic Development, University of Natural Resources and Life Sciences, 1180 Vienna, Austria
21Rosen Center for Advanced Computing, Purdue University, West Lafayette, IN, USA
22Sino-French Institute of Earth System Sciences, Peking University, 100871 Beijing, China
23Department of Environmental Sciences, University of Basel, Petersplatz 1, 4003 Basel, Switzerland
Abstract. Crop models are increasingly used to simulate crop yields at the global scale, but so far there is no general framework on how to assess model performance. Here we evaluate the simulation results of 14 global gridded crop modeling groups that have contributed historic crop yield simulations for maize, wheat, rice and soybean to the Global Gridded Crop Model Intercomparison (GGCMI) of the Agricultural Model Intercomparison and Improvement Project (AgMIP). Simulation results are compared to reference data at global, national and grid cell scales and we evaluate model performance with respect to time series correlation, spatial correlation and mean bias. We find that global gridded crop models (GGCMs) show mixed skill in reproducing time series correlations or spatial patterns at the different spatial scales. Generally, maize, wheat and soybean simulations of many GGCMs are capable of reproducing larger parts of observed temporal variability (time series correlation coefficients (r) of up to 0.888 for maize, 0.673 for wheat and 0.643 for soybean at the global scale) but rice yield variability cannot be well reproduced by most models. Yield variability can be well reproduced for most major producing countries by many GGCMs and for all countries by at least some. A comparison with gridded yield data and a statistical analysis of the effects of weather variability on yield variability shows that the ensemble of GGCMs can explain more of the yield variability than an ensemble of regression models for maize and soybean, but not for wheat and rice. We identify future research needs in global gridded crop modeling and for all individual crop modeling groups. In the absence of a purely observation-based benchmark for model evaluation, we propose that the best performing crop model per crop and region establishes the benchmark for all others, and modelers are encouraged to investigate how crop model performance can be increased. We make our evaluation system accessible to all crop modelers so that other modeling groups can also test their model performance against the reference data and the GGCMI benchmark.

Citation: Müller, C., Elliott, J., Chryssanthacopoulos, J., Arneth, A., Balkovic, J., Ciais, P., Deryng, D., Folberth, C., Glotter, M., Hoek, S., Iizumi, T., Izaurralde, R. C., Jones, C., Khabarov, N., Lawrence, P., Liu, W., Olin, S., Pugh, T. A. M., Ray, D. K., Reddy, A., Rosenzweig, C., Ruane, A. C., Sakurai, G., Schmid, E., Skalsky, R., Song, C. X., Wang, X., de Wit, A., and Yang, H.: Global gridded crop model evaluation: benchmarking, skills, deficiencies and implications, Geosci. Model Dev., 10, 1403-1422, doi:10.5194/gmd-10-1403-2017, 2017.
Publications Copernicus
Download
Short summary
Crop models are increasingly used in climate change impact research and integrated assessments. For the Agricultural Model Intercomparison and Improvement Project (AgMIP), 14 global gridded crop models (GGCMs) have supplied crop yield simulations (1980–2010) for maize, wheat, rice and soybean. We evaluate the performance of these models against observational data at global, national and grid cell level. We propose an open-access benchmark system against which future model versions can be tested.
Crop models are increasingly used in climate change impact research and integrated assessments....
Share