Journal cover Journal topic
Geoscientific Model Development An interactive open-access journal of the European Geosciences Union
Geosci. Model Dev., 9, 1065-1072, 2016
https://doi.org/10.5194/gmd-9-1065-2016
© Author(s) 2016. This work is distributed under
the Creative Commons Attribution 3.0 License.
Development and technical paper
17 Mar 2016
Application of all-relevant feature selection for the failure analysis of parameter-induced simulation crashes in climate models
Wiesław Paja1, Mariusz Wrzesien2, Rafał Niemiec2, and Witold R. Rudnicki3,4 1Department of Computer Science, Faculty of Mathematics and Natural Sciences, University of Rzeszów, Rzeszów, Poland
2Department of Artificial Intelligence and Expert Systems, Faculty of Applied Informatics, University of Information Technology and Management, Rzeszów, Poland
3Institute of Informatics, University of Białystok, Białystok, Poland
4Interdisciplinary Centre for Mathematical and Computational Modelling, University of Warsaw, Warsaw, Poland
Abstract. Climate models are extremely complex pieces of software. They reflect the best knowledge on the physical components of the climate; nevertheless, they contain several parameters, which are too weakly constrained by observations, and can potentially lead to a simulation crashing. Recently a study by Lucas et al. (2013) has shown that machine learning methods can be used for predicting which combinations of parameters can lead to the simulation crashing and hence which processes described by these parameters need refined analyses. In the current study we reanalyse the data set used in this research using different methodology. We confirm the main conclusion of the original study concerning the suitability of machine learning for the prediction of crashes. We show that only three of the eight parameters indicated in the original study as relevant for prediction of the crash are indeed strongly relevant, three others are relevant but redundant and two are not relevant at all. We also show that the variance due to the split of data between training and validation sets has a large influence both on the accuracy of predictions and on the relative importance of variables; hence only a cross-validated approach can deliver a robust prediction of performance and relevance of variables.

Citation: Paja, W., Wrzesien, M., Niemiec, R., and Rudnicki, W. R.: Application of all-relevant feature selection for the failure analysis of parameter-induced simulation crashes in climate models, Geosci. Model Dev., 9, 1065-1072, https://doi.org/10.5194/gmd-9-1065-2016, 2016.
Publications Copernicus
Download
Short summary
Climate simulation crashes are caused by inadeqaute knowledge of parameters in models describing physical phenomena. We have performed reanalysis of the data on simulation crashes and have shown that they can be attributed to three parameters of the ocean model. This is a significant improvement over the original study, where crashes were attributed to eight parameters. Our results will help researchers to develop improved models.
Climate simulation crashes are caused by inadeqaute knowledge of parameters in models describing...
Share