Journal cover Journal topic
Geoscientific Model Development An interactive open-access journal of the European Geosciences Union
Journal topic

Journal metrics

Journal metrics

  • IF value: 5.154 IF 5.154
  • IF 5-year value: 5.697 IF 5-year
    5.697
  • CiteScore value: 5.56 CiteScore
    5.56
  • SNIP value: 1.761 SNIP 1.761
  • IPP value: 5.30 IPP 5.30
  • SJR value: 3.164 SJR 3.164
  • Scimago H <br class='hide-on-tablet hide-on-mobile'>index value: 59 Scimago H
    index 59
  • h5-index value: 49 h5-index 49
Volume 10, issue 1
Geosci. Model Dev., 10, 413-423, 2017
https://doi.org/10.5194/gmd-10-413-2017
© Author(s) 2017. This work is distributed under
the Creative Commons Attribution 3.0 License.
Geosci. Model Dev., 10, 413-423, 2017
https://doi.org/10.5194/gmd-10-413-2017
© Author(s) 2017. This work is distributed under
the Creative Commons Attribution 3.0 License.

Development and technical paper 27 Jan 2017

Development and technical paper | 27 Jan 2017

The compression–error trade-off for large gridded data sets

Jeremy D. Silver and Charles S. Zender
Related authors  
Development and evaluation of pollen source methodologies for the Victorian Grass Pollen Emissions Module VGPEM1.0
Kathryn M. Emmerson, Jeremy D. Silver, Edward Newbigin, Edwin R. Lampugnani, Cenk Suphioglu, Alan Wain, and Elizabeth Ebert
Geosci. Model Dev., 12, 2195-2214, https://doi.org/10.5194/gmd-12-2195-2019,https://doi.org/10.5194/gmd-12-2195-2019, 2019
Short summary
Estimating global gross primary productivity using chlorophyll fluorescence and a data assimilation system with the BETHY-SCOPE model
Alexander J. Norton, Peter J. Rayner, Ernest N. Koffi, Marko Scholze, Jeremy D. Silver, and Ying-Ping Wang
Biogeosciences Discuss., https://doi.org/10.5194/bg-2019-83,https://doi.org/10.5194/bg-2019-83, 2019
Manuscript under review for BG
Short summary
Estimating global gross primary productivity using chlorophyll fluorescence and a data assimilation system with the BETHY-SCOPE model
Alexander J. Norton, Peter J. Rayner, Ernest N. Koffi, Marko Scholze, Jeremy D. Silver, and Ying-Ping Wang
Biogeosciences Discuss., https://doi.org/10.5194/bg-2018-270,https://doi.org/10.5194/bg-2018-270, 2018
Revised manuscript has not been submitted
Short summary
Modelling the impact of climate change on the atmospheric transport and the fate of persistent organic pollutants in the Arctic
K. M. Hansen, J. H. Christensen, C. Geels, J. D. Silver, and J. Brandt
Atmos. Chem. Phys., 15, 6549-6559, https://doi.org/10.5194/acp-15-6549-2015,https://doi.org/10.5194/acp-15-6549-2015, 2015
Contribution from the ten major emission sectors in Europe and Denmark to the health-cost externalities of air pollution using the EVA model system – an integrated modelling approach
J. Brandt, J. D. Silver, J. H. Christensen, M. S. Andersen, J. H. Bønløkke, T. Sigsgaard, C. Geels, A. Gross, A. B. Hansen, K. M. Hansen, G. B. Hedegaard, E. Kaas, and L. M. Frohn
Atmos. Chem. Phys., 13, 7725-7746, https://doi.org/10.5194/acp-13-7725-2013,https://doi.org/10.5194/acp-13-7725-2013, 2013
Related subject area  
Numerical Methods
LSCE-FFNN-v1: a two-step neural network model for the reconstruction of surface ocean pCO2 over the global ocean
Anna Denvil-Sommer, Marion Gehlen, Mathieu Vrac, and Carlos Mejia
Geosci. Model Dev., 12, 2091-2105, https://doi.org/10.5194/gmd-12-2091-2019,https://doi.org/10.5194/gmd-12-2091-2019, 2019
Short summary
FESOM-C v.2: coastal dynamics on hybrid unstructured meshes
Alexey Androsov, Vera Fofonova, Ivan Kuznetsov, Sergey Danilov, Natalja Rakowsky, Sven Harig, Holger Brix, and Karen Helen Wiltshire
Geosci. Model Dev., 12, 1009-1028, https://doi.org/10.5194/gmd-12-1009-2019,https://doi.org/10.5194/gmd-12-1009-2019, 2019
Short summary
A new method (M3Fusion v1) for combining observations and multiple model output for an improved estimate of the global surface ozone distribution
Kai-Lan Chang, Owen R. Cooper, J. Jason West, Marc L. Serre, Martin G. Schultz, Meiyun Lin, Virginie Marécal, Béatrice Josse, Makoto Deushi, Kengo Sudo, Junhua Liu, and Christoph A. Keller
Geosci. Model Dev., 12, 955-978, https://doi.org/10.5194/gmd-12-955-2019,https://doi.org/10.5194/gmd-12-955-2019, 2019
Short summary
Efficiency and robustness in Monte Carlo sampling of 3-D geophysical inversions with Obsidian v0.1.2: Setting up for success
Richard Scalzo, David Kohn, Hugo Olierook, Gregory Houseman, Rohitash Chandra, Mark Girolami, and Sally Cripps
Geosci. Model Dev. Discuss., https://doi.org/10.5194/gmd-2018-306,https://doi.org/10.5194/gmd-2018-306, 2019
Revised manuscript accepted for GMD
Short summary
DCMIP2016: the splitting supercell test case
Colin M. Zarzycki, Christiane Jablonowski, James Kent, Peter H. Lauritzen, Ramachandran Nair, Kevin A. Reed, Paul A. Ullrich, David M. Hall, Mark A. Taylor, Don Dazlich, Ross Heikes, Celal Konor, David Randall, Xi Chen, Lucas Harris, Marco Giorgetta, Daniel Reinert, Christian Kühnlein, Robert Walko, Vivian Lee, Abdessamad Qaddouri, Monique Tanguay, Hiroaki Miura, Tomoki Ohno, Ryuji Yoshida, Sang-Hun Park, Joseph B. Klemp, and William C. Skamarock
Geosci. Model Dev., 12, 879-892, https://doi.org/10.5194/gmd-12-879-2019,https://doi.org/10.5194/gmd-12-879-2019, 2019
Short summary
Cited articles  
Baker, A. H., Xu, H., Dennis, J. M., Levy, M. N., Nychka, D., Mickelson, S. A., Edwards, J., Vertenstein, M., and Wegener, A.: A methodology for evaluating the impact of data compression on climate simulation data, in: Proceedings of the 23rd International Symposium on High-performance Parallel and Distributed Computing, Vancouver, BC, Canada, 23–27 June 2014, ACM, 203–214, 2014.
Berger, T.: Rate-Distortion Theory, in: Wiley Encyclopedia of Telecommunications, John Wiley & Sons, Inc., https://doi.org/10.1002/0471219282.eot142, 2003.
Caron, J.: Converting GRIB to netCDF-4: Compression studies, presentation to the workshop "Closing the GRIB/netCDF gap", European Centre for Medium Range Weather Forecasts (ECMWF), 24–25 September 2014, Reading, UK, available at: http://www.ecmwf.int/sites/default/files/elibrary/2014/13711-converting-grib-netcdf-4.pdf, last access: 17 June 2016, 2014.
Dee, D., Uppala, S., Simmons, A., Berrisford, P., Poli, P., Kobayashi, S., Andrae, U., Balmaseda, M., Balsamo, G., Bauer, P., and Bechtold, P.: The ERA-Interim reanalysis: Configuration and performance of the data assimilation system, Q. J. Roy. Meteor. Soc., 137, 553–597, 2011.
Deutsch, L. P.: DEFLATE compressed data format specification version 1.3, Tech. Rep. IETF RFC1951, Internet Engineering Task Force, 2008.
Publications Copernicus
Download
Short summary
Many modern scientific research projects generate large amounts of data. Storage space is valuable and may be limited; hence compression is vital. We tested different compression methods for large gridded data sets, assessing the space savings and the amount of precision lost. We found a general trade-off between precision and compression, with compression well-predicted by the entropy of the data set. A method introduced here proved to be a competitive archive format for gridded numerical data.
Many modern scientific research projects generate large amounts of data. Storage space is...
Citation