This paper introduces a
Representation of cloud processes in numerical models is crucial for weather
and climate prediction. Taking climate modelling as an example, one may learn
that numerous distinct modelling systems are designed in similar ways,
sharing not only the concepts but also the implementations of some of their
components
This brings us to the conclusion that there is a potential demand for a library-type cloud-microphysics software package that could be readily reused and that would enable its users to easily benefit from developments of other researchers (by gaining access to enhancements, corrections, or entirely new schemes). The library approach would not only facilitate collaboration, but also reduce development time and maintenance effort by imposing a separation of cloud microphysics logic from other model components such as the dynamical core or the parallelisation logic. Such strict separation of concerns is also a prerequisite for genuine software testing.
The popularity of several geoscientific-modelling software packages that
offer shared-library functionality suggests the soundness of such an approach
– e.g. libRadtran
The motivation behind the development of the libclouph++ library introduced
herein is twofold. First, we intend to exemplify the possibilities of
library-based code reuse in the context of cloud modelling. Second, in the
long run, we intend to offer the community a range of tools applicable for
research on some of the key topics in atmospheric science such as the
interactions between aerosol, clouds and precipitation – phenomena that
still pose significant challenges for the existing tools and methodologies
The library can be used in simulation frameworks of different
dimensionalities, different dynamical cores, different parallelisation
strategies, and, in principle, models written in different programming
languages. The presented library is written in C++, a choice motivated by the
availability of high-performance object-oriented libraries and the built-in
“template” mechanism. C++ templates allow the implemented algorithms not to
be bound to a single data type, single array dimensionality, or single
hardware type (e.g. CPU/GPU choice). The library code and documentation are
released as free (meaning both gratis and libre) and open-source software –
a prerequisite for use in auditable and reproducible research
Openness, together with code brevity and documentation, are also crucial for enabling the users not to treat the library as a “black box”. While a self-contained package with a well-defined interface is black-box approach compatible, the authors encourage users to inspect and test the code.
Modelling of atmospheric clouds and precipitation employs computational
techniques for particle-laden flows. These are divided into Eulerian and
Lagrangian approaches
In the current release, a discussion of the key assumptions, a formulation of the scheme, a definition of the programming interface (API), an overview of the implementation, and example results.
The particle-based scheme, being a novel approach to modelling clouds
and precipitation, is discussed in more detail than the bulk schemes.
A description of the programming interface of
The library is equipped with Python bindings that allow one to use all of the
The paper is structured as follows. Formulation of an example modelling
framework is presented in Sect.
Appendix
The
Being a library,
A simple 2-D kinematic framework allows, and limits, one to study cloud
microphysical processes decoupled from cloud dynamics. In fact, the
differences between simulations when feedback on the dynamics is taken out
can lead to a better understanding of the role of flow dynamics
The flow model formulation is inspired by the 2-D framework described in
One may notice that the stationarity of the dry-air density field, together with phase-change-related variations in time of the temperature and the water vapour mixing ratio, imply time variations of the pressure profile. The deviations from the initial (hydrostatic) profile are insignificant, though.
The constant-in-time 2-D velocity field used in the presented
simulations. See discussion of Eqs. (
Example results presented in the following sections are based on
a modelling set-up designed for the 8th International Cloud Modelling
Workshop
The definition of the stream function
The initial profiles of liquid-water potential temperature
The domain is assumed to contain aerosol particles. Their dry size
spectrum is a bi-modal log-normal distribution:
In the examples presented in this paper, the model was initialised with
To maintain steady mean temperature and moisture profiles (i.e. to compensate
for gradual water loss due to precipitation and warming of the boundary layer
due to latent heating), mean temperature and moisture profiles are relaxed to
the initial profile. The temperature and moisture equations include an
additional source term in the form
For models that include a description of the cloud droplet size spectrum, the initial data for the droplet concentration and size are obtained by initialising the simulation with a 2 h long spin-up period. During the spin-up, precipitation formation, cloud drop sedimentation and the relaxation terms are switched off. The spin-up period is intended to adjust the initial cloud droplet size spectrum (not specified by the set-up) to an equilibrium with the initial condition.
The grid steps are 20
A sequence
diagram depicting control flow in a conceptual solver described in
Sect.
The conceptual solver is meant to perform numerical integration of a system
of heterogeneous transport equations, each equation of the form
The proposed solver features uncentered-in-time integration of the right-hand-side terms. The source terms that are not formulated as time derivatives are referred to as adjustments. The adjustments are applied after advection but before updating the right-hand-side terms.
The library code is not bound to this particular solver logic
– it is just an example intended to present the library API.
We refer the reader to
State variables for the three implemented schemes. Number of state variables
times the number of Eulerian grid cells plus the number of particle
attributes times the number of Lagrangian computational particles gives an
estimation of the memory requirement of a given scheme.
See Appendix
A common approach to represent cloud water and precipitation in
a numerical simulation is the so-called single-moment bulk approach.
The concepts behind it date back to the seminal works of
In an Eulerian framework, two transport equations for the cloud water
mixing ratio
Single-moment bulk microphysics is a simplistic approach. Without information about the shape of droplet size distribution, the model is hardly capable of being coupled to a description of aerosol- or radiative-transfer processes.
The basic idea is to maintain saturation in the presence of cloud water. Condensation/evaporation of cloud water triggered by supersaturation/subsaturation occurs instantaneously. Rain water forms through autoconversion of cloud water into rain (the negligible condensation of rain water is not considered). Autoconversion occurs only after a prescribed threshold of the cloud water mixing ratio is reached. Subsequent increase in rain water is possible through the accretion of cloud water by rain.
Cloud water is assumed to follow the airflow, whereas rain water falls relative to the air with a sedimentation velocity. Rain water evaporates only after all available cloud water has been evaporated and saturation is still not reached. In contrast to cloud water, rain water evaporation does not occur instantaneously. The rain evaporation rate is a function of relative humidity and is parameterised with an assumed shape of the raindrop size distribution.
Phase changes of water are represented with the so-called saturation
adjustment procedure. Unlike in several other formulations of the
saturation adjustment procedure
Any excess of water vapour with respect to saturation is instantly
converted into cloud water, bringing the relative humidity to
100 %. Similarly, any deficit with respect to saturation causes
instantaneous evaporation of liquid water. The formulation of the
saturation adjustment procedure takes the latent heat
release equation as a starting point. The heat source depicted with
When saturation is reached through condensation or evaporation of the
cloud water, the second integral in Eq. (
Noteworthy: the name adjustment reserved in Sect.
The collisions and coalescence of droplets are modelled with two separate
processes: autoconversion and accretion. Autoconversion represents collisions
between cloud droplets only, while accretion refers to collisions between
rain drops and cloud droplets. Both are parameterised in a phenomenological
manner as right-hand-side (rhs) terms following
In the Kessler formulation, the autoconversion source term is proportional to
Representation of sedimentation of rain water is formulated as a rhs
term Another commonly used approach is to alter the vertical
component of the Courant number when calculating advection
Employment of the upstream scheme brings several consequences. First, unlike
the cell-wise formulation of phase changes and coalescence, the sedimentation
scheme is defined over a grid column. Second, the combination of terminal
velocity, vertical grid cell spacing
The single-moment bulk scheme's API consists of one structure (composite data
type) and three functions, which are all defined within the
The
The saturation adjustment of state variables (cf. Sect. C++ Standard Template Library
The function arguments include references to containers storing
Forcings due to autoconversion and accretion are obtained through
a call to the
Representation of sedimentation is included in a separate function
Sequence
diagram of
Example results from a 2-D kinematic
simulation using the single-moment bulk scheme. All panels depict
model state after 30
With the prototype solver concept defined in Sect.
The single-moment bulk scheme is implemented as a header-only C++ library (i.e. one does not have to build it separately and link with it; just the header files are needed to use it). The implementation of the single-moment bulk scheme requires a C++ compiler compliant with the C++11 version of the language.
Variables, function arguments, and return values of physical meaning are all
typed using the Boost.units classes
The integrals in Eq. (
The simulation framework described in Sect.
A common extension of the single-moment bulk approach is a double-moment bulk
scheme. Similarly to the single-moment approach, the double-moment warm-rain
scheme assumes that condensed water is divided into two categories: cloud
water and rain water. In addition to the total mass of water in both
categories, concentrations of droplets and drops are also predicted. As
a result, the scheme considers two moments of particle size distribution,
hence the name. In the Eulerian framework, four transport equations for cloud
droplet concentration
The double-moment scheme implemented in
The formulation of the double-moment bulk scheme assumes aerosol, cloud, and rain spectra shapes (log-normal, gamma, and exponential, respectively). Aerosol is assumed to be well mixed throughout the whole domain and throughout the whole simulation time (uniform concentration per unit mass of dry air). Cloud water forms only if some of the aerosol particles are activated due to supersaturation. Activation and subsequent growth by condensation are calculated by applying the predicted supersaturation. As in the single-moment scheme, rain water forms through autoconversion and is further increased by accretion. Prediction of the mean size of cloud droplets and rain drops allows one to better link the parameterisation of autoconversion and accretion to the solutions of the collision–coalescence equation. As in the single-moment scheme, cloud water is assumed to follow the airflow, whereas rain water falls relative to the air. Evaporation of cloud and rain water is included in the formulation of phase changes (and hence includes the negligible diffusional growth of rain water).
Cloud droplets form from activated aerosol. The number of activated droplets
is derived by applying the Köhler theory to the assumed multi-modal
log-normal size distribution of aerosols. Freshly activated cloud droplets
are assumed to have a radius of
The size distribution of aerosols is not resolved by the model. To take into
account the decrease in aerosol concentration due to previous activation, in
each time step the number of available aerosols is approximated by the
difference between the initial aerosol concentration and the concentration of
pre-existing cloud droplets. Note that this approximation is valid for weakly
precipitating clouds only. For a strongly raining cloud, the model should
include an additional variable, the concentration of activated cloud
droplets. It differs from the droplet concentration because of
collision–coalescence
The changes in cloud and rain water due to condensation and evaporation
follow Eq. (8) in
The decrease in number concentration due to evaporation of cloud droplets and
rain drops is computed following the approach of
Parameterisation of autoconversion and accretion follows
Sedimentation is calculated in the same way as in the single-moment scheme
(see Sect.
The double-moment bulk scheme's API consists of one structure and two
functions, all defined within the
All processes are represented as right-hand-side terms. Contributions to the
rhs terms, due to phase changes and coalescence, are obtained through a call
to
The meaning of the template parameters and the function arguments is analogous
to the single-moment bulk scheme's API (see
Sect.
Sequence
diagram of
Example results from a 2-D kinematic
simulation using the double-moment bulk scheme. All panels depict
model state after 30
The cell-wise-formulated processes are handled in the following order:
activation, condensation/evaporation of cloud droplets, autoconversion,
accretion, and condensation/evaporation of rain. In principle, there are no
guarantees that the summed contributions from all of those processes,
multiplied by the time step, will be smaller than the available water
contents or droplet concentrations. To prevent negative values of water
contents or concentrations, each contribution to the rhs term evaluated
within
A diagram with an example calling sequence for the double-moment scheme is
presented in Fig.
The implementation of the double-moment scheme follows closely the
implementation of the single-moment scheme (see Sect.
Simulations presented in Sect.
Similarly to the results from the single-moment scheme presented in
Fig.
The cloud droplet concentration plot reveals that the model captures the
impact of the cloud-base vertical velocity (and hence the supersaturation) on
the concentration of activated cloud droplets. The highest concentrations are
found near the updraught axis, and the lowest near the updraught edges. There
is a difference in shape between the rain drop concentration field
The third scheme available in
The formulated particle-based model involves a Lagrangian component and an
Eulerian component (that is not part of the library). The Eulerian component
is responsible for advecting multiplicity location (i.e. spatial coordinates with zero, one, two or three components), wet radius squared dry radius cubed hygroscopicity parameter
Multiplicity depicts the number of particles represented by a
computational particle. All particles represented by one
computational particle are assumed to be spherical aqueous solution
droplets of radius
The amount of solvent is represented by the dry radius
The list of particle attributes can be extended. For example, parameters describing chemical composition of the solution or the electrical charge of a particle can be added. Adding new particle attributes does not increase the computational expense of the Eulerian component of the solver. However, extension of the phase space by a new dimension (the added attribute) potentially requires using more computational particles to achieve sufficient coverage of the phase space.
Most of the assumptions of the bulk models described in
Sects.
There are, however, two notable consequences of the assumptions of all
particles being composed of an aqueous solution and spherically shaped.
First, the humidity within the domain and the hygroscopicity of the aerosol
substance must both be high enough for the solution to be dilute. For
tropospheric conditions and typical complex-composition internally mixed
aerosol, this assumption is generally sound
It is also assumed that a particle never breaks up into multiple particles.
It is a reasonable assumption for the evaporation of cloud particles into
aerosol
There is not yet any mechanism built into the model to represent aerosol sources (other than regeneration of aerosol by evaporation of cloud droplets).
In the current version of the library, it is assumed that particle motion has two components: advection by the fluid flow and gravitational sedimentation with the terminal velocity. The library interface expects that the user will pass information on the flow velocity in the form of Courant number fields, one per each dimension. The Courant number is defined as the flow velocity times the ratio of the time step to the grid step in a given dimension. The Arakawa-C staggered grid is used and hence the Courant numbers represent velocities at the edges of the Eulerian grid cells.
Transport of particles by the flow is computed using the backward Euler
scheme:
The same procedure is repeated in other spatial dimensions if applicable (i.e. depending on the dimensionality of the Eulerian component). Periodic horizontal boundary conditions are assumed.
The growth rate of particles is calculated using the single-equation
(so-called Maxwell–Mason) approximation to the heat and vapour
diffusion process
Vapour and heat diffusion coefficients
As in the particle-based ice-microphysics model of
Particle terminal velocities used to
evaluate Sh and Nu are calculated using the parameterisation of
The representation of condensation and evaporation in the Lagrangian
component involves a sub-stepping logic in which the Eulerian component time
step
Within each sub-step, the drop growth equation is solved for each
computational particle with an implicit scheme with respect to the wet radius
but explicit with respect to
After each sub-step, in addition to application of a fraction of advective
tendency, the thermodynamic fields
Phase-change calculations are performed before any other processes. This is
because condensation and evaporation are the only processes modifying the
The coalescence scheme is an implementation of the Super Droplet Method (SDM)
described in
If geometric collisions are considered, the coalescence kernel has the
following form:
In each sub-time-step, the evaluated probability
Unlike in the formulation of
The “multiple coalescence” feature of SDM introduced in
Noteworthy: the collisional growth is represented in
a numerical-diffusion-free manner, that is, Lagrangian in particle radius
space (both dry and wet radius). This is an advantage over the Eulerian-type
schemes based on the Smoluchowski equation which exhibit numerical diffusion
Sedimentation velocity is computed using the formula of
One of the key parameters of the particle-based simulation is the number of
computational particles used. As in several recent cloud studies employing
particle-based techniques, the initial particle spatial coordinates are
chosen randomly using a uniform distribution. Consequently, the initial
condition has a uniform initial mean density of computational particles
(assuming all cells have the same volume). The value of this initial mean
density defines the sampling error in the particle parameter space,
particularly in the context of phase changes and coalescence, which are both
formulated on a cell-wise basis. The ranges of values used in the recent
studies are 30–250
The dry radii of the computational particles are chosen randomly with a uniform distribution in the logarithm of the radius. The minimal and maximal values of the dry radius are chosen automatically by evaluating the initial dry-size distribution. The criterion is that the particle multiplicity (i.e. the number of particles represented by a computational particle) for both the minimal and maximal radii be greater than or equal to 1.
The initial spectrum shape is arbitrary. Externally mixed aerosol may be
represented using multiple spectra, each characterised by a different value
of
In one and two dimensions, the grid cell volume
Equation (
For set-ups assuming the initial presence of cloud water within the model
domain, the equilibrium condition may be applied only to sub-saturated
regions within the model domain. The initial wet radius of particles within
the supersaturated regions is set to its equilibrium value at a given
threshold relative humidity
The particle-based scheme's API differs substantially from bulk schemes'
APIs. It features the object-oriented approach of equipping data structures
(referred to as classes) with functions (referred to as methods).
Furthermore, unlike the bulk schemes' APIs, the particle-based scheme is not
implemented as a header-only library, but requires linking to the
As in the case of bulk schemes, the scheme options are stored in a separate
structure
Other options of the particle-based scheme not meant to be altered during
simulation are grouped into a structure named
Unlike in the case of the bulk schemes, here the actual geometry and memory
layout of the Eulerian grid need to be known to map the particle spatial
coordinates to the Eulerian grid cell indices. The memory layout of array
data is represented in the API using the
The state of the Lagrangian component of the model (notably, the values of
particle attributes) is stored in an instance of the
Initialisation, time-stepping, and data output are performed by calling
The
Time-stepping is split into two methods:
The
Sequence diagram of
The last element of the particle-based scheme's API is the
Figure
In the case of bulk schemes (Figs.
In the presented calling sequence, the diagnostic methods are only called
within the output step. Depending on the modelling framework, such calls may
also be needed in every time step, for example, to provide data on a particle
surface for a radiative-transfer component, or the data on a particle mass
for a dynamical component of the solver. Note that a single call to
The Lagrangian component of the model is implemented using the Thrust library
the low data exchange rate between these two components (there
is never a need to transfer the state of all computational particles to the
Eulerian component residing in the main memory; only the aggregated size
spectrum parameters defined per each grid box are needed); and the possibility of performing part of the microphysics computations
asynchronously, simultaneously with other computations carried out on CPU(s)
(cf. Sect.
Since the version of the CUDA compiler available at the time of development
did not support C++11, the particle-based scheme was implemented using C++03
constructs only. Furthermore, the CUDA compiler does not support all C++
constructs used by the Boost.units library. For this reason,
a
The asynchronous launch/wait logic is left to be handled by the caller. In
the example program
Both in the cases of GPU and CPU configurations, the Mersenne Twister
Example results
from a 2-D kinematic simulation using the particle-based scheme. All panels
depict the model state after 30 min simulation time (excluding the spin-up
period). The black overlaid squares mark grid cells for which the dry and wet
size spectra are shown in Fig.
Plots of
dry and wet size spectra for ten locations within the model domain. The
locations and their labels
Figures
Figure
The plot of the effective radius in Fig.
The ten black squares overlaid on each plot in Fig.
To match the pathway of cloud evolution, we shall discuss the panels in
Fig.
Computational cost
of the three microphysics schemes expressed as wall-clock time per time step
per grid box. Values measured for different settings of process-toggling
options shown (bottom horizontal axis). Results obtained with the
particle-based scheme are grouped by the number of computational particles
used (upper helper horizontal axes). See Sect.
Computational cost of a microphysics scheme is one of the key factors
determining its practical applicability. Here, we present a basic analysis of
the computational cost of the three schemes presented in this paper. The
analysis is based on timing of simulations carried out with the kinematic
framework and the simulation set-up described in Sect. advection only, advection and phase changes, advection, phase changes and coalescence, and all of the above plus sedimentation.
For the particle-based scheme, the advection-only runs include transport of
particles and the Eulerian fields (moisture and heat).
Simulations were performed with a 6-core AMD Phenom II CPU and a 96-core
nVidia Quadro 600 GPU (an example 2010 prosumer desktop computer). The CPU
code was compiled using GCC 4.8 with -Ofast, -march
In order to eliminate from the reported values the time spent on simulation startup, all simulations were repeated twice, performing a few time steps in the first run and a dozen time steps in the second run. The long and short run times were subtracted and the result was normalised by the difference in the number of time steps.
In order to reduce the influence of other processes on the wall-clock timing, all simulations were additionally repeated three times, and the shortest measured time is reported.
The particle-based simulations were performed with three different mean
densities of computation particles, 8, 32 and 128 per grid cell, and with
four “backend” settings:
serial backend, OpenMP backend using two threads, OpenMP backend using four threads, and CUDA backend using the GPU.
The test was completed for single-precision arithmetics. The GPU used offered
about 3 times higher performance at single precision. Higher-performance GPU
hardware available in computing centres is expected to deliver similar
performance for double precision. Execution times for CPU-only calculations
hardly change when switching from double to single precision.
Figure
Figure
Finally, Fig.
The library is still at its initial stage of development, and improvements in performance are expected.
The main goal of developing
The implementation of the library was done with maintainability and
auditability as priorities. This is reflected in
the choice of C++ with its concise and modularity-encouraging syntax As of the current release, the separation of code elements related to the schemes' formulation (formulæ) from other elements of the library (API, numerics); the adoption of compile-time dimensional analysis for all physically meaningful expressions in the code; the delegation of a substantial part of the library implementation to external libraries (including the dimensional analysis, algorithm parallelisation and GPU hardware handling); and the hosting of library development and handling of code dissemination through a public code repository.
All of the above, supported by the choice of the GNU General Public License,
underpin our goal of offering reusable code.
This section presents some key elements of a mostly standard approach to analytic description of motion of moist air, particularly in the context of modelling of the warm-rain processes. It is given for the sake of completeness of the formulation and to ease referencing of particular equations from within the text and the source code.
There are three key types of matter considered in the model formulation, and
their densities
It is assumed already in Eq. (
Equations (
A continuity equation for water vapour density
The way the potential temperature was defined in the preceding section gives
a degree of freedom in the choice of
Substituting
Another common choice of
The principal role of any cloud-microphysics scheme is to close the equation
system defined by Eqs. ( water vapour mixing ratio, and potential temperature.
The example simulations discussed in the text were performed with
The code of
Build automation for
A tree of libcloudph++'s and icicle's major dependencies. In addition to these libraries, several components require the C++11 compiler and CMake at build time.
Control over simulation options of
Simulations may be stopped at any time by sending the process a SIGTERM or
SIGINT signal (e.g. using the
The library is released under GNU General Public License v3.0. The 1.0
release of the library accompanying this publication is available for
download as an electronic supplement to the paper and is tagged as “1.0.0”
at the project repository. See project website for a list of pointers to
relevant resources:
In the current development workflow, we employ continuous integration on
Linux with GNU
S. Arabas thanks Shin-ichiro Shima (University of Hyogo, Japan) for
introducing the particle-based simulations. We thank D. Jarecka (University
of Warsaw) and G. Feingold (NOAA) for insightful discussions and comments on
the initial version of the manuscript. We acknowledge contributions to