To help develop and compare physical parametrizations such as those found in a numerical weather or climate model, a new tool was developed. This tool provides a framework with which to plug external parametrizations, run them in an offline mode (using one of the two time-advance methods available), save the results and plot diagnostics. The software can be used in a 0-D and a 1-D mode with schemes originating from various models. As for now, microphysical schemes from the Meso-NH model, the AROME (Applications of Research to Operations at Mesoscale) model and the Weather Research and Forecasting model have been successfully plugged. As an application, Physical Parametrizations with PYthon (PPPY) is used in this paper to suppress the origin of the time-step dependency of the microphysical scheme used in the Météo-France small-scale operational numerical weather model. The tool helped to identify the origin of the dependency and to check the efficiency of the introduced corrections.
A weather or climate numerical model contains several parametrizations (e.g., turbulence, convection) which interact not only together but also with the dynamical core. When a parametrization is being developed or debugged, these interactions can hide and/or amplify a tested modification through compensatory errors and feedbacks. When the goal is to compare two parametrizations hosted by different models, these interactions distort the comparison, as the other model components can be very different (dynamical core, discretization and other parametrizations). To circumvent these effects, one can reduce the interactions by unplugging other parametrizations (ideal cases, aqua-planet experiments) or by reducing the problem size (2-D vertical simulations, single-column model).
The choice of the comparison strategy depends on the intended goal: a full 3-D model is able to represent all the interactions, whereas simplified models represent only a subset of these interactions.
Even the single-column model (one of the simplest configurations), however, is not always sufficient for separating the impact of the different parametrizations (see, for example, point 8 of the conclusion of
An example of a toy model used to compare microphysical schemes is given by
Development motivation came from the observation of a difference on 3-D simulations with the Meso-NH model
When several hydrometeors are mixed in a model cell without exchange of matter or heat with the exterior, the microphysical processes tend towards a state of equilibrium.
This final state must not depend on the time step used.
In addition, when two (or more) simulations running with different time steps are compared, they should have the same results for common output times.
In Fig.
In the COnsortium for Small-scale MOdeling (COSMO) model,
The time-step dependency of the microphysical scheme is not specific to the ICE parametrization.
The dependency is also observed with the Liquid Ice Multiple Aerosols (LIMA) scheme
Section
Time evolution of the mixing ratio of the different hydrometeors (in kg kg
Same as Fig.
Same as Fig.
Documentation is provided with the software (see the code availability section). To complement this documentation, this section gives some details on how to add a parametrization and how the software works, as well as describing the functionalities related to the parametrizations and the comparisons.
The tool consists of a Python package which drives the simulations and performs the comparison: initialization, the calling of the Fortran routines (using the original source code of the parametrization), the saving of the results (in HDF5 files using the h5py module) and the plotting of the results (through the matplotlib module).
Two kinds of objects exist: those which represent a parametrization and those representing the comparison.
A standard object (an abstract class) is provided in order to define a parametrization (the PPPY box in Fig.
For the comparison, the provided class (PPPYComp in the figure) can be used directly or can be complemented (by inheritance, UserComp box in the figure) to add new diagnostics (e.g., new plot kind, computation of a derived variable to plot). An instance of the class is created for each comparison to perform (Comp box). A comparison is defined by the list of the parametrizations to use, the simulation length and the initial state. This comparison instance drives the parametrization instances to carry out the simulations and to plot the result.
These different objects are described in the following subsections.
In addition, in the provided package, the
Tool organization diagram. The top six boxes represent the parametrization objects, while the lower three boxes are the comparison objects.
The trickiest part comes from the interfacing between Python and the parametrization. This part is quite technical but important as the main difficulty in using PPPY with a new parametrization lies in this interfacing task.
If the parametrization was written using Python (like the box-Lagrangian scheme used in Sect.
The PPPY user is free to use whichever Python–Fortran interfacing method (among the two aforementioned or other ones). The ctypesForFortran way intends to help the interfacing of Fortran functions and subroutines on a Linux system. It handles memory allocations and array memory order. Internally, ctypesForFortran uses the Python ctypes module (which normally handles the C shared libraries) to interact with the library without adding a C or Fortran layer. It deals with Boolean, strings, integers and floats (32- and 64-bit) but does not support structures. The array and string arguments must be explicitly defined (no “:”, “..” or “*” are allowed in the interfaces), and no argument can be optional. If this is not the case, a wrapper must be written in Fortran meeting these requirements and calling for the original subroutine.
For the potential C-written parametrizations, interfacing can directly employ the ctypes module.
The compilation being a complex process (that can involve scripts that modify, on the fly, source codes), it could be difficult to isolate and compile, outside of the box, the source code needed for a given parametrization.
To reduce this difficulty, the various examples provided with PPPY (in the example directory) follow this procedure:
modification of the model compilation script and/or Makefile file to include the option to build a position-independent code, suitable for dynamic linking (“-fPIC” option); normal compilation of the model; and use of the various object codes and/or static libraries built during the normal compilation step to build a shared library with the different entry points needed by the parametrization.
At this stage, the remaining difficulty is to identify the different routines that must be called upon to perform the parametrization initialization and execution.
Once the compilation part is completed, a Python object must be created in order to manipulate the compiled library. An abstract class (PPPY) is provided for this purpose and must be used, by inheritance, to build a class specific to the parametrization employed.
The abstract class has placeholders for the requested standardized methods and these must be implemented.
Following the order of the execution flow, the first method to adapt is that of initialization. In this method, all the available options of the scheme are defined and consistency checks can be achieved. Among these options, one is the time step (mandatory); the others are specific to the scheme.
The following method concerns the setup. Here, the computations that need to be done once per simulation are performed. For example, it could be constant definitions, pre-calculation of lookup tables or file fetching.
The initial state provided at the beginning of a simulation is the same for all the parametrizations involved in the comparison. This state contains all the variables that must be monitored by all the schemes, although it is common for a given scheme to need additional prognostic variables. For example, when comparing a one-moment microphysical scheme with a two-moment microphysical scheme, only the first moment (mass) can be compared, but the two-moment scheme needs to monitor another moment (concentration). The build_init_state method is the place in which to define and initialize these additional variables and to add them to the initial state of the simulation. The output of this step is the first state saved in the output file.
The execute method is in charge of calling the actual code of the parametrization, making use of the compiled library. It might be necessary to perform numerical and/or physical conversions before and after this call. Indeed, the same quantities must be monitored by every scheme even if, internally, each scheme uses its own set of variables; physical conversion (for example, changing the temperature variable from potential temperature to true temperature) may therefore be needed. Moreover, modifications in memory representation may also be required (all variables are 64-bit in the Python script but can be converted into 32-bit, for instance).
This class is then instantiated by providing the required options (time step and options specific to the parametrization).
The parametrizations, which are instances created from the PPPY class (as stated above), are intended to be used by a Python object in order to perform a comparison.
This comparison object (instantiated from the PPPYComp class, to get the Comp object of Fig.
PPPY allows the comparison of several parametrizations.
The different parametrizations can differ from the underlying code or can differentiate themselves by the choice of the parameters controlling the scheme.
The comparison between two identical parametrizations using different time steps or different time-advance methods is also possible.
Two time-advance methods exist:
In In
The development was conducted in such a way as to allow the comparison of any parametrizations, not only microphysical schemes. The set of variables monitored is not limited to predefined ones; the user can add any variable of any dimensions. Moreover PPPY is able to use schemes from different models (interfacing with AROME, Meso-NH and WRF parametrizations has been done).
Two plot kinds are currently available but others can be written by extending the tool.
Plot methods already in existence can draw results for 0-D and 1-D simulations.
The
Time advance for a step-by-step simulation.
Time advance for a one-step simulation.
The ICE microphysical scheme is divided into three parts: a statistical adjustment to the saturation (to balance cloud water and ice with the vapor, according to the temperature), the core microphysical processes and the sedimentation. Each of these three parts may contain sources of time-step dependency.
The adjustment to saturation modifies the temperature and hence modifies also the saturation point and then the cloud content.
This feedback could be a source of time-step dependency, but it was checked (not shown) that the saturation adjustment used reaches an equilibrium very quickly; the impact of a second iteration can barely be detected.
The cloud ice fraction (the ice content divided by the total – ice and liquid – content) depends on the temperature (for temperatures above 0
In this section, two examples of the tool usage are shown. The first one deals with the time-step dependency due to the processes of the microphysical scheme (without adjustment and without sedimentation) in a 0-D mode; the second one is a comparison of several sedimentation schemes in a 1-D mode.
The final goal of this application was to suppress the time-step dependency of the simulations shown in Fig.
The simulations have been carried out several times, activating and deactivating the different microphysical processes. To do this, the processes have been called individually by the PPPY software (when they are written in separate subroutines) or activated through switches or, at worst, (un)commented in the source code. This trial-and-error experimentation makes it possible to identify the processes that led to the oscillations and to the time-step dependency, and allowed the checking of each correction individually from the others.
The purpose of this paper is not to detail the modifications that were needed to suppress the time-step dependency; however, the more important ones are listed in Appendix
With the revised version of the microphysical scheme, the simulated values for the different simulations shown in Figs.
The 0-D simulations were very useful for the identifying and correcting of the processes involved in the time-step dependency of the ICE microphysical scheme. It would certainly have been possible to achieve the same result with another method but this one was convenient (0-D simulations are very rapid, and a single tool performs the simulations, compares the outputs and plots the comparison) and allowed the complete isolation of the processes of interest from the other parts of the model (dynamics with transport, other physical parametrizations, sedimentation and adjustment).
Same as Fig.
Same as Fig.
Same as Fig.
A similar time-step sensitivity test is done regarding the sedimentation scheme used in the model (all other parametrizations, including the microphysical processes, were turned off).
Two schemes are available: the AROME operational one ( The term “internal time step” is reserved, in this paper, for the time step used internally in the scheme to perform the time splitting. It is different from the (external) time step used for the scheme integration.
Time evolution of the vertical profile of rain mixing ratio (the color scale represents the mixing ratio in
Vertical profiles of the rain mixing ratio (the color scale represents the mixing ratio in
Same as Fig.
Same as Fig.
A vertical profile was initialized with a rain mixing ratio of 0.1
Figure
For the Eulerian scheme (Fig.
Neither of these two schemes is correct; both diffuse the precipitation and have too low a fall speed. Additionally, the BSB2010 scheme exhibits a time-step dependency, whereas the Eulerian scheme is costly. A compromise can be found by optimizing the Eulerian scheme in order to obtain a scheme without time-step dependency and with a reasonable cost. In the optimized version, at each iteration in the time splitting, the time of integration is computed from the maximum CFL number on each column and for each species instead of using a constant (in space and time) value.
The result is shown in Fig.
In order to test further the impact of the algorithm on the sedimentation results, the box-Lagrangian scheme (used previously to build the reference results) is used in a simulation mode (using the step-by-step time-advance method).
Figure
On one hand, the algorithm choice may well be important because the selection modifies the effective fall speed and the relative position of precipitation on ground with respect to the cloud which generated it, through the horizontal advection. On the other hand, none of the schemes tested in this study are able to reproduce the reference simulation due to the diffusion appearing during the fall which leads in turn to a reduced speed. Additionally, with the mixing induced by the dynamic and the turbulence, and with the interaction with the other microphysical processes, this choice has little impact on the resulting simulation of a real 3-D case (not shown). It is believed that no scheme could perform drastically better with the one-moment hypothesis.
As from the conclusions of
This section illustrated how PPPY can be used to compare and exhibit the main behaviors of different 1-D sedimentation schemes written in Fortran and Python using different time-advance methods.
In this paper, a new software designed to allow the comparison of Physical Parametrizations with PYthon (PPPY) independently of all other model components was described technically and functionally. Its ability to use Fortran-compiled library from different models, as well as Python-based parametrizations, has been shown and used through two examples. It has been successfully used (in a 0-D mode) to identify the sources of the time-step dependency which were present in the microphysical scheme in use in the AROME and Meso-NH models. Solutions have been proposed to correct the scheme and have been tested with the software. The sedimentation schemes have then been plugged and compared (in a 1-D mode) to a reference box-Lagrangian scheme. These two examples have shown that it would be beneficial to use this kind of tool systematically when developing a parametrization in order to perform simple tests providing a first validation step (mass conservation, time-step dependency, absence of oscillations), before undergoing more complex validation stages (1-D model, full simulations).
In addition to the ICE scheme, the LIMA scheme and some of the WRF microphysical schemes have been plugged. It could now be used to compare microphysical schemes originally hosted by different models (AROME, Meso-NH and WRF). In order to compare these, it will be necessary to work on the initialization of the different schemes to ensure that the results can in fact be compared. In addition, it will be necessary to overcome the time-step dependency issue in order to allow the comparison of the different schemes. One way to do this could be to make a compromise by selecting a time step that is not too large to be able to mask the time-step dependency and, at the same time, not too small to limit the computation cost.
PPPY is not limited to microphysical schemes and, in the future, it could also be used to compare other parametrizations such as mass flux or turbulence schemes.
Several examples of PPPY usage are provided with the software.
Among them, a special example is intended to show how the different Python objects interact with each other and with the Fortran code; this is the test example which can be found in the
Let us assume that the following code is put inside a file named
It is suggested that the compilation procedure of the model be employed.
One must therefore ensure that the normal compilation of the model builds a position-independent code, suitable for dynamic linking (
To use ctypesForFortran, a wrapper must be written to hide certain characteristics.
The exposed dummy arguments
must not be of assumed shape or assumed rank (including string length) type; must not be optional; and of Boolean type must be 1 byte long.
In the test example, a wrapper (written in a file named
When it is possible, it is suggested that this wrapper be included in the source code of the model in order to benefit from the Makefile or compilation script. If this proves impossible, the wrapper must be compiled outside the model environment but one must use the same compilation options for the wrapper as those used for the model.
Often, a parametrization must be initialized by calling a specific subroutine (in particular to set up constant values).
In this example, this step is achieved by calling the following subroutine (which is also included in the
In addition, the example includes a
It is then supposed that a normal model compilation produces the compiled version of all these subroutines.
In the example, the compilation is obtained (using gfortran) by the following command lines:
The last step of the compilation process is to build a shared library with
To use the previously build shared library from Python using the ctypesForFortran module, the following code is needed:
The Python function must return three elements:
the list of the values expected in input by the Fortran subroutine; the list of the dummy arguments of the Fortran subroutine; and the type of the returned value for a Fortran function (“None” for a subroutine).
Each dummy argument is described by a tuple: type of the argument expressed as a NumPy type, shape (or “None” for a scalar) of the argument and input/output status.
More examples are available inside the ctypesForFortran module source code.
It should be noted that this part can be replaced by the use of other interfacing method such as f2py.
To use the parametrization with PPPY, a Python class must be written (see the __init__: this method can be implemented to deal with the possible parametrization options. setup: this method does the initialization part that cannot be done earlier (in the __init__method) or needs to be done again before each of the simulations. In the various provided examples, this is the place where the shared library is opened, where signatures of Fortran routines are defined and where the initialization of the Fortran modules is performed. finalize: this method can be useful for cleaning the memory or disk after running a simulation. build_init_state: when performing a comparison, each of the parametrizations is called with the same initial state. This method is the place at which to add state or diagnostic variables specific to the parametrization. execute: this method calls the parametrization to perform a time advance.
More details about these methods are given in the PPPY documentation.
The comparison is performed by the
Firstly, the parametrizations must be defined by choosing the time step (d
The comparison is defined and the simulations are performed by
The most important modifications that were needed to suppress the time-step dependency in the ICE scheme are listed below:
Heat budgets must be computed when the feedback on temperature can stop the process. For example, when the temperature exceeds 0 In the original version of ICE, the snow content rimed by cloud droplets (to produce graupel) was computed as an adjustment: the process provided the mass of snow to convert into graupel and this mass was then divided by the time step. The mass of transformed snow did not take into account the quantity of cloud water involved. These two characteristics were at the origin of a time-step dependency. The process was modified using the The graupel grows mainly by collecting other species. When this collection implies liquid species (rain and/or cloud), there are two possibilities (called growth modes) depending on a heat balance: the graupel is able to freeze the entire collected liquid (dry mode) or a thin liquid film appears at the graupel surface (wet mode). In the original version of the ICE scheme, there was confusion between the maximum content of liquid water that can be frozen (which must be used in the heat balance) and the content of liquid and ice water that can be collected in wet growth mode (which must be used to compute the graupel tendency). The correction made the mode choice more continuous. And because, ultimately, the graupel growth mode has an impact on the collection efficiency of icy species (snow and cloud ice) with the graupel, this choice can lead to significant differences in the collection rates. Hence, the scheme including the correction is less time-step dependent. The water shedding (cloud droplets becoming raindrops when collected and not frozen by the graupel) was activated only for temperature below 0 Several modifications have been carried out on the processes involving the hail category as a prognostic field: the processes dealing with hail are now completely symmetric with those dealing with the graupel category (to ensure consistency even if this did not produce time-step dependency). A conversion fraction is computed from the heat balance used to choose the graupel growth mode. In the original version of the scheme, this fraction was applied on the total content of graupel; this induced a conversion tendency directly linked to the number of times the rate is applied (hence to the time step for a given simulation length). On the contrary, in the new version, the conversion fraction is applied on the wet growth rate; this way, no time-step dependency is produced. This was the main reason for the time-step dependency on the hail category.
The modifications listed above aim at suppressing the time-step dependency present inside each of the microphysical processes.
These modifications were sufficient to suppress or, at least, limit the dependency for time steps up to 10
The modified scheme allows two splitting methods: a classical time-splitting method that uses a fixed sub-time step and a “mixing-ratio-splitting” method that computes, at each iteration, the sub-time step to use in order not to have a single mixing ratio change exceeding a given threshold. The second method has the advantage of adapting the number of iterations to the intensity of the microphysical processes. When little happens, only one iteration is performed; on the contrary, when the exchanges between species are intense, several iterations are performed.
To produce Figs.
PPPY is freely available under CeCILL-C license agreement (a French equivalent to the L-GPL license;
The author declares that there is no conflict of interest.
I wish to thank Andrew Barrett and the anonymous referee who have reviewed this paper. I thank Benoît Vié for having plugged the LIMA scheme into PPPY, as well as the trainees who worked on the plugging in of the WRF microphysical schemes: Aurélie Riandet, Louis Richecoeur and Marie-Laure Roussel. I also thank Christine Lac, Benoît Vié and Didier Ricard for their helpful comments on the manuscript.
This paper was edited by Klaus Gierens and reviewed by Andrew Barrett and one anonymous referee.