Earth system models are fundamental to understanding and projecting climate change. The models have continued to improve over the years, but considerable biases and uncertainties in their projections remain. A large contribution to this uncertainty stems from differences in the representation of phenomena such as clouds and convection that occur at scales smaller than the resolved model grid. These long-standing deficiencies in the representation of subgrid scale physical processes, so-called parameterizations, have motivated developments of global high-resolution cloud-resolving models (horizontal grid resolutions of a few kilometres). While high-resolution models can explicitly resolve clouds and convection, due to high computational costs, they cannot be run at climate time-scales (multiple decades or longer). Yet short simulations from the computationally costly high-resolution models together with observations can serve as information to develop machine learning (ML)-based parameterizations that are then incorporated into Earth system models. The ICOsahedral Non-hydrostatic (ICON) model is an open-access modelling framework, which is used on a variety of time-scales and resolutions, ranging from numerical weather predictions to climate projections. Here we utilize existing regional and global cloud-resolving ICON simulations with data-driven techniques to train ML-based parametrizations. The newly developed ML-based parameterizations are coupled to the ICON Earth system model (ICON-ESM) via the Fortran-Keras Bridge, resulting in the ICON-ESM-ML hybrid model. We adjust or “tune” its properties in agreement with the observed state of the Earth’s climate system. The targeted properties of the model are the radiation balance at the top of the atmosphere, global mean temperature, clouds, precipitation and wind fields. We evaluate the ICON-ESM-ML hybrid model with observations using the Earth System Model Evaluation Tool (ESMValTool). The ESMValTool is a community-driven performance metrics tool that provides a comprehensive set of diagnostics for mean climate state and variability evaluation against observational data sets, and enables benchmarking against its predecessor (ICON-ESM).

In our talk, we propose to use neural networks in a hybrid modelling setup to learn sub-grid-scale dynamics of sea-ice that cannot be resolved by geophysical models. The multi-fractal and stochastic nature of the sea-ice dynamics imposes new problems for neural networks, and we will here introduce specific neural network architectures suited for this kind of task. To proof our concept, we will show results from idealized twin experiments with a simplified Maxwell-Elasto-Brittle sea-ice model which includes only sea-ice dynamics in a channel-like setup. In these idealized twin experiments, the goal of our hybrid modelling approach is to correct errors of low-resolution « forecast » runs compared to high-resolution « reality » runs before they appear. As these two kind of runs are on different grid resolutions, we have to interpolate them onto the same grid to train neural networks with such data. Thus, we will additionally present how the « reality » runs at higher resolution can be interpolated to the lower resolution grid by taking prior physical knowledge into account. Using the interpolated states of the « reality » runs as initial conditions, we can generate our low-resolution « forecast » runs that forecast the sea-ice dynamics at the next time step. Based on the forecasted and interpolated states at this next time step, we can build-up a catalogue for training and testing. This catalogue allows us then to learn and screen different neural network architectures with supervised training in an offline learning setting. Together with the simplified training, this screening will help us to select appropriate architectures for the representation of multi-fractality and stochasticity in the sea-ice dynamics. Later, these selected architectures can be also scaled to larger and more complex sea-ice models like neXtSIM. There, in conjunction with data assimilation, the neural networks can be then possibly learned with observations.

Accurate weather and climate models are essential for short-term weather resilience and long-term climate adaptation. Physics-based weather and climate models, however, can become computationally too expensive to be run many times, e.g., for uncertainty quantification, decision exploration, or real-time inference. Recent works propose physics-informed neural networks (PINNs) that can solve single instances of partial differential equations (PDEs) up to three orders of magnitude faster than traditional numerical solvers. However, most PINNs operate on fixed meshes and parameter choices. Hence, they have to be retrained for every choice of mesh or parameter and less applicable for uncertainty quantification.

A novel formulation, called neural operators, learns to approximate the PDE solver itself, not a single instance, and can solve a family of PDE equations over different meshes and parameter choices. Neural operators can be seen as an extension of neural networks to infinite-dimensions: They encode infinite-dimensional inputs into a finite-dimensional representations, such as Eigen or Fourier modes, and learn the nonlinear temporal dynamics in the encoded state. Neural operators, however, currently assume the existence of near-infinite PDE solutions as training samples, which is infeasible in the case of computationally expensive weather and climate models.

We are proposing a novel method, called matryoshka neural operator, that leverages a scheme from multi-scale fluid modeling, called superparametrizations. The new formulation allows us to assume large-scale dynamics as known, and leverage neural operators to only learn the fine-scale dynamics. Sharing the neural operator across large-scale grid cells drastically reduces the amount of required training data. The novel formulation is expected to significantly decrease the training cost and enable the learning of surrogate models over large domains, such as in weather and climate modeling. We show results on the multiscale Lorenz '96 equation that demonstrate a significant speed-up with respect to fully-resolved PDE solvers, less training data than neural operator-based PDE solvers, and accuracy gain with respect to polynomial-based parametrizations. We show a thorough benchmark on the best input/output combinations to train neural operators for learning fine-scale dynamics.

Particulate matter suspended in the air, called aerosols, have major negative impacts on the environment. It is the greatest environmental threat for human health, causing millions of deaths worldwide every year. Moreover, this air pollutant directly impacts the Earth radiative balance, scattering and absorbing light, consequently affecting the climate system. It's therefore of a great importance to be able to estimate and forecast the spatial distribution and variability of aerosols and how they interact with radiation. This last aspect is described by their optical properties, such as the aerosol optical depth (AOD).
Chemistry-transport models simulate the spatial distribution of aerosols by implementing numerically a set of equations describing atmospheric physics and chemistry. Models such as CHIMERE have proven a good capability to simulate the distribution of aerosols, but large uncertainties may be encountered, especially in the regions lacking ground-based measurements for constraining the simulations. Satellite measurements offer a great potential for observing the spatial distribution of aerosols from regional to global scale. However, they are only available for cloud free conditions and mostly once or twice a day (for the low orbit satellite). Currently, efforts to constraint model simulations using satellite measurements of AOD are mainly based on data assimilation techniques such as variational or Kalman filter approaches, whose implementation is complex, and it uses large computing resources.
In this work, we develop a new machine learning-based approach for constraining CHIMERE model simulations with AOD satellite measurement from the MODIS sensor. The machine learning module is used as post-processor for correcting systematic errors in the maps of AOD simulated by CHIMERE, using a three month period over the supra-equatorial African region. We compared the performance of three different machine learning methods: a deep feedforward neural network, a random forest, and k-nearest neighbours model. The best performing post processor has shown to clearly enhance the accuracy of the simulated AOD maps as compared to satellite observations.

Evapotranspiration (ET) constitutes a central water flux in the global hydrological cycle, closely coupled to the energy balance and carbon cycle, and is primarily governed by different meteorological variables and plant traits (Ajami, 2021; Jung et al., 2010). The characterization of ET remains challenging, since it involves non-linear processes associated with the different heterogeneous factors of the ecosystem such as plant physiological and biophysical traits, phenology, soil properties and climate regime (Chen et al., 2014; Friedl, 1996; Sellers et al., 1997). Different mathematical models have been proposed, which are either based on first principles, or based on the statistical extraction of information from observational data. The former, the so-called physics-based models, such as the Penman-Monteith (PM) equation (Jensen et al., 1990; Monteith, 1965; Penman, 1948), conserve physical laws and phenomenological behaviours. This enables the modelling of dynamic ET fluxes in response to the governing meteorological variables as well as interactions and feedbacks to the atmosphere (de Bezenac et al., 2017; Krasnopolsky, 2013). However, biases remain in these process-based models as they struggle to reproduce observed ET fluxes. These uncertainties and data issues are rooted in the limited understanding of how plants control the water flux by closing and opening their leaf stomata (Polhamus et al., 2013).
To overcome these uncertainties, it has been proposed to learn the controlling relationships and parameters from data using novel techniques of statistical learning (e.g. Jung et al., 2010). For example, the estimation of ET in terrestrial ecosystems using machine learning (ML) techniques has been increasingly applied. These methods revealed previously unknown or unidentified latent processes in controlling ET, allowing for better characterization and estimation of ET at an ecosystem scale (Dou & Yang, 2018). However, these black-box models are constrained by their need for sufficient data quantity and quality, by their inability to generalize to out-of-sample scenarios, and their inability to produce interpretable results that obey physical laws for conservation of energy and mass (Karpatne, Atluri, et al., 2017; Karpatne, Watkins, et al., 2017). Recent efforts have proposed different approaches to circumvent the issues rooted in pure physics-based and data-driven ML models. By combining the complementary strengths of both techniques, these hybrid models are thought to capture dynamic patterns, improve accuracy and make physically plausible predictions (Kraft et al., 2021; Reichstein et al., 2019). Therefore, we propose a novel approach to combining data-driven and physics-based modelling to better characterize and improve estimates of ET, and the biological regulator of the evaporative water flux, the stomata. The framework comprises setting up a Feed-forward Neural Network and integrating the physically-constraining PM equation in the loss function of the latent heat flux (LE). The stomatal resistance (rs) and aerodynamic resistance (ra) are modelled as intermediate latent variables, based on observations from the FLUXNET dataset. FLUXNET is a global network of eddy covariance towers, providing insight into the evaluation of fundamental terrestrial processes in the carbon and water cycles (Baldocchi et al., 2001; Li et al., 2018). For baseline comparison, two ML models have been set up, where the first model simulates LE directly without imposing any physical constraints and the second model follows the approach proposed by Zhao et al. (2019), where the main distinction lies in the formulation of the loss function. Preliminary results for the hybrid model show that it captures well the diurnal variations between the mean values of predicted and observed LE. To better understand the behaviour of the predicted rs, it was evaluated against its predictor variables, such as the vapor pressure deficit (VPD). The analysis shows physically realistic relationships, where, e.g., an increase in VPD results in an increase in stomatal resistance due to stomatal closure as a consequence of a higher atmospheric demand for evapotranspirated water. The outlook of hybrid modelling can be extended to global generalizations of flux estimates, and integrating it as part of a more complex framework of land surface models through the parameterization of rs.

Bibliography

Ajami, H. (2021). Geohydrology: Global Hydrological Cycle. In Encyclopedia of Geology (pp. 393–398). Elsevier. https://doi.org/10.1016/b978-0-12-409548-9.12387-5
Baldocchi, D., Falge, E., Gu, L., Olson, R., Hollinger, D., Running, S., Anthoni, P., Bernhofer, C., Davis, K., Evans, R., Fuentes, J., Goldstein, A., Katul, G., Law, B., Lee, X., Malhi, Y., Meyers, T., Munger, W., Oechel, W., … Wofsy, S. (2001). FLUXNET: A New Tool to Study the Temporal and Spatial Variability of Ecosystem-Scale Carbon Dioxide, Water Vapor, and Energy Flux Densities. Bulletin of the American Meteorological Society, 82(11), 2415–2434. https://doi.org/10.1175/1520-0477(2001)082 < 2415:FANTTS > 2.3.CO;2
Chen, Y., Xia, J., Liang, S., Feng, J., Fisher, J. B., Li, X., Li, X., Liu, S., Ma, Z., Miyata, A., Mu, Q., Sun, L., Tang, J., Wang, K., Wen, J., Xue, Y., Yu, G., Zha, T., Zhang, L., … Yuan, W. (2014). Comparison of satellite-based evapotranspiration models over terrestrial ecosystems in China. Remote Sensing of Environment, 140, 279–293. https://doi.org/10.1016/j.rse.2013.08.045
de Bezenac, E., Pajot, A., & Gallinari, P. (2017). Deep Learning for Physical Processes: Incorporating Prior Scientific Knowledge. ArXiv. http://arxiv.org/abs/1711.07970
Dou, X., & Yang, Y. (2018). Evapotranspiration estimation using four different machine learning approaches in different terrestrial ecosystems. Computers and Electronics in Agriculture, 148, 95–106. https://doi.org/10.1016/j.compag.2018.03.010
Friedl, M. A. (1996). Relationships among remotely sensed data, surface energy balance, and area-averaged fluxes over partially vegetated land surfaces. Journal of Applied Meteorology and Climatology, 35(11), 2091–2103. https://doi.org/10.1175/1520-0450(1996)035 < 2091:RARSDS > 2.0.CO;2
Halladay, K., & Good, P. (2017). Non-linear interactions between CO 2 radiative and physiological effects on Amazonian evapotranspiration in an Earth system model. Climate Dynamics, 49(7–8), 2471–2490. https://doi.org/10.1007/s00382-016-3449-0
Jensen, M. E., Burman, R. D., & Allen, R. G. (1990). Evapotranspiration and irrigation water requirements. ASCE Manuals and Reports on Engineering Practice No. 70, 1, 360. https://cedb.asce.org/CEDBsearch/record.jsp?dockey=0067841
Jung, M., Reichstein, M., Ciais, P., Seneviratne, S. I., Sheffield, J., Goulden, M. L., Bonan, G., Cescatti, A., Chen, J., De Jeu, R., Dolman, A. J., Eugster, W., Gerten, D., Gianelle, D., Gobron, N., Heinke, J., Kimball, J., Law, B. E., Montagnani, L., … Zhang, K. (2010). Recent decline in the global land evapotranspiration trend due to limited moisture supply. Nature, 467(7318), 951–954. https://doi.org/10.1038/nature09396
Karpatne, A., Atluri, G., Faghmous, J. H., Steinbach, M., Banerjee, A., Ganguly, A., Shekhar, S., Samatova, N., & Kumar, V. (2017). Theory-guided data science: A new paradigm for scientific discovery from data. IEEE Transactions on Knowledge and Data Engineering, 29(10), 2318–2.
Karpatne, A., Watkins, W., Read, J., & Kumar, V. (2017). Physics-guided Neural Networks (PGNN): An Application in Lake Temperature Modeling. ArXiv. http://arxiv.org/abs/1710.11431
Kraft, B., Jung, M., Körner, M., Koirala, S., & Reichstein, M. (2021). Towards hybrid modeling of the global hydrological cycle. Hydrology and Earth System Sciences Discussions, 1–40. https://doi.org/10.5194/HESS-2021-211
Krasnopolsky, V. M. (2013). The application of neural networks in the Earth system sciences.
Li, L., Wang, Y., Arora, V. K., Eamus, D., Shi, H., Li, J., Cheng, L., Cleverly, J., Hajima, T., Ji, D., Jones, C., Kawamiya, T., Li, W., Tjiputra, J., Wiltshire, A., Zhang, L., & Yu, Q. (2018). Evaluating global land surface models in CMIP5: Analysis of ecosystem water- and light-use efficiencies and rainfall partitioning. Journal of Climate, 31(8), 2995–3008. https://doi.org/10.1175/JCLI-D-16-0177.1
Monteith, J. L. (1965). Evaporation and environment. Fogg (Ed.), Symposium of the Society for Experimental Biology, The State and Movement of Water in Living Organisms, Vol. 19, 205–234.
Mystakidis, S., Davin, E. L., Gruber, N., & Seneviratne, S. I. (2016). Constraining future terrestrial carbon cycle projections using observation‐based water and carbon flux estimates. Global Change Biology, 22(6), 2198–2215. https://doi.org/10.1111/gcb.13217
Penman, H. L. (1948). Natural evaporation from open water, hare soil and grass. Proceedings of the Royal Society of London. Series A, Mathematical and Physical Sciences, 193(1032), 120–145. https://doi.org/10.1098/rspa.1948.0037
Polhamus, A., Fisher, J. B., & Tu, K. P. (2013). What controls the error structure in evapotranspiration models? Agricultural and Forest Meteorology, 169, 12–24. https://doi.org/10.1016/j.agrformet.2012.10.002
Reichstein, M., Camps-Valls, G., Stevens, B., Jung, M., Denzler, J., Carvalhais, N., & Prabhat. (2019). Deep learning and process understanding for data-driven Earth system science. Nature, 566(7743), 195–204. https://doi.org/10.1038/s41586-019-0912-1
Sellers, P. J., Dickinson, R. E., Randall, D. A., Betts, A. K., Hall, F. G., Berry, J. A., Collatz, G. J., Denning, A. S., Mooney, H. A., Nobre, C. A., Sato, N., Field, C. B., & Henderson-Sellers, A. (1997). Modeling the exchanges of energy, water, and carbon between continents and the atmosphere. In Science (Vol. 275, Issue 5299, pp. 502–509). American Association for the Advancement of Science. https://doi.org/10.1126/science.275.5299.502
Wang, K., & Dickinson, R. E. (2012). A review of global terrestrial evapotranspiration: Observation, modeling, climatology, and climatic variability. Reviews of Geophysics, 50(2). https://doi.org/10.1029/2011RG000373
Zhao, W. L., Gentine, P., Reichstein, M., Zhang, Y., Zhou, S., Wen, Y., Lin, C., Li, X., & Qiu, G. Y. (2019). Physics‐Constrained Machine Learning of Evapotranspiration. Geophysical Research Letters, 46(24), 14496–14507. https://doi.org/10.1029/2019GL085291

Oceanic numerical model outputs, reproducing the surface velocities often differ based on the model used in certain regions, especially on the level of (sub)mesoscale dynamics. The geostrophic velocity model output might even differ from the geostrophic velocities retreived through satellite altimetry observations of the Sea Surface Height, even if on-track measurements are often assimilated in numerical models. Satellite imagery, such as Sea Surface Temperature consists of a source of observation independent from altimetry measurements, where (sub)mesoscale structures, such as eddies have a visible signature. These eddy objects, detected through their signature on altimetry and visible imagery, can be used as an independent information to measure and validate the accurate reproduction of the surface dynamics by numerical models.

Machine Learning methods have proved very prominent in exploiting complex remote sensing information, such as the eddy signatures found on visible satellite imagery. We build Convolutional Neural Network which can accurately detect the position, shape and form of mesoscale eddies in satellite observations. Our CNN which only 3% of coherent structures, with more than 20km radius and a clear signature on the Sea Surface Temperature, compared with a 34% miss rate of standard eddy detection methods on altimetric maps. Additionally, while standard altimetric detection has a 10% false positive rate (“ghost eddies”) the neural network employed outputs less than 1% of ghosts.

This Neural Network is employed to validate in near real-time the surface dynamic outputs of oceanic numerical models, and serves as a tool to pick-and-choose between multiple model choices. As a real-time operational case study we compare the Operational Mercator and Mediterranean Forecast System models in the Mediterranean Sea, validating them through more than 2000 reference eddy detections per year. Potentially, these accurately detected reference eddy structures could be included in the data assimilation pipeline of numerical models as dynamical objects.