Earth System Model Evaluation Tool (ESMValTool)

Earth System Model Evaluation Tool (ESMValTool)

Veronika Eyring 1Deutsches Zentrum für Luft- und Raumfahrt e.V. (DLR), Institute of Atmospheric Physics, Oberpfaffenhofen, Germany 2University of Bremen, Institute of Environmental Physics, Bremen, Germany

WGCM/WGNE Meeting 10 October 2017 Exeter, UK

CMIP6: Participating Model Groups

New in CMIP: 2 new model groups from Germany (AWI, MESSY-Consortium) 4 new model groups from China (CAMS, CasESM, NUIST, THU) 1 new model group from Brazil (INPE) 1 new model group from India (CCCR-IITM) 1 new model group from Taiwan, China (TaiESM) 1 new model group from USA (DOE) 2 new model group from Republic of Korea (NIMS-KMA, SAM0-UNICON) 1 new model group from South Africa / Australia (CSIR-CSIRO) ======================================== ⇒  13 new model groups so far

* Other models can join providing DECK and historical simulations are submitted

Institution Country Institution Country Institution Country 1 AWI Germany 12 DOE USA 23 MRI Japan 2 BCC China 13 EC-Earth-Cons Europe 24 NASA-GISS USA 3 BNU China 14 FGOALS China 25 NCAR USA 4 CAMS China 15 FIO-RONM China 26 NCC Norway 5 CasESM China 16 INM Russia 27 NERC UK 6 CCCma Canada 17 INPE Brazil 28 NIMS-KMA Republic of Korea 7 CCCR-IITM India 18 IPSL France 29 NOAA-GFDL USA 8 CMCC Italy 19 MESSY-Cons Germany 30 NUIST China 9 CNRM France 20 MIROC Japan 31 TaiESM Taiwan, China 10 CSIR-CSIRO South Africa 21 MOHC UK 32 THU China 11 CSIRO-BOM Australia 22 MPI-M Germany 33 Seoul Nat.Uni Republic of Korea

More models (>70)

New models

More complex models

Higher resolution models

CMIP5 MMM

CMIP5 MMM - OBS

Similar to Figure 9.7 of AR5

Similar to Figure 9.24 of AR5 Similar to Figure 9.5 of AR5

Similar to Figure 9.24 of AR5

Broad Characterization of Model Behavior

(incl. IPCC AR5 Chap 9 & 12 diagnostics in ESMValTool)

Net Cloud radiative effect against CERES EBAF

Running alongside the ESGF

Tools such as the community-developed Earth System Model Evaluation Tool (ESMValTool, Eyring et al., ESMValTool, GMD (2016b)) that includes other software packages such as the NCAR CVDP (Phillips et al., 2014)), and the PCMDI Metrics Package (PMP, Gleckler et al., EOS (2016)) to

produce well-established analyses as soon as CMIP model output is submitted.

Monsoon Precipitation Intensity

Link to projections

How to characterize the wide variety of models in CMIP6? - Routine Benchmarking and Evaluation Central Part of CMIP6 -

ESMValTool version 1.0 released as open source software

•  Community diagnostics and performance metrics tool for the evaluation of Earth System

•  Standardized model evaluation can be performed against observations, against other models or to compare different versions of the same model

•  Many diagnostics and performance metrics covering different aspects of the Earth System (dynamics, radiation, clouds, carbon cycle, chemistry, aerosol, sea-ice, etc.) and their interactions

•  Well-established analysis based on peer-reviewed literature

•  Ensuring traceability and provenance (e.g. input data, metadata, diagnostics (incl .citation), tool version, doi)

•  Documentation and user guide •  Currently ≈ 80 scientist from >30 institutions part of the

development team and > 120 users •  Development in several projects (e.g. APPLICATE,

CRESCENDO, C3S-MAGIC, ESA CMUG, PRIMAVERA) •  Rapidly expanding

DLR.de • Chart 4

h"p://www.esmvaltool.org/ Eyring et al., GMD, ESMValTool v1.0, 2016

Examples of European Projects with ESMValTool Development and Application

Name Funder Duration Scientific Focus Technical Focus Partners APPLICATE EU Horizon

2020 11/16-10/20

Arctic, user-relevant impact metrics, linkages in atmosphere & ocean, sea ice

--- AWI and other APPLICATE partners

C3S-MAGIC C3S-SQUARE4ECVs

Copernicus Climate Change Serv.

10/16-03/19

Metrics incl. extreme events, coastal, water, energy and insurance Quality Control Observations

Quasi-operational on new C3S-Server, rewrite of backend with IRIS

NLeSC, KNMI, DLR, URead, BSC, ISAC-CNR, SMHI

CMIP6-DICAD BMBF 07/16-06/20

Routine Benchmarking Coupling to ESGF at DKRZ; visualization

DLR, DKRZ, FUB

CRESCENDO EU Horizon 2020

11/15-10/20

IPCC Ch. 9&12, ESM diagnostics & metrics (terrestrial, marine, chemistry, aerosols)

Coupling to ESGF at BADC; reporting and testing

DLR, ETH, LMU, UREAD, ENEA, SMHI, UNEXE

DLR Projects DLR 2010-ongoing

Emergent constraints, aerosols, chemistry, clouds, sea ice

ESMValTool coord., efficiency, provenance

DLR

EMBRACE EU FP7 11/11-02/16

ESM diagnostics and metrics Get it running on all CMIP5 models, Documentation

DLR, SMHI, KNMI, MPI-M, FMI, ETH, UEA, UNEXE, METUK, CNRS-IPSL, CNRS-MF

ESA CCI CMUG

ESA 07/14-06/17

ESA CCI data and diagnostics Reporting DLR, LMU, SMHI, MetOffice

PRIMAVERA EU Horizon 2020

11/15-10/19

Assess added level of high res.; processes (e.g., AMO, Gulf stream, interactions of ice & polar storms, northward ocean heat transport

Improving the backend‘s efficiency

BSC and other PRIMAVERA partners

Current Status: Contributing Institutions (currently ~80 scientist from >30 institutions part of the development team )

1.  Deutsches Zentrum für Luft- und Raumfahrt (DLR), Institut für Physik der Atmosphäre, Germany PI 2.  Alfred-Wegener-Institute Bremerhaven (AWI), Germany Core Developer APPLICATE 3.  Barcelona Computing Center (BSC), Spain, Core Developer PRIMAVERA 4.  Ludwig Maximilian University of Munich, Germany, Core Developer CRESCENDO 5.  University of Reading, UK, Core Developer Copernicus MAGIC / CRESCENDO 6.  Colorado State University, USA 7.  Deutsches Klimarechenzentrum (DKRZ), Germany 8.  Agenzia nazionale per le nuove tecnologie, l’energia e lo sviluppo economico sostenibile (ENEA), Italy 9.  ETH Zurich, Switzerland 10.  Finnish Meteorological Institute (FMI), Finland 11.  GERICS Climate Service Center, Hamburg, Germany 12.  Geophysical Fluid Dynamics Laboratory (GFLD) NOAA, USA 13.  Instituto Nacional de Pesquisas Espaciais (INPE), Brazil 14.  Institute of Atmospheric Sciences and Climate – Consiglio Nazionale delle Ricerche (ISAC-CNR), Italy 15.  IPSL, France 16.  Ludwig Maximilian University of Munich, Germany 17.  Max-Planck-Institute (MPI) for Meteorology, Hamburg, Germany 18.  MPI for Biogeochemistry, Jena, Germany 19.  Met Office Hadley Centre, UK 20.  Meteo France, France 21.  MetNorway, Norway 22.  New Mexico Tech, USA 23.  Nansen Environmental and Remote Sensing Center, Norway 24.  National Center for Atmospheric Research (NCAR), USA 25.  Netherlands e-Science Center (NLeSC) 26.  KNMI, The Netherlands

27.  SMHI, Norrköping, Sweden 28.  Tyndall Centre, UK 29.  University of Arizona, USA 30.  University of East Anglia (UEA), UK 31.  University of Exeter, Exeter, UK 32.  University of Hamburg, Germany 33.  University of Leeds, UK 34.  Wageningen University, The Netherlands

Several institutes working on technical improvements and backend

www.esmvaltool.org • Chart 7

Technical Work ESMValTool Maintenance, Technical Infrastructure, Interfaces, and Documentation

•  Changed to versioning system git •  Documentation and user’s guide converted (Sphinx) •  Tagging and improved provenance (work in progress) •  New backend using Iris, full merge with Auto-Assess (work in progress) •  Automated testing and reporting package •  Quicklook capability added •  Visualization with FREVA (work in progress) •  Several coding workshops were held to enhance the ESMValTool


Diagnos>cs and metrics included •  Aerosol •  Blocking diagnostics •  Catchment analysis, runoff, ET •  Clouds •  Cloud regime error metric (CREM) •  CO2 and CH4 •  Diurnal cycle of convection •  Emergent constraints •  Evapotranspiration •  Indices for extreme events (Climdex) •  IPCC AR5 chapter 9 and 12 •  Land and ocean components of the

global carbon cycle •  Land-atmosphere coupling •  Land cover •  Marine biogeochemistry •  Madden-Julian Oscillation (MJO) •  NCAR climate variability diagnostics

package (CVDP)

•  Ozone, precursors and climate impacts •  Performance metrics •  Shifts in Austral jets •  Snowfall •  Soil moisture •  Sea surface temperature •  South Asian monsoon •  Sea ice •  Soil moisture •  Southern Hemisphere •  Southern Ocean (SOCCOM) •  Standardized precipitation index (SPI) •  Tropical variability •  Tropospheric Ozone •  West African monsoon •  Land cover •  Precipitation – soil moisture


Examples of ESMValTool Namelist •  namelist_CVDP.xml •  namelist_DiurnalCycle_xxx.xml •  namelist_Emmons.xml •  namelist_EmergentConstraints.xml •  namelist_Evapotranspiration.xml •  namelist_GlobalOcean.xml •  namelist_SAMonsoon.xml •  namelist_SPI.xml •  namelist_SeaIce.xml •  namelist_SouthernHemisphere.xml •  namelist_SouthernOcean.xml •  namelist_TropicalVariability.xml ESMV •  namelist_WAMonsoon.xml •  namelist_aerosol_CMIP5.xml •  namelist_anav13jclim.xml •  namelist_clouds_bias.xml •  namelist_eyring13jgr.xml

•  namelist_flato13ipcc.xml •  namelist_lauer13jclim.xml •  namelist_lauer17rse.xml •  namelist_mjo_mean_state.xml •  namelist_mmm.xml •  namelist_perfmetrics_CMIP5.xml •  namelist_reformat.xml •  namelist_reformat_obs.xml •  namelist_righi15gmd_ECVs.xml •  namelist_righi15gmd_Emmons.xml •  namelist_righi15gmd_tropo3_CMIP5.xml •  namelist_runoff_et.xml •  namelist_sm_pr.xml •  namelist_wenzel14jgr.xml •  namelist_williams09climdyn_CREM.xml


Diagnos>cs – IPCC AR5 chapter 9


Diagnos>cs – IPCC AR5 chapter 12

www.pa.op.dlr.de/ESMVal/ • Chart 12

Diagnos>cs – Performance metrics

0 0 M O N T H 2 0 1 6 | V O L 0 0 0 | N A T U R E | 3

LETTER RESEARCH

0.13% ± 0.03% per p.p.m.v. for high latitudes and 0.11% ± 0.03% per p.p.m.v. for the extratropics), but with significantly reduced uncertainties. Models without nitrogen limitations span the full range of CO2 fertilization (20%–60%), whereas the current models that include nitrogen limitations appear to underestimate CO2 fertilization (20%–25%), especially for the extratropical domain. These emergent constraints therefore give a consistent picture of a substantial CO2-fertilization effect and point to the need for further improvements in the treatment of nutrient limitations in ESMs.Online Content Methods, along with any additional Extended Data display items and Source Data, are available in the online version of the paper; references unique to these sections appear only in the online paper.

Received 15 June 2015; accepted 8 August 2016. Published online 28 September 2016.

1. Norby, R. J. et al. Forest response to elevated CO2 is conserved across a broad range of productivity. Proc. Natl Acad. Sci. USA 102, 18052–18056 (2005).

2. Leakey, A. D. B. et al. Elevated CO2 e!ects on plant carbon, nitrogen, and water relations: six important lessons from FACE. J. Exp. Bot. 60, 2859–2876 (2009).

3. Friedlingstein, P. et al. Climate–carbon cycle feedback analysis: results from the C4MIP model intercomparison. J. Clim. 19, 3337–3353 (2006).

4. Ciais, P. et al. in Climate Change 2013: the Physical Science Basis (eds Stocker, T. F. et al.) 465–570 (IPCC, Cambridge Univ. Press, 2013).

5. Anav, A. et al. Evaluating the land and ocean components of the global carbon cycle in the CMIP5 earth system models. J. Clim. 26, 6801–6843 (2013).

6. Piao, S. et al. Evaluation of terrestrial carbon cycle models for their response to climate variability and to CO2 trends. Glob. Chang. Biol. 19, 2117–2132 (2013).

7. Friedlingstein, P. et al. Uncertainties in CMIP5 climate projections due to carbon cycle feedbacks. J. Clim. 27, 511–526 (2014).

8. Allen, M. R. & Ingram, W. J. Constraints on future changes in climate and the hydrologic cycle. Nature 419, 224–232 (2002).

9. Hall, A. & Qu, X. Using the current seasonal cycle to constrain snow albedo feedback in future climate change. Geophys. Res. Lett. 33, L03502 (2006).

10. Cox, P. M. et al. Sensitivity of tropical carbon to climate change constrained by carbon dioxide variability. Nature 494, 341–344 (2013).

11. Wenzel, S., Cox, P. M., Eyring, V. & Friedlingstein, P. Emergent constraints on climate-carbon cycle feedbacks in the CMIP5 Earth system models. J. Geophys. Res. Biogeosci. 119, 794–807 (2014).

12. Keeling, C. D., Chin, J. F. S. & Whorf, T. P. Increased activity of northern vegetation inferred from atmospheric CO2 measurements. Nature 382, 146–149 (1996).

13. Zhao, F. & Zeng, N. Continued increase in atmospheric CO2 seasonal amplitude in the 21st century projected by the CMIP5 Earth system models. Earth Syst. Dynam. 5, 423–439 (2014).

14. Gray, J. M. et al. Direct human in"uence on atmospheric CO2 seasonality from increased cropland productivity. Nature 515, 398–401 (2014).

15. Graven, H. D. et al. Enhanced seasonal exchange of CO2 by northern ecosystems since 1960. Science 341, 1085–1089 (2013).

16. Keeling, C., Whorf, T., Wahlen, M. & Plicht, J. Interannual extremes in the rate of rise of atmospheric carbon dioxide since 1980. Nature 375, 666–670 (1995).

17. Piao, S. et al. Net carbon dioxide losses of northern ecosystems in response to autumn warming. Nature 451, 49–52 (2008).

18. Taylor, K. E., Stou!er, R. J. & Meehl, G. A. An overview of CMIP5 and the experiment design. Bull. Am. Meteorol. Soc. 93, 485–498 (2012).

1.4

1.6

1.2

1.4

1.6

1.2

CanESM2 CESM1-BGC GFDL-ESM2M NorESM1-ME MIROC-ESM MPI-ESM-LA HadGEM2-ES

0.02 0.04 0.06 0.08 0.10 Sensitivity of CO2 amplitude to CO2

0.000 0.010 0.020 0.030 Sensitivity of CO2 amplitude to CO2

8

6

4

2

0 0.6

Pro

babi

lity

dens

ityP

roba

bilit

y de

nsity

0.8 1.0 1.2 1.4 1.6 1.8 2.0

8.0

6.0

4.0

2.0

0.00.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0

GPP(2 × CO2)/GPP(1 × CO2)

GPP(2 × CO2)/GPP(1 × CO2)

GP

P(2

× C

O2)

/GP

P(1

× C

O2)

GP

P(2

× C

O2)

/GP

P(1

× C

O2) 10

a

c

b

d

Figure 3 | Emergent constraints on the relative increase of large-scale GPP for a doubling of CO2. a, c, Correlations between the sensitivity of the CO2 amplitude to annual mean CO2 increases at BRW (x axis) and the high-latitude (60° N–90° N) CO2 fertilization on GPP at 2 × CO2 (a) and the same for KMK and extratropical (30° N–90° N) GPP (c). The grey shading shows the range of the observed sensitivity. The red line shows the linear best fit across the CMIP5 ensemble together with the prediction error (orange) and error bars show the standard deviation for each data point. b, d, The probability density histogram for the unconstrained CO2 fertilization of GPP (black) and the conditional PDF arising from the emergent constraints (red) for BRW (b) and KMK (d).

Acknowledgements This work was funded by the Horizon 2020 European Union’s Framework Programme for Research and Innovation under Grant Agreement No. 641816, the Coordinated Research in Earth Systems and Climate: Experiments, kNowledge, Dissemination and Outreach (CRESCENDO) project and the DLR Klimarelevanz von atmosphärischen Spurengasen, Aerosolen und Wolken: Auf dem Weg zu EarthCARE und MERLIN (KliSAW) project. We acknowledge the World Climate Research Programme (WCRP) Working Group on Coupled Modelling (WGCM), which is responsible for CMIP, and we thank the climate modelling groups for producing and making available their model output. For CMIP the US Department of Energy’s Program for Climate Model Diagnosis and Intercomparison provides coordinating support and led the development of software infrastructure in partnership with the Global Organization for Earth System Science Portals. We thank ETH Zurich for help in accessing data from the ESGF archive.

Author Contributions S.W. led the study and analysis and drafted the manuscript with support from P. M.C. V.E. and P.F. contributed to the concept of the paper and the interpretation of the results. All co-authors commented on and provided edits to the original manuscript.

Author Information Reprints and permissions information is available at www.nature.com/reprints. The authors declare no competing financial interests. Readers are welcome to comment on the online version of the paper. Correspondence and requests for materials should be addressed to S.W. ([email protected]).

Reviewer Information Nature thanks V. Brovkin and the other anonymous reviewer(s) for their contribution to the peer review of this work.

Rel

ativ

e G

PP in

crea

se a

t CO

2 dou

blin

g

Wenzel et al., Nature, 2016 60oN-‐90oN ~37%

Doubling CO2 concentration will lead to an

increase in land photosynthesis of about a third.

Emergent Constraints Emergent Constrains are relationships across an ensemble of models, between some aspect of Earth system sensitivity and an observable trend or variation in the current climate.

Sherwood et al., 2014

radiosondes and reanalyses

Tian, 2015: Southern ITCZ index

www.pa.op.dlr.de/ESMVal/ • Chart 14

Observa>onal data

Data sets are grouped into 3 classes •  Tier 1

Data sets from the obs4MIPs and ana4MIPs archives: https://www.earthsystemcog.org/projects/obs4mips/ https://www.earthsystemcog.org/projects/ana4mips/

•  Tier 2 Other freely available data sets

•  Tier 3 Restricted data sets (e.g., license agreement required)


Available observa>onal data v1.0

•  ACCESS (mmrbc) •  AERONET (od550aer) •  AURA-MLS-OMI (tropoz) •  AURA-TES (vmro3) •  CARSNET (od550aer) •  CASTNET (concso4, concno3, concnh4) •  CIRRUS (mmrbc, mmrbcfree) •  CONCERT (mmrbc, conccnSTP14) •  CR-AVE (mmrbc) •  DC3 (mmrbc) •  EANET (concso4, concno3, concnh4) •  EMEP (concso4, concno3) •  EMMONS (various trace gases) •  ESRL (co2) •  GLOBALVIEW (vmrco) •  GTO-ECV (toz) •  HIPPO (mmrbc)

•  IMPROVE (concso4, concno3, concnh4, concbc, concoa, concpm2p5, concpm10)

•  INCA (conccnSTP5, conccnSTP14, conccnSTP120)

•  LACE (sizecn) •  Melpitz (sizecn) •  MODIS (od550aer) •  NIWA (toz) •  Putaud (sizecn) •  SALTRACE (mmrbc) •  TC4 (mmrbc) •  Texas (mmraer, mmrbc) •  Tilmes (vmro3) •  UCN-Pacific (conccnSTP3)

Aerosols + chemistry



•  AIRS (hur, ta) •  CERES (rsuscs, rsus, rsdscs, rsds,

rluscs, rlus, rldscs, rlds, rsutcs, rsut, rlutcs, rlut)

•  CloudSat (clt) •  CMAP (pr) •  CRU (tas, pr) •  ERA-40 •  ERA-Interim (ta, ua, va, zg, hus, tas,

tos, ps, psl, tauu, tauv, clwvi, clivi, sftlf, pr, evspsbl, hfls, hfss, rsns, rlns)

•  GPCC (pr)

•  HadCRUT (tas) •  HALOE (vmrh2o) •  MERRA (pr) •  MODIS (clivi, clwvi, clt) •  NCEP (ta, ua, va, zg, hus, tas) •  NOAA-PSD (rlut) •  OAFlux (hfls) •  SRB (rsut, rlut, rlutcs) •  TRMM (pr) •  Uwisc (clwvi)

Meteorology



Land •  GCP (co2flux) •  LandFlux-EVAL (et, et-sd)

Ocean •  Dong08-ARGO (mlotst) •  ETH-SOM-FFN (spco2) •  HadISST (ts) •  SeaWIFS (chl) •  SOCAT (spco2) •  Takahashi14 (talk) •  WOA09 (so, sos, to, tos) •  Woa2005 (o2)

Sea ice •  HadISST (sic) •  NSIDC (sic)


Examples of new observa>onal datasets

•  ACCESS-2 (conccnd5, conccnd10) •  Asmill (aerosol size) •  CFSR (psl) •  CloudSat (clt) •  ESA CCI AEROSOL (od550aer,

abs550, od550lt1aer, od870aer) •  ESA CCI CLOUD (clt, clwvi, clivi) •  ESA CCI GHG (xco2, xch4) •  ESA CCI OZONE (tro3, tropoz, toz) •  ESA CCI SEAICE (sic) •  ESA CCI SOILMOISTURE (sm) •  ESA CCI SST (ts) •  ESRL (surface CO2)

•  HadCRUT4 (tas) •  HIPPO (mmrbc) •  HWSD (soil carbon content) •  ISCCP (albisccp, clisccp, cltisccp,

cttisccp) •  JMA-TRANSCOM (CO2 exchange) •  LAI3g (leaf area index) •  MTE (gross primary productivity of

carbon) •  NDP (vegetation carbon content) •  NIWA (toz) •  TOMS (toz) •  WHOI-OAFlux (hfls, hfss)

Aerosols/chemistry/meteorology/land/ocean/sea ice

Envisaged Workflow for Routine Evaluation in CMIP - Ensuring traceability and provenance of the results -

Eyring et al., ESD (2016)

obs4MIPs ana4MIPs

Results at http://cmip-esmvaltool.dkrz.de/

Example for integration of the ESMValTool into the CMIP6 Workflow at the DKRZ*

DLR.de • Chart 20

Data management phase Produc>on phase

DKRZ ESGF

cmorized

na,ve

Technical quality control write simulation

output timestep publication to ESGF

ESGF local

ESGF remote Create compliance to CMIP conventions (CMORize, Metadata, etc.)

monitoring

routine evaluation web visualization

*Defined in the Project CMIP6-DICAD

MPI-ESM

EMAC

Rou>ne evalua>on

Scientific quality control AWI-CM


Some Questions

•  When do we make the results publically available? •  Can we establish common terms of use for the CMIP evaluation tools? •  How can we encourage active participation of the model groups in the

quality control of the CMIP evaluation results? •  How can we coordinate and quality control results from different tools (e.g.

performance metrics plot from PMP versus ESMValTool)? •  How can we encourage the CMIP6-Endorsed MIPs to contribute

additional diagnostics and metrics?

Veronika Eyring (DLR, Germany), Peter Cox (University of Exeter, UK), Greg Flato (CCCma, Canada) and Peter Gleckler (PCMDI, USA)

organized a workshop at the Aspen Global Change Ins,tute on

Earth System Model Evalua,on and Emergent Constraints (involving 6 CRESCENDO Researchers)

ESMValTool Workflow for routine evaluation at DKRZ

Derived from: Eyring et al., ESMValTool v1.0, GMD, 2016

Download data to Cache

plot

netCDF

log file

Web based Visualisation


http

://cm

ip-e

smva

ltool

.dkr

z.de

/ Visualiza>on with FREVA (Coopera>on with FUB in BMBF

CMIP6-‐DICAD) project, see h"p://cmip-‐esmvaltool.dkrz.de/

Filter by: •  theme •  variable •  model •  etc.

Preview Results

http

://cm

ip-e

smva

ltool

.dkr

z.de



Show ESMValTool configuration

http

://cm

ip-e

smva

ltool

.dkr

z.de



Alex was a member of the ESMValTool core development t e a m , h o s t e d s e v e r a l ESMValTool workshops in his department at the LMU, and w a s a g r e a t p a r t n e r i n CRESCENDO RT3 and other ESMValTool projects. He was an excellent scientist, full of humor, ideas, commitment, and professionality. It was just so enjoyable to work with him.

2nd Technical ESMValTool Workshop 15-16 November 2016, LMU Munich, Germany

1st ESMValTool Documentation and

Visualization Workshop 15-19 May 2017, LMU Munich

Documents

Earth System Model Evaluation Tool (ESMValTool)