Representing 3D effects in atmospheric radiation schemes

Representing 3D effects in
atmospheric radiation schemes

Robin Hogan, Najda Villefranque, David Meyer, Sophia Schaefer,
Mark Fielding, Peter Dueben, Carolin Klinger, Bernhard Mayer,
Meg Stretton, William Morrison, Sue Grimmond

ECMWF
r.j.hogan@ecmwf.int

 © ECMWF April 8, 2021

Overview
• Mechanisms for 3D radiative transfer and methods to represent them in atmospheric models
• The SPARTACUS method
 – Observational evaluation of shortwave side illumination mechanism from direct/diffuse surface fluxes
 – Shortwave entrapment mechanism
 – Possible mechanisms to explain the apparent overestimate of longwave 3D effects

• Machine learning to accelerate representation of 3D radiative effects
• Other applications of the SPARTACUS technique
• Outlook

 EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS October 29, 2014 2

Errors due to neglecting 3D effects
● Shortwave side illumination
 – Strongest when sun near horizon
 ● Shortwave entrapment
 – Increases chance of sunlight intercepting cloud – Horizontal transport beneath clouds
 makes reflection to space less likely

 ● Longwave effect
 – Radiation can be emitted from cloud sides
 – 3D effects can increase surface cloud forcing by
 a factor of 3 (for an isolated, optically thick, cubic
 cloud in vacuum, Schaefer et al. 2016)

Methods to represent 3D effects in atmospheric models
• Generalized Overlap • TenStream • SPARTACUS
 – Tompkins & Di Giuseppe (2007) – Jakub & Mayer (2015) – Hogan et al. (2016)
 – Zero cost – Cost x5-10 – Cost x4-5
 – Proposed for shortwave but – Suitable for cloud-resolving – Suitable for large-scale models
 neglects entrapment; better models
 – Part of ECMWF’s “ecRad”
 suited to longwave
 radiation scheme (open source)

 Radiation passed
 to adjacent
 gridboxes

 EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS October 29, 2014 4

SPARTACUS “Speedy Algorithm for Radiative Transfer through Cloud Sides”
 a
• Starting point is “Tripleclouds” method: va ua
 – Represent cloud heterogeneity via three regions
 a b b b c
 at each height (Shonk & Hogan 2008)
 v u
 a b c
 a
• Extra terms added to two-stream equations:

 = − 1 + 2 + − − + New terms
 representing
 exchange between
 − = − 1 + 2 + + − + regions
 
• Assuming clouds are randomly distributed and radiation is uniformly spread across each region:
 Length of cloud perimeter per unit gridbox area

 Fraction of gridbox occupied by clear skies (region a)

 Hogan & Shonk (JAS 2013), Hogan et al.,29,Schäfer
 October 2014 et al. (JGR 2016)

Observational evaluation of shortwave side illumination in SPARTACUS
• Low-cloud observations from the Azores:
 – Cloudnet retrievals → profiles of cloud
 fraction and liquid water content
 – Total-sky imager → cloud cover
 – Surface observations of direct and total
 solar irradiance

• At large solar zenith angle, direct beam
 blocked by cloud sides, an effect captured
 well by SPARTACUS but not Tripleclouds

 EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS Villefranque and Hogan (submitted
 October 29, 2014 to GRL) 6

What about the 60% of cloud scenes that are multi-layered? Entrapment!
• Some 3D transport is not associated with flow through cloud sides, so not treated correctly by SPARTACUS
• Solver computes an albedo matrix A at layer interfaces

 Region b
 
 
 Region a

 Tripleclouds, ICA: No 3D effects Better approach: compute RMS 2016 SPARTACUS: full
 requires diagonal albedo matrix horizontal migration distance, x, horizontal homogenization
 of light paths beneath cloud of radiation under clouds
 0
 = =
 0 /2 /2
 =
 /2 /2
 EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS October 29, 2014 7

Evaluating fluxes using Monte Carlo calculations on 100x100km CRM scenes
 • SPARTACUS with explicit entrapment matches Monte Carlo well, on average
 • Huge difference between maximum entrapment and zero entrapment Hogan et al. (JAS 2019)

 Cumulus Frontal cloud Stratocumulus Decaying Cb

 October 29, 2014

 8

Evaluating fluxes using all 65 scenes from “GEM” cloud-resolving model
• 3D radiative effect predicted by SPARTACUS agrees quite well with Monte Carlo
• Entrapment appears to win over side-illumination, but dependent on the realism of CRM scenes
• By contrast, Generalized Overlap method can only approximate side illumination, not entrapment

 Solar zenith angle: 20° Solar zenith angle: 80°

 Entrapment
 wins

 Side-illumination wins

 HoganOctober
 et al.29,(JAS
 2014
 2019) 9

Parameterizing cloud edge length L Fielding et al. (QJRMS 2020)

 Hypothesis refuted
Hypothesis 1: cloud
effective scale CS
constant with cloud
fraction a
 4 (1− )
 – =
 
Hypothesis 2: cloud
number, and hence Hypothesis validated
effective separation
CX, constant with a
 4 (1− )
 – =
 
 • Parameterization for cloud effective separation, CX, and hence edge
 length, as a function of cloud fraction, pressure and gridbox size
 EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS October 29, 2014 10

Global cloud radiative
 effect (CRE) from offline
 radiation calculations on
 ERA5 cloud field

 • This is “Tripleclouds” control
 calculation on one cloud field
 at 00 UTC, 1 April 2001
 • Cloud geometry similar to
 ECMWF operations: Exp-Ran
 overlap, fractional standard
 deviation of water content = 1

 cooling warming
EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS October 29, 2014 11

cooling warming
 Shortwave 3D effects
 • 3D effect is SPARTACUS
 minus Tripleclouds
 • Cloud spacing scale CX
 computed for resolution of 40
 km (close to ERA5)
 • Strong entrapment for deep
 clouds (fronts and deep
 convection)
 • TOA shortwave 3D effect
 close to zero, on average,
 due to cancellation between
 warming when sun is high
 and cooling when sun is low
 • Also strong cancellation
 between side illumination
 (cooling) and entrapment
 (warming)
EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS October 29, 2014 12

cooling warming
 Shortwave and
 longwave 3D effects
 • Longwave 3D effects warm
 everywhere, weakly
 • Instantaneous shortwave 3D
 effects are stronger but of
 both signs
 • Net warming from long-term
 average is 1.4 W m-2, which
 warms climate system by 1 K
 in coupled simulations (3 K at
 the North pole)
 • …but how accurate are
 longwave 3D effects from
 SPARTACUS?

EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS October 29, 2014 13

Preliminary evaluation of longwave 3D effects LW up
using MYSTIC Monte Carlo on 65 GEM scenes TOA

 • SPARTACUS appears to overestimate magnitude of
 3D effect
 • TOA overestimate is strongest for cases containing
 high clouds (x)
 • However, the MYSTIC calculations are not “clean”: 1D
 and 3D differ by 1 W m-2 in clear skies, explaining the
 apparent but unphysical 3D cooling effect
 • Pending a clean reference dataset, what mechanisms
 could explain an overestimate of longwave 3D effects LW down
 in SPARTACUS? surface

 Unphysical!

 EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS October 29, 2014 14

Why might longwave 3D effects be overestimated by SPARTACUS?
1. Clustering: 3D effect reduced, represented approximately by scaling
down the cloud edge length by factor of ~1.5 (Schaefer et al. 2016)

3. Shielding: some side emission 4. Angular dependence: steeper
immediately absorbed by layer rays more likely to reach ground
below; easier to represent using unabsorbed, but less likely to
Generalized Overlap method than escape from cloud sides: need
SPARTACUS more than two streams?
 2. “Norway clouds”: fractal
 behaviour means cloud edge length
 More chance of depends on scale, and may
 being absorbed by
 overestimate the chance of radiation
 gases
 passing through cloud sides
 October 29, 2014 15

Approximate representation of clustering and shielding effects
• Clustering: multiply • Shielding: use
 cloud scale by 1.5 Generalized Overlap
 (reducing perimeter rather than SPARTACUS,
 length by factor of 1.5) but using cloud scale to
 modify overlap parameter
• Somewhat better
 agreement, but still • In combination with
 overestimate 3D effect clustering factor, TOA
 at TOA for four scenes bias is reduced, although
 containing high clouds surface 3D effect appears
 to be underestimated

 • More work needed to
 understand and
 represent these effects!

 October 29, 2014 16

Machine learning of 3D effects
 SW up TOA LW up TOA
• Machine learning of full radiation
 scheme is difficult:
 – Flux biases need to be less than
 0.5 W m-2 (around 0.25%)
 – Difficult to avoid noise in
 stratospheric heating rates

• Alternatively, run a standard 1D
 radiation scheme and emulate the
 3D effect (training on 25,000
 SPARTACUS profiles)
 SW down LW down
 – 3D effect to within 20-30%,
 surface surface
 commensurate with random error
 in SPARTACUS
 – Avoid 4-5x increase in
 computational cost
 Meyer et al. (submitted to JAMES)

 October 29, 2014
 17

With Meg Stretton, Sue Grimmond and
 Other application of the SPARTACUS approach Will Morrison (University of Reading)

• “SPARTACUS-Surface” (Hogan 2019) for urban & forest radiative transfer, available on GitHub
• Compare to slow but accurate reference 3D radiation calculations from the “DART” model for a 2x2 km
 area of London (St Pauls and the Barbican), solar zenith angle = 75°, albedo = 0.1 and 0.5:

• Good agreement!
• We have also shown that complex city geometry can be characterized well by a handful of variables:
 surface building fraction, mean building height and effective building diameter, useful for NWP
• SPARTACUS-Surface has now been implemented in the Town Energy Balance model (TEB)

 October 29, 2014 18

Summary and outlook
• Entrapment is an important shortwave mechanism, well captured by SPARTACUS, and while
 locally the 3D effect can exceed 30 W m-2, there is strong cancellation across diurnal cycle
• Good agreement with direct/diffuse surface observations: can SPARTACUS be used to
 improve solar energy forecasts?
• Longwave 3D effect systematically warms the Earth system and has a net effect larger than
 shortwave
• Offline calculations (not shown in this talk) with SPARTACUS suggest 3D effects lead to an
 instantaneous net heating of 1.4 W m-2, which online lead to a 1 K warming globally, 3 K at the
 North pole
• BUT longwave effect is currently overestimated by SPARTACUS; more work is needed to
 improve it, and hence to revise estimates of the climate impact of 3D radiative effects…
• Could then retrain neural network to represent 3D effects with virtually no cost
• SPARTACUS idea well suited to other 3D radiative transfer problems involving randomly
 distributed obstacles, such as vegetation and urban areas
 See also http://www.met.reading.ac.uk/clouds/spartacus/
 EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS October 29, 2014 19

20

ecRad test dataset
• Pole-to-pole slice from ERA5 including Saharan dust, marine stratocumulus, deep convection
 and Arctic stratus
• Useful for testing different settings in ecRad, both for education and research

 EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS October 29, 2014 21

Visualization of 3D effects from SPARTACUS and machine learning
Total fluxes and heating rates 3D effects from SPARTACUS 3D effects from machine learning

 EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS October 29, 2014 22