COVID 19 non pharmaceutical intervention portfolio effectiveness and risk communication predominance - Nature
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
www.nature.com/scientificreports
OPEN COVID‑19 non‑pharmaceutical
intervention portfolio effectiveness
and risk communication
predominance
Louis Yat Hin Chan1,2,3*, Baoyin Yuan1,4 & Matteo Convertino2,5
Non-pharmaceutical interventions (NPIs) including resource allocation, risk communication, social
distancing and travel restriction, are mainstream actions to control the spreading of Coronavirus
disease 2019 (COVID-19) worldwide. Different countries implemented their own combinations of
NPIs to prevent local epidemics and healthcare system overloaded. Portfolios, as temporal sets
of NPIs have various systemic impacts on preventing cases in populations. Here, we developed a
probabilistic modeling framework to evaluate the effectiveness of NPI portfolios at the macroscale.
We employed a deconvolution method to back-calculate incidence of infections and estimate the
effective reproduction number by using the package EpiEstim. We then evaluated the effectiveness
of NPIs using ratios of the reproduction numbers and considered them individually and as a
portfolio systemically. Based on estimates from Japan, we estimated time delays of symptomatic-
to-confirmation and infection-to-confirmation as 7.4 and 11.4 days, respectively. These were used
to correct surveillance data of other countries. Considering 50 countries, risk communication and
returning to normal life were the most and least effective yielding the aggregated effectiveness of 0.11
and − 0.05 that correspond to a 22.4% and 12.2% reduction and increase in case growth. The latter
is quantified by the change in reproduction number before and after intervention implementation.
Countries with the optimal NPI portfolio are along an empirical Pareto frontier where mean and
variance of effectiveness are maximized and minimized independently of incidence levels. Results
indicate that implemented interventions, regardless of NPI portfolios, had distinct incidence
reductions and a clear timing effect on infection dynamics measured by sequences of reproduction
numbers. Overall, the successful suppression of the epidemic cannot work without the non-linear
effect of NPI portfolios whose effectiveness optimality may relate to country-specific socio-
environmental factors.
Pandemic and interventions. The ongoing COVID-19 pandemic is determining an unprecedented sys-
temic impact on socio-economic activities worldwide as well as positive environmental outcomes1,2. COVID-19
is certainly the most serious public health crisis after the 1918 flu pandemic, but in many regards the worst crisis
of humanity considering the systemicity of the problem in relation to the globalization of h umanity3. For this
very precise reason and the unparalleled availability of data, this pandemic also offers the opportunity to ana-
lyze and reflect, as never before, about all elements of risks—vulnerability, exposure, and hazard—in an holistic
way. The pandemic initially emerged in Wuhan, China in December 20194. As of the 14th June 2020, more than
7.6 million confirmed cases and 0.4 million deaths from 215 countries or regions all over the world have been
reported according to the World Health Organization (WHO)5. Currently, the virus continues to spread glob-
ally, independently of country development status, or rather affecting more developed countries than low- and
middle-income countries due to their globalization enhancing transmission. Europe experienced a downward
trend6 after the second wave but third waves are emerging in many places, including Europe and Asia, where
1
Graduate School of Medicine, Hokkaido University, Sapporo, Japan. 2Nexus Group, Graduate School of
Information Science and Technology, Hokkaido University, Sapporo, Japan. 3Department of Infectious Disease
Epidemiology and Modelling, Norwegian Institute of Public Health, Oslo, Norway. 4School of Mathematics, South
China University of Technology, Guangzhou, China. 5Institute of Environment and Ecology, Tsinghua Shenzhen
International Graduate School, Tsinghua University, Shenzhen, China. *email: imlouischan@gmail.com
Scientific Reports | (2021) 11:10605 | https://doi.org/10.1038/s41598-021-88309-1 1
Vol.:(0123456789)www.nature.com/scientificreports/
the epidemic was thought to be contained. Thus, we cannot expect the pandemic to end anytime soon and
comprehensive public health efforts are necessary to control the pandemic in the future. In relation to this,
data- and decision-driven efforts that look into the effectiveness of multiple joined interventions in a portfolio
perspective7–10 are necessary rather than just producing case forecasts as many efforts are o ngoing11,12. Some
13
evaluation tools have been proposed recently, such as the Global COVID Index and the Government Response
Stringency Index14, but these tools focus on ranking countries or measuring the amount of interventions rather
than dissecting and quantifying vulnerabilities and interventions precisely. Additionally, these indices rely on
information of the pandemic as it is without embracing a probabilistic approach and questioning wether sys-
tematic uncertainties to correct are present. With these issues in mind the fundamental question is whether one
predominant intervention, connected to all feasible others, exists in abating the risk.
Intervention portfolio and uncertainty. In response to the rapid spread of COVID-19 governments
around the world have implemented multiple non-pharmaceutical interventions (NPIs) to mitigate or suppress
COVID-19 infections15–19. Pharmaceutical interventions such vaccines started to be available in January 2021 so
NPIs only affected incidence rate in previous periods from COVID-19 onset with different degrees of individual
restriction. According to various social and political conditions, each country has taken different schemes of
NPIs involving different intervention options considering type, timing, duration and hypothesized s trength20,21.
These interventions have been driven by a reaction approach in which most countries were reacting to increasing
cases by implementing what other countries implemented to restrict individual behavior leading to infection.
Certainly this approach is suboptimal because it does not consider (i) the underlying socio-environmental and
economic factors of a country, which can also encompass measures to reduce exposure; and (ii) the portfolio
effect related to all interventions’ interdependencies, with the inclusion of this shared effect information7–9. A
balanced intervention strategy to significantly reduce morbidity and mortality due to infections, but also to
minimize the impact on daily life of people, is considered an optimal intervention portfolio. A portfolio where
environmental health and risk information quality (or misinformation, vice versa) are also considered as pri-
mary non coercive interventions at the population scale. Thus, the systemic evaluation of the effectiveness of
interventions is valuable for public health authorities to develop preparedness plans timely, evaluate interven-
tions over time, and design policies to decrease population vulnerabilities in long term. This evaluation should
be based on reliable information which has also practical implications for risk communication a posteriori.
Incidence and information. Epidemic curves of incidence provide the most valuable and intuitive infor-
mation (in terms of estimates and visualization) of the temporal trend of epidemics. Generally, the epidemic
curve available from public health authorities are based on dates of illness onset or reported cases. These dates
are defined by the moment in which surveillance systems report cases but these dates may come late and do not
take into account the time delay related to symptom manifestation and official reporting. Thus, the reconstruc-
tion of unobserved epidemic curves by dates of true infection would be immediately useful to estimate the
effective reproduction numbers. In order to reconstruct epidemic curves we employ a deconvolution method
that reconstructs the expected signal of infections before a cause (biological and systematic delays) produces
modifications on that signal. Some deconvolution methods were proposed in literature to back-calculate infec-
tion curves, for example for HIV infections from AIDS time series22 and for influenza infections from death
curves23. The Richardson-Lucy (RL) deconvolution was used in some recent studies to evaluate transmission of
the COVID-19 p andemic24–26 and this is the model we use here to reconstruct case time series from imperfect
surveillance data. The RL model was slightly modified for a better treatment of noise and the easy incorporation
of available non-linear a priori information27.
Proposed intervention effectiveness. The aim of this study is to develop a probabilistic pattern-ori-
ented framework to quantitatively analyze the instantaneous and systemic impact of interventions by evaluating
their effectiveness considering portfolios of implemented interventions. We emphasize how our approach is a
complex system approach like Haug et al.10 where with a non-mechanistic model and limited assumptions we
search for emerging patterns in data, in this case of effectiveness. This has implications for extracting epidemio-
logical and intervention information useful for epidemic evaluation and control decision making. The analysis is
performed by using the incidence curves of COVID-19 for 50 countries. The procedure to evaluate the effective-
ness of interventions from incidence curves of confirmed cases consists of three steps:
• First, misreported curves of confirmed cases are corrected by removing negative incidence. After, incidence
curves of infections were back-calculated from the curves of confirmation by using the RL deconvolution
method23. The reporting delay was assumed as the convolution sum of the incubation period (with estimated
mean and standard deviation of 5.5 and 2.3 d ays28) and the symptomatic-to-confirmation reporting delay
that was estimated using maximum likelihood method (MLE);
• Second, the non-overlapping instantaneous reproduction numbers were estimated by using the EpiEstim
package29,30. The generation time, defined as the time interval between primary and secondary cases, was
assumed to be identical to the serial interval, defined as the time interval of illness onset, following a gamma
distribution yielding mean 4.5 days and SD 4.5 d ays31. The non-overlapping reproduction numbers were
assumed to be constant between implementations; and,
• Third, the effectiveness of portfolios, defined as the ratio of the reproduction numbers for each time in which
one or multiple interventions were implemented, was evaluated. Then single intervention and aggregated
portfolio effectiveness was quantified for each country.
Scientific Reports | (2021) 11:10605 | https://doi.org/10.1038/s41598-021-88309-1 2
Vol:.(1234567890)www.nature.com/scientificreports/
Time delay distribution
infection−to−confirmation
symptomatic−to−confirmation
infection−to−symptomatic (incubation)
0.15
Probability
0.10
0.05
0.00
0 5 10 15 20 25 30 35
Days
Figure 1. Distribution of time delays. The distribution of infection-to-confirmation f ec (t) (black) is a
convolution of log-normal and gamma distributions, symptomatic-to-confirmation f sc (t) (red) and incubation
period f se (t) (blue). The log-mean and log-SD of symptomatic-to-confirmation distribution are 1.79 and 0.65,
estimated using the data from Japan. The shape and scale parameters of incubation period distribution are 5.807
and 0.948 r espectively28. The mean values of infection-to-confirmation, symptomatic-to-confirmation and
incubation period were 11.4, 7.4, and 5.5. Standard deviations were 4.7, 5.4 and 2.3, respectively.
The proposed portfolio model of intervention effectiveness is enormously useful because it allows one to (i)
rectify data and evaluate surveillance systems; (ii) quantify the best intervention in term of risk abatement and
in consideration of portfolio interdependencies, which can be useful to evaluate countries holistically; and (iii)
tailor interventions during epidemics and design public health policy where population specific environment
and communication, which contribute to systemic vulnerability, are taken into account.
Results
Time delay from infection to confirmation. The first step was to remove misreported incidence that is
the reported negative incidence; then the % of misreporting was the proportion of this negative incidence. The
overall misreporting percentage is low, i.e. 0.4%, and that corresponds to 24 out of 5405 cases that were misre-
ported (Table S1). The symptomatic-to-confirmation period (i.e., confirmation or reporting period) following a
log-normal distribution is more appropriate in consideration of the many factors affecting reporting as well as
the event that reporting may not even occur. Thus, a distribution with a fatter tail is more appropriate to capture
all these elements. Conversely, the incubation period distribution is much more “deterministic” and related to
the biology of Coronavirus that is invariant across geographies (at least for the period we considered, i.e. the first
wave). Thus, the symptomatic-to-confirmation time was fitted by a log-normal distribution using 8917 individu-
als from Japan32, yielding the estimates of the mean and standard deviation were 7.4 (95%CI 7.2–7.5) and 5.4
(95%CI 5.2–5.6) days. The time delay distribution of infection-to-confirmation is a convolution of gamma and
log-normal distributions (incubation period and symptomatic-to-confirmation, respectively). The mean and
standard deviation values were 11.4 and 4.7 days, respectively. All three distributions are shown in Fig. 1.
Japan intervention effectiveness. The evaluation of transmission and intervention dynamics was first
applied to Japan as the time delay is estimated using the data from Japan. Japan is the only country with reported
reliable data of time delays from infection to symptoms and from symptoms to confirmation. Figure 2A,B show
the cumulative and incidence curves of both infection and confirmation. The infection curve was obtained by
iterating rc = 4 times in the deconvolution process (see SI Fig. 24). We found that the first case was infected on
14-Jan-2020 (Day 0). The infection curve reached the peak on 2-Apr-2020 (Day 80) with 633 infected cases. The
difference between two cumulative curves illustrates the time delay of about 12 days.
We initially considered 124 interventions implemented on N = 49 days from 21-Jan-2020 to 4-May-2020
in Japan. They were then categorized into 8 major themes and they became 79 intervention implementations
represented by the black bars in Fig. 2E. Further, the first intervention on 21-Jan-2020 was discarded due to a low
number of cumulative cases to estimate the reproduction number. As a result, 78 interventions implemented on
N = 48 days were included in this study (see SI CSV file and the original data20,21 for further details).
Both non-overlapping Rn and overlapping instantaneous reproduction numbers Rt were estimated using the
reconstructed infection curve (Fig. 2C). The mean of initial reproduction number Rn=0 was 2.26 from 14-Jan-2020
to 22-Jan-2020. The reproduction number was high at the beginning, and reached the maximum Rn=10 = 2.58
on 6-Feb-2020. The reproduction number mostly kept above one for two months (Feb-2020 and Mar-2020) and
dropped below one, Rn=41 = 0.93 from 6-Apr-2020. After the last implementation on 4-May-2020, Rn=48 = 0.76
Scientific Reports | (2021) 11:10605 | https://doi.org/10.1038/s41598-021-88309-1 3
Vol.:(0123456789)www.nature.com/scientificreports/
A Japan (JP)
Confirmation
Cumulative
15000
Infection
curves
10000
5000
0
B Confirmation
600
Incidence
Infection
curves
400
200
0
C
Reproduction
4 Rn (non−overlapping)
numbers
3 Rt (overlapping)
2
1
0
D
effectiveness
100.2
Portfolio
100.1
100
10−0.1
10−0.2
E log10(Effectiveness) 0.0 0.2 0.4
viii
Intervention
vii
themes
vi
v
iv
iii
ii
i
an
an
eb
ar
ar
pr
pr
ay
ay
un
−M
−M
−A
−A
−M
−M
−F
−J
−J
−J
04
20
15
31
07
16
03
19
06
22
Figure 2. Reconstructed epidemic curves and evaluation of interventions implemented in Japan. (A) The
cumulative curves of confirmation (red) and infection (black). (B) The incidence curves of confirmation (red)
and infection (black). (C) The estimated instantaneous reproduction number, using non-overlapping windows
Rn (black) and using weekly overlapping windows Rt (blue). The lines and shaded areas represent the means and
confidence intervals, respectively. (D) The portfolio effectiveness of each implementation ηn, which is defined as
(m)
the ratio of mean R̄n−1 to R̄n. Note that y-axis is shown in log10-scale. (E) The intervention ηn (in rectangles)
and aggregated η (in circles) effectiveness of each intervention theme. The eight themes are (i) Case
(m)
identification, contact tracing and related measures, (ii) Environmental measures, (iii) Healthcare and public health
capacity, (iv) Resource allocation, (v) Risk communication, (vi) Social distancing, (vii) Travel restriction, and (viii)
Returning to normal life. The blue (or red) color refers to the interventions that successfully reduced infections
(or increment of infections). The log10-values of aggregated effectiveness of the six implemented themes were
(i) − 0.16, (iii) 0.09, (iv) − 0.12, (v) 0.53, (vi) 0.18, and (vii) − 0.06, respectively.
stayed constant for a month until 7-Jun-2020. Additionally, we found that the overlapping Rt grew above one
and reached Rt = 1.16 on 22-May-2020, which Rn=48 could not capture.
As the non-overlapping reproduction numbers Rn varied over time, the portfolio effectiveness was evaluated
with a large variation shown in Fig. 2D. Fluctuations were large at the beginning and relatively stable in terms of
magnitude at a later stage. Of 48 implementations, 30 were effective with ηn > 100 = 1. All intervention imple-
mentations from 19-Mar-2020 to 1-May-2020 were effective as the corresponding reproduction numbers were
decreasing. However, after the reproduction number crossed the critical value of one, the last two implementa-
tions did not effectively suppress infections from 2-May-2020 with η47 = 10−0.06 and η48 = 10−0.14.
The intervention and aggregated portfolio effectiveness of each intervention theme was evaluated for each
portfolio implementation and for the whole time period (Fig. 2E, circle and squares, respectively). The theme
(v) Risk communication was implemented 19 times, and it was the most implemented intervention; while (ii)
Environmental measures and (viii) Returning to normal life were not implemented in Japan. In general, there
were more interventions implemented at earlier periods, in Feb-2020 and Mar-2020. In particular, there were 4
intervention themes at most implemented on the same day, the 13-Feb-2020 and 18-Mar-2020, and those were
assumed to contribute equally to the calculation of their intervention effectiveness. Of 78 interventions, 44 were
(m)
effective with ηn > 100 = 1, where the vast majority of them belonged to the theme (v) Risk communication.
Scientific Reports | (2021) 11:10605 | https://doi.org/10.1038/s41598-021-88309-1 4
Vol:.(1234567890)www.nature.com/scientificreports/
A New Zealand (NZ) B Italy (IT)
Confirmation Confirmation
75 6000
Incidence Infection Infection
curves
50 4000
25 2000
0 0
log10(Effectiveness) 0.05 0.10 0.15 0.20
log10(Effectiveness) 0.00 0.05 0.10 0.15
viii viii
Intervention
vii vii
themes
vi vi
v v
iv iv
iii iii
ii ii
i i
eb
ar
ar
ar
r
r
an
an
eb
ay
ay
ay
un
ec
ar
ar
r
ay
ay
un
p
p
p
−M
−M
−M
−A
−A
−M
−M
−A
−M
−M
−M
−M
−M
−D
−F
−F
−J
−J
−J
−J
11
23
13
13
31
07
07
26
19
08
20
31
08
26
26
04
15
27
02
20
C Thailand (TH) D United States of America (US)
Confirmation Confirmation
150 Infection 30000 Infection
Incidence
curves
100 20000
50 10000
0 0
log10(Effectiveness) −0.5 0.0 0.5
log10(Effectiveness)−0.05 0.00 0.05 0.10 0.15 0.20
viii viii
Intervention
vii vii
themes
vi vi
v v
iv iv
iii iii
ii ii
i i
an
an
eb
eb
ar
ar
pr
ay
eb
ay
un
an
eb
ar
ar
pr
pr
ay
ay
un
−M
−M
−A
−M
−M
−A
−A
−M
−M
−M
−M
−F
−F
−F
−F
−J
−J
−J
−J
−J
17
04
20
07
24
07
16
07
10
27
01
17
15
31
04
20
04
21
06
22
Figure 3. Intervention effectiveness for four prototypical countries. The upper panels of subfigures show the
incidence curves of confirmation (red) and infection (black) in (A) New Zealand, (B) Italy, (C) Thailand and
(m)
(D) United States of America. The lower panel of each subfigure shows the intervention ηn (in rectangles) and
aggregated η(m) (in circles) effectiveness of each intervention theme. The eight themes are (i) Case identification,
contact tracing and related measures, (ii) Environmental measures, (iii) Healthcare and public health capacity, (iv)
Resource allocation, (v) Risk communication, (vi) Social distancing, (vii) Travel restriction, and (viii) Returning to
normal life. The blue (or red) color refers to the interventions that successfully reduced infections (or increment
of infections).
The corresponding aggregated effectiveness of risk communication was the highest with η(m=v) = 100.53. In
contrast, the theme (i) Case identification, contact tracing and related measures was the worst performing with
η(m=i) = 10−0.16.
Other country’s intervention effectiveness. We selected four prototypical countries: New Zealand,
Italy, Thailand and United States of America (Fig. 3). We found that all intervention portfolio implemented in
New Zealand were effective and thus incidence dropped gradually after reaching the peak at 18- and 19-Mar-
2020. Italy is one of the four countries implemented all eight intervention themes. All except (iv) Resource alloca-
tion were effectively reduced transmission. We also found that Thailand effectively implemented (v) Risk com-
munication interventions with η(m=v) = 100.64, which is the second highest ranked after Singapore (SI Fig. 52).
However, the mean value of portfolio effectiveness is extremely low (second lowest, see SI Fig. 54) mainly due to
a significantly increment of infections during the earlier stage with inefficient control of (vii) Travel restriction.
The US as the country with the highest case number was not able to control transmission or suppress incidence
as other three countries. There were only half of the implemented portfolios being effective, which was the third
lowest among all 50 countries. It demonstrated a negative relationship between the total confirmed cases and the
fraction of effective portfolios (Fig. 4). The fraction of effective portfolios is the number of positive intervention
sets divided by the total number of implemented portfolios.
The aggregated effectiveness of each intervention theme in 50 countries were estimated and shown in Fig. 5.
Similar to Japan, most countries implemented all intervention themes but (ii) Environmental measures and
Scientific Reports | (2021) 11:10605 | https://doi.org/10.1038/s41598-021-88309-1 5
Vol.:(0123456789)www.nature.com/scientificreports/
A log10(Effectiveness)
−1 0 1 2
Intervention viii
themes vii
vi
v
iv
iii
ii
i
France (FR)
Honduras (HN)
Sweden (SE)
Bosnia and Herzegovina (BA)
Albania (AL)
Austria (AT)
Belgium (BE)
Brazil (BR)
Canada (CA)
Croatia (HR)
Czechia (CZ)
Ecuador (EC)
Denmark (DK)
El Salvador (SV)
Estonia (EE)
Finland (FI)
Germany (DE)
Greece (GR)
Hungary (HU)
Iceland (IS)
India (IN)
Indonesia (ID)
Ireland (IE)
Italy (IT)
Japan (JP)
Kazakhstan (KZ)
Kosovo (XK)
Kuwait (KW)
Lithuania (LT)
Malaysia (MY)
Republic of Korea (KR)
Singapore (SG)
Mauritius (MU)
Mexico (MX)
Montenegro (ME)
Netherlands (NL)
New Zealand (NZ)
North Macedonia (MK)
Norway (NO)
Romania (RO)
Portugal (PT)
Serbia (RS)
Slovakia (SK)
Slovenia (SI)
Spain (ES)
Switzerland (CH)
Syrian Arab Republic (SY)
Thailand (TH)
The United Kingdom (GB)
United States of America (US)
B viii
vii
Intervention
vi
themes
v
iv
iii
ii
i
−1 0 1 2
log10(Effectiveness)
Figure 4. Total confirmed cases as a function of the fraction of effective portfolios. The fraction of effective
portfolios is defined as the number of positive intervention sets considering the total. 50 countries are shown in
red circles with their ISO codes. The black line and gray area represent the regression line and corresponding
confidence interval, respectively. Note that the total confirmed cases on y-axis is shown in log10-scale.
(viii) Returning to normal life were the least implemented. Only Czechia, Ecuador, Hungary and Italy were the
countries that explicitly implemented all 8 themes. Note that these findings are just based on officially reported
interventions and do not consider interventions already realized independently of COVID. We discarded the
earlier implementations in the Syrian Arab Republic and considered only one due to the uncertainty in estimating
reproduction numbers. Figure 5A shows that Ecuador implemented the most and least effective interventions
but this was caused by the sudden and suspicious incidence rise that is believed to be a surveillance malfunction.
All themes but (viii) Returning to normal life were effective interventions as manifested by the positive median
values (Fig. 5B). Specifically, the theme (v) Risk communication was the best intervention with the median of
η(m=v) = 100.10, while (vi) Social distancing and (iv) Resource allocation were the second and third best interven-
tions with the median at 100.06 and 100.05. In contrast, the theme (viii) Returning to normal life increased infection
with a median value of effectiveness at 10−0.05. Among all interventions, (ii) Environmental measures was the one
with the smallest variance. The geographic distribution of aggregated effectiveness of each intervention theme
can be found in SI Fig. 55.
Uncertainty and sensitivity. Surveillance uncertainties. In order to quantify potential surveillance fal-
lacies related to systematic and suspicious fluctuations (treated as noise), we performed a sensitivity analysis
and reconstructed the infection curves using an alternative setting, in which the uninformative initial condition
(baseline setting) in RL iterative deconvolution process is replaced by back-shifted confirmation curves. The
alternative infection curves are spiky with noise due to the direct usage of the surveillance data. SI Figs. 1–50
show full estimates obtained by two different settings. Taking Japan as an example, the estimates are similar to
the baseline setting. The incidence reached the peak on 2-Apr-2020 with 811 cases and the reproduction num-
ber dropped below one from the early April 2020. Generally for all countries, the effectiveness is sensitive to the
initial condition and this is an expected behavior of complex system processes that are highly dependent on the
starting point of analyses. The magnitude of effectiveness is larger than the baseline setting due to higher fluctua-
Scientific Reports | (2021) 11:10605 | https://doi.org/10.1038/s41598-021-88309-1 6
Vol:.(1234567890)www.nature.com/scientificreports/
US
106
BR
IN GB
ES IT
DE
FR
Total confirmed cases
MX
105 CA
BE
G
SG EC NL SE
KW
W PT CH ID
IE
RO
JP
J AT
A
KZ KKR
DKKR RS
104 OM
NOMY
CZ
FI
HN
HUU
TH SV MK GR
BA
R
HR
EE LT IS
SK SI N
NZ
3 XK AL
10
ME MU
SY
102
0.5 0.6 0.7 0.8 0.9 1.0
Fraction of effective portfolios
Figure 5. Aggregated portfolio effectiveness of intervention themes implemented in 50 selected countries. The
eight themes are (i) Case identification, contact tracing and related measures, (ii) Environmental measures, (iii)
Healthcare and public health capacity, (iv) Resource allocation, (v) Risk communication, (vi) Social distancing,
(vii) Travel restriction, and (viii) Returning to normal life. (A) The blue (or red) color represents the interventions
that successfully reduced infections (or increment of infections). (B) The boxes shows the distributions of the
aggregated effectiveness in log10-scale. The median values of the eight themes were (i) 0.05, (ii) 0.01, (iii) 0.03,
(iv) 0.05, (v) 0.10, (vi) 0.06, (vii) 0.03, and (viii) − 0.05, respectively.
tions in incidence curves and reproduction numbers. As the baseline scenario for Japan, the themes (v) Risk com-
munication and (i) Case identification, contact tracing and related measures were the best and worst interventions.
The aggregated effectiveness of each intervention theme given the alternative initial condition distributes
slightly different than the baseline scenario (SI Fig. 51). Nevertheless, the first and second ranked intervention
theme were (v) Risk communication and (vi) Social distancing. The theme (viii) Returning to normal life remained
not effective. However, (i) Case identification, contact tracing and related measures and (ii) Environmental meas-
ures surprisingly showed a negative effectiveness in the alternative scenario; a result that underlines how these
interventions may be erroneously evaluated in presence of strong surveillance uncertainties, also because their
portfolio effectiveness is not very high.
Discussion
In this study, we assessed the impact of various interventions implemented during the pandemic in 50 countries.
Interventions are grouped into eight themes and evaluated in terms of their effect on daily incidence reduction.
We employed the deconvolution method to reconstruct the incidence curve of infection, estimated the effective
reproduction number of each implementation using the EpiEstim package, and evaluated the portfolio, inter-
vention and aggregated effectiveness. In response of the destructive pandemic, all governments have taken as
many interventions as possible to slow down the spread of the COVID-19 infections. However, it is important
to design a set of interventions whose organization maximize their systemic effectiveness in reducing total cases.
This is by far a non-trivial question considering the many interventions that are feasible and country specific
underlying socio-environmental conditions. However, the important question is also about how effective each
intervention is overall independently of fine scale details.
Back-calculation of infection curves is important as it reveals the “true” incidence curves on which effective-
ness should be calculated. Deconvolution or back-calculation methods are extremely popular in many areas of
science (e.g. in rainfall-runoff processes for instance) where the interest is to reconstruct the primary forcing of an
effect and to attribute the variability of forcing to effects (i.e. interventions in our case). The calculation of effec-
tiveness can be done on confirmation curves; however, conceptually, interventions aim to decrease infections and
yet it is important to evaluate their impact on the primary population outcome without secondary effects related
to incidence reporting that constitute confirmation curves. There are many other factors affecting confirmation
and some of these factors have little to do with interventions. Additionally, confirmation curves contain a lot of
spurious noise that is removed when back-calculating (inferring) infection curves for evaluating effectiveness.
We stopped our back-calculation when we reached a feasible convergence defined by the simultaneous minimiza-
tion and maximization of the root-mean-square difference (RMSD between incidence curves) and its curvature,
respectively (see “Inference of incidence curves of infection with the Richardson-Lucy deconvolution” section).
Then, the back-calculated incidence curve is the curve that allows one to predict the confirmation curve with
the highest accuracy given the symptomatic-to-confirmation distribution as a transfer function. This infection
curve is univocally defined among many other suboptimal infection curves. Ricci curvature is proportional
to system?s entropy and this impacts predictability that is the highest when the curvature is at its m inimum33.
Precious studies showed that the maximum curvature corresponds to the highest information flow (in this case
Scientific Reports | (2021) 11:10605 | https://doi.org/10.1038/s41598-021-88309-1 7
Vol.:(0123456789)www.nature.com/scientificreports/
between curves) and this maximizes the portfolio effect for which the highest systemic risk is observed (in our
case study it is about infection growth). Yet, effectiveness can also be interpreted as the relative ratio between
curvatures to identify the most impacting interventions. The one with the highest curvature ratio is the one that
is the most effective. Curvature, in the broad sense, is a measure by which a geometrical object deviates from
being flat. In the context of networks, “flatness” reflects the connectivity and interdependence between distant
nodes (in this case can be between curves or interventions, while in a spatially explicit context it can be between
communities, populations or individuals).
Non‑pharmaceutical intervention effectiveness. Below we discuss about each intervention based on
their median values of aggregated effectiveness irrespective of country’s details as synthesized by Fig. 5B. Some
of these interventions target population or individual behavior, environmental exposure factors, health capacity
and monitoring, or other non-health system resources. Further details about each intervention theme in each
country can be found in20,21. Specifics of these interventions are provided in Supplementary Information with
explicit descriptions of actions and links to each implemented action.
We validated the very high influence of risk communication for case reduction in all 50 countries considered.
This is a quite striking result considering the wide differences in socio-environmental factors as well as interven-
tion portfolios in each country. This result emphasizes that information in general, enhanced by any risk com-
munication or other intervention implementation, is effectively linking all interventions directly or indirectly.
For example, the announcement of travel alerts and warnings is a risk communication intervention that has an
effect on human mobility which is categorized as “travel restriction”34. Risk communication, that can inform
about any other intervention such as the use of face masks (conceivable as an “environmental intervention”) or
provide broader COVID-19 information about potential transmission, is in general not enforcing any behavior
on individuals but having a large impact on populations. Risk communication is particularly effective in Japan,
Singapore and Thailand as empirically found from the media.
Social distancing is the second high scored intervention. Data show that mass gathering cancellation and
closure of educational institutions have been efficient to curb the spread of infection. Other studies analyzed
some specific effects of social distancing in reducing COVID-19 morbidity in the US and European setting35,36.
These results confirm the high effectiveness of this intervention aimed to control population behavior in a more
defined and strict sense than risk communication. Effectiveness of social distancing is positive across all coun-
tries, with the exception of Kuwait, with peaks for France, Malaysia and Switzerland.
The third ranked theme in terms of effectiveness is resource allocation to systems that are not related to the
health system (considering both public health surveillance and healthcare). Interventions within this theme are
in the form of individual aid and support to government organizations, such as: state aid, taxation and social
security (e.g. the COVID-19 aid given to individuals and businesses in Japan and USA); support to police and
army interventions to ensure the enforcement of individual restrictions such as mobility bans; and resources for
crisis management plans and business continuity plans, including cost payments associated to emergency law
formulation. Overall, resource allocation resulted quite effective for all countries. Brazil, Iceland and Mauritius
seem to score high in this intervention category.
Case identification, contact tracing and related measures is the fourth ranked intervention theme and the
first among specific public health intervention. Active contract tracing of infection clusters also contributed to
case identification. As a result, more infections are identified. For example, Japan did not adopt the suggestion of
massive individual tracing as suggested by W HO37, but performed targeted test to people with the most severe
conditions38,39. The highest incidence in the early stage of the epidemic was considered and individuals traced
after cluster identification. Likely, this lack of massive tracing implied that the effectiveness for this intervention
in Japan is negative while for other prototypical countries, such as USA and Italy, is positive. Case identification
and contact tracing effectiveness is also negative for European countries with elevated incidence such as France
and Germany, as well as Asian countries such as India and South Korea. The last one was initially praised as a
great example in contact tracing but a second wave hit the country and that reduced the effectiveness of tracing.
Travel restrictions show one of the highest fluctuations of effectiveness considering all interventions. In the
case of Japan, travel restrictions in the form of entry ban for people with travel history to/from China, at the
early stage of the pandemic, did not produce noticeable effect. One of the potential explanations could be about
the implementation timing of the entry ban when China already started a strict mobility restriction within the
country, including the complete lockdown of Wuhan, the epicenter of the pandemic. The later implementation
of the entry ban in late March for people from Europe and Americas had a positive effect in mitigating the mag-
nitude of the second infection wave (due to imported cases). Overall, for travel restriction in Japan the average
effectiveness is very similar to zero due to the initial negative and later positive effects. A similar situation is
experienced by many countries such as UK, USA, and Italy. Additionally, other countries show similar time-
varying effectiveness of Japan where the reduction of incidence instantaneously occurred upon implementations
but the aggregated effectiveness is not one of the highest. Thailand in contrast showed a negative effectiveness
likely because of the very mild total incidence on which this intervention was supposed to have an effect. These
results corroborate the idea that just travel restriction is not enough in abating drastically the transmission but
many other interventions are needed.
Healthcare and public health capacity include a set of interventions strongly focused on the healthcare sys-
tem and predominantly the following actions: increase in medical supplies and equipment; increase in health-
care workforce; increase of isolation and quarantine facilities; increase supply of personal protective equip-
ment (including face masks); enhanced laboratory testing capacity; increase patient capacity; and healthcare
research. Healthcare and public health capacity was expected to have played a vital role in reducing incidence,
but our results did show this intervention was as effective as other interventions. This results can actually be
Scientific Reports | (2021) 11:10605 | https://doi.org/10.1038/s41598-021-88309-1 8
Vol:.(1234567890)www.nature.com/scientificreports/
well explained considering a triplet of factors. The first one is about the fact that healthcare is predominantly
about post infection treatment and does not reduce infections in the population; thus, even in countries where
healthcare capacity is extremely high, such as in USA, effectiveness of this theme is low. The second factor is
related to the likely increase in risk of cross-infection in hospitals and other healthcare facilities, once suspected,
asymptomatic and and infected individuals are present in great numbers and get in contact with other people.
This may occur even when patient capacity increases. The last factor is about the portfolio effect for which the
effectiveness of one intervention is also a function of all others. Due to the high interdependence of all other
interventions, healthcare and public health capacity seems more independent and yet it exhibits a different trend
considering its effect on case reduction. Based on our results, Brazil, India, Malaysia, Portugal, South Korea and
Singapore are scoring the lowest for this theme effectiveness. Strangely, Ecuador has a very high effectiveness but
this seems related to the large reduction in cases, after a peak of about 10,000 or higher, where resource allocation
to healthcare and public health capacity was the last intervention implemented.
The last type of direct interventions are environmental measures in the form of actions aimed to reduce expo-
sure to Coronavirus. These actions include: approval of new biocidal products; and environmental cleaning and
disinfection occurred in public spaces such as public transports, public buildings (schools, hospitals, government
offices, etc.), and grocery stores. We observe that only 13 out of 50 countries implemented environmental meas-
ures. This is because many environmental measures are already adopted by different countries independently of
COVID, such as in J apan40, and because in reality many countries did not target exposure reduction as a valuable
intervention for reducing COVID-19 transmission. Many studies appeared in the scientific community about
Coronavirus lifetime on different surfaces41–43 but these results generated more risk communication44,45 than
acting directly on spreading media. Yet, part of information and effectiveness related to environmental measures
is contained in risk communication.
The last intervention theme considered is “returning to normal life”, that is relaxing or removing some of other
intervention themes adopted previously. Thus, partial lift of the curfew, quarantine, and landing bans; conditional
re-opening of activities (certain shops, cultural institutions, places of worship, touristic sites, and short-term
accommodations), conditional lift of restriction on gathering, special relief for movements, and reopening land
borders are are types of actions seen in the countries considered. Data related to returning to normal life actions
are not available for many countries due to the fact many countries are still under rigid measures for contain-
ing COVID-19 spreading. In some countries, like Slovakia and the Netherlands, the epidemic was believed
to be effectively suppressed because daily new incidence appeared in continuous declination. Therefore, local
governments loosened restriction policies to allow people to return to normal life. However, a second wave of
cases was recorded and this is the reason for which effectiveness of returning to normal life is negative for this
countries. To a certain extent secondary new cases are expected after relaxation of constraining interventions,
but any relaxation should be handled carefully due to the highly probability of resurgence of local outbreaks is
as shown in Ecuador, Norway, and more recently in USA and Japan. There is certainly the need to loose some of
intervention limitation but a complete return to normal life as it was without COVID-19 is almost impossible at
least in short term. Coronavirus is not going to disappear and it is circulating in the environment where millions
of people live. Yet, it is our belief that a portfolio approach is needed where COVID-19 becomes part of our life
and interdependent interventions aim to educate us toward behaviors that reduce exposure and transmission.
By comparing all interventions it is worth noticing that the most loose intervention in terms of enforcing
behavior on individuals, i.e. risk communication, is the most effective on populations. This is related to the fact
that risk communication is addressing many other interventions and educating population across the spectrum
of elements of risk, i.e. hazards, exposures, vulnerability and controls. This also supports the belief that stringent
controls on populations have psychology-driven negative feedbacks of generating the opposite behavior of what
is enforced46. In other words, coercion seems to increase the number of d efectors47 and this may be one of the
many interpretations for describing the lower effectiveness of behavior-constraining interventions. In a modern
portfolio (Markowitz) perspective it is interesting to note that mean effectiveness of intervention scales with its
variance (SI Fig. 53). Thus, the most effective interventions has also the largest variability. By analyzing portfolios
for all countries it is possible to reconstruct an empirical and discrete Pareto frontier that maximizes and mini-
mizes mean and variance of effectiveness (SI Fig. 54). Countries along the Pareto frontier are theoretically the
ones with an optimal portfolio considering the occurred incidence; the further one country is from the Pareto
frontier the worst the collective portfolio and yet all intervention effectiveness. As for COVID-19, New Zealand
and Thailand are the most optimal countries, while USA and Brazil are the least optimal in terms of portfolio
effectiveness. Japan and Italy are instead suboptimal countries with respect to their total incidence and show a
vey diverse organization of interventions over time: Japan has one of the largest number of implemented inter-
vention but restricted to 6 themes only, and Italy has a more regular but delayed set of interventions considering
all feasible 8 themes. This emphasizes how organization of implemented interventions (i.e., portfolio over time)
is primarily important vs. the diversity of the portfolio. Previous portfolio applications have considered these
issues, also when facing multiple diseases s imultaneously8, that can be considered more practically for COVID-
19 emergency management.
It should be kept in mind that some of these interventions are not quite easily implementable rapidly dur-
ing a pandemic like with COVID. In particular, environmental interventions more broadly defined are related
to underlying socio-environmental features and that is establishing the vulnerability determining the initial or
baseline risk of a population40. Nonetheless, environmental interventions are important because they reduce
exposures dramatically but unfortunately not many countries implemented these interventions.
Comparison with other studies. Our paper is the closest to the approach of10 since we used the same
dataset and a conceptually similar model. In10 the authors interestingly tested four different statistical models on
Scientific Reports | (2021) 11:10605 | https://doi.org/10.1038/s41598-021-88309-1 9
Vol.:(0123456789)www.nature.com/scientificreports/
three datasets (among which one dataset is the one we used) and found a correlation between the entropy of the
distribution of NPI occurrences (in the normalized ranking of countries) and site-specificity of interventions:
high values of the normalized entropy signal that the performance of NPIs depends largely on the geographical
region considered. The effectiveness varies considerably across countries, and yet the predictability of interven-
tion effects is much smaller without knowledge of other specific factors, e.g. compliance, and other factors. These
high entropy interventions have typically small effectiveness. Conversely, NPIs have a “universal performance”
for entropy close to zero; this means that effectiveness is stable across countries independently of the specific
details, and that increases the predictability of intervention effects at the macroscale. This is for instance the
case of risk communication10 also implicitly recognized the idea of intervention portfolio by reconstructing a
network of co-implementation but without quantitatively assess the joint effectiveness of NPIs as in our work.
On the contrary of our fi ndings10 showed that the most effective measures include closing and restricting most
places where people gather in smaller or larger numbers for extended periods of time (businesses, bars, schools
and so on). Note that, however, these interventions are the byproduct of risk communication strategies. This
result (i.e., the most effective measures are closing and restricting access to gathering places) holds when NPIs
are accounted in isolation but not in a portfolio perspective that is when all other NPIs are considered simulta-
neously. Haug et al.10 also find a number of highly effective NPIs that can be considered less costly. For instance,
as much as in our study, they found that risk-communication strategies feature prominently amongst consen-
sus NPIs. This includes government actions intended to educate and actively communicate with the public,
including about social distancing interventions. Haug et al.10 considered a more specific analysis of interventions
but found similar conclusions. “Level 2” risk communication in order of effectiveness are: educate and actively
communicate with the public, travel alert and warning, and actively communicate with managers. Level 3 risk
communication in order of effectiveness are: call for return of nationals living abroad, promote social distanc-
ing measures, promote workplace safety measures and self-initiated isolation of people with mild respiratory
symptoms, encourage self-initiated quarantine, information about travels, and warning against travel to and
return from high risk. The simultaneous consideration of many distinct NPI themes allows us to move beyond
the simple evaluation of individual NPIs to assess; the collective choice of specific sequences of interventions
impacts the systemic effectiveness of the whole intervention portfolio and of each NPI. Additionally we propose
a method to rank countries based on the pair mean-variance effectiveness (Fig. 6)—to maximize and minimize,
respectively—that is a measure of absolute performance independently of total cases experienced by a country,
in relation to the distance from an optimal Pareto frontier. The (Pareto) optimal NPI frontier (i.e. the set of
optimal intervention portfolios which can be constructed empirically as in our case in Fig. 6) can inform about
interventions to undertake in other countries in order to maximize effectiveness locally and globally. This is
also our novelty compared to Haug et al.10. Overall our estimated effectiveness of risk communication is ∼ 23%
that seems much more realistic than the 48% predicted by Haug et al.10. We believe Haug et al.10 does not truly
capture the portfolio effect because it estimates interventions in isolation (by removing them one by one, that
creates time series of cases dependent on interactions of all other interventions only and that is dynamically
dependent on time) leading to overestimation of intervention effectiveness. Other papers such as Wu et al.48
used a data-driven mechanistic Bayesian model to quantify the effect of NPIs based on observed US fatalities;
stay-at-home orders and advisory were estimated as the only effective NPI in metropolitan and urban counties,
confirming the importance of risk communication. Other studies were just focused on one NPI or c ountry49–53,
thus it is hard to draw comparisons.
Assumptions and potential limitations. One caveat of the study is that effectiveness is strictly a predict-
able change in reproduction number over time than “true effectiveness” whether that is truly measurable. For
instance, surveillance exhibits a negative effectiveness (that is, an increase) on Rt , presumably related to the fact
that more testing allows for more cases to be identified. Obviously this is a good thing because more knowledge
of events leads to better decisions but numerically this corresponds to a positive change in Rt , yet to a negative
effectiveness. This also happens in10, thus a lot of care is needed when interpreting results and as with any model
true causality and predictability do not necessarily coincide54. Further sensitivity analysis proved that results
(patterns of effectiveness) are minimally sensitive to distributions of symptomatic-to-confirmation periods. This
low sensitivity to epidemiological infection-to-confirmation distribution also underlines the dominance of few
strong NPIs in the intervention portfolio and the second-order importance of all other NPIs. The atemporal
null hypothesis would be that no heterogeneous patterns of effectiveness emerge (for any country and among
countries) without the diverse effect of interventions. Another source of uncertainty coming from data and the
model is related to the simultaneous co-occurrence of interventions, for which nobody can precisely attribute
the relative effectiveness of each NPI theme for the same time t. This is also considering that NPI portfolio effec-
tiveness at time t is dependent on time t − δ where δ is temporal gap into the past; in other words, contingency
matters but we verified how close the overlapping Rt (when δ is considered) is to the non-overlapping Rt . For
this complexity inexplicability we decided to adopt a minimalist pattern-oriented model that divides equally the
effectiveness of interventions for those co-occurring at the same time step. The temporal null hypothesis in this
case is that if all interventions are equal then no heterogeneous patterns of effectiveness would arise at any time
step and for the whole period. Any assumption (extra factor, e.g. compliance that is already embedded into the
signature of infections and confirmations) on the attribution of effectiveness would just bias the inferred effec-
tiveness because: (i) it is extremely hard to measure specific factors especially at the beginning of the epidemic;
and (ii) these factors are highly country dependent and would involve the use of other more uncertain datasets.
All extra factors affecting effectiveness as the predicted macroscopic variable are already embedded into the
data from which we want to derive effectiveness patterns. These null-hypotheses are equivalent to the “what if??
scenarios?? of10 where the authors, rather than attributing effectiveness equally to co-occurring interventions,
Scientific Reports | (2021) 11:10605 | https://doi.org/10.1038/s41598-021-88309-1 10
Vol:.(1234567890)www.nature.com/scientificreports/
A
iii
Variance of aggregated effectiveness
101
100 v
iv
v
vi
vii
i
10−1
viii
iii
10−2
ii
10−3
10−0.04 10−0.02 100 100.02 100.04 100.06 100.08 100.1 100.12
Mean of aggregated effectiveness
EC
B
100
Variance of portfolio effectiveness
TH BR
DE
−1 KR IS MY
10 SG
KW US
IN ES MU
JP KZ
FR AT
DK FI
SV
XK MX BE
SK NOHN HR
RS ME
GB PT BA
AL CH
10−2 MK
ID LT
NZ
IT EE IE
CZSE GR
NL HU
CA
SI
RO
10−3
100 100.01 100.02 100.03 100.04 100.05 100.06
Mean of portfolio effectiveness
Figure 6. Mean and variance of portfolio effectiveness for each NPI and for countries. (A) Mean and variance
of aggregated effectiveness across time and countries; 8 themes are shown in red circles. The black line and gray
area represent the regression line and corresponding confidence interval, respectively. Note that both axes are
shown in log10-scale. (B) Mean and variance of portfolio effectiveness. 50 countries are shown in red circles
with their ISO codes. The gray dashed line represents an empirical Pareto frontier where variance and mean of
portfolio effectiveness is the risk level and the expected return, respectively. Note that both axes are shown in
log10-scale.
performed predictions by creating artificial sequences of NPIs (by selectively deleting one NPI at a time from
all sequences, whose effectiveness to be estimated) in each country and comparing them to the actual sequence.
However, the removal of one intervention does not remove the effects (due to interactions) that intervention has
on others. Systemic effectiveness must consider the contingency determined by time-dependent co-occurrence
of events (infections) and interventions. After removal of one intervention any future prediction is just specu-
lative because that removal affects all others? effectiveness; yet, we believe portfolio approaches are better to
capture estimates of non-linear effects.
Conclusions
The proposed probabilistic model demonstrates the systemic impact of intervention portfolios from which dis-
tinct NPI effectiveness is inferred without a-priori assumption on NPI interdependencies. The model is uniquely
applied to 50 countries to test intervention effectiveness defined as the impact on total case growth. The following
critical findings are worth mentioning:
• A deconvolution model is proposed to reconstruct expected cases of infection from raw observations of con-
firmation considering surveillance fallacies (delays and potential under-reporting). The initial condition of
the iterative procedure is established by a uniform infection curve that is flat and unbiased (i.e., a Maximum
Scientific Reports | (2021) 11:10605 | https://doi.org/10.1038/s41598-021-88309-1 11
Vol.:(0123456789)You can also read
