Evidence of bias in the Eurovision song contest: modelling the votes using Bayesian hierarchical models
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Evidence of bias in the Eurovision song contest:
modelling the votes using Bayesian hierarchical models
Gianluca Baio
University College London
Department of Statistical Science
gianluca@stats.ucl.ac.uk
(Joint work with Marta Blangiardo, Imperial College London)
2014 International Conference of the Royal Statistical Society
Sheffield, 3 September 2014
G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 1 / 15Set your priorities straight... G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 2 / 15
Eurovision & statistical modelling
• The Eurovision song contest (ESC) is an annual musical competition held
among active members of the European Broadcasting Union
• Since 1962, based on positional voting
– Several iterations until current system: S = {12, 10, 8, 7, 6, 5, 4, 3, 2, 1, 0}
points allocated to each act
– Tele-voting established in 1998 — current system a mixture of tele-voting and
“expert” juries
G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 3 / 15Eurovision & statistical modelling
• The Eurovision song contest (ESC) is an annual musical competition held
among active members of the European Broadcasting Union
• Since 1962, based on positional voting
– Several iterations until current system: S = {12, 10, 8, 7, 6, 5, 4, 3, 2, 1, 0}
points allocated to each act
– Tele-voting established in 1998 — current system a mixture of tele-voting and
“expert” juries
• Especially with tele-voting, accusations of bias in the voting system brought
forward by several commentators
– Famously, Sir Terry Wogan in 2008 quit as the BBC commentator
– Before the last edition, UKIP’s leader Nigel Farage said in a radio interview
that he “absolutely hate” the ESC and thought Britain would never win
because of European prejudice
G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 3 / 15Eurovision & statistical modelling
• The Eurovision song contest (ESC) is an annual musical competition held
among active members of the European Broadcasting Union
• Since 1962, based on positional voting
– Several iterations until current system: S = {12, 10, 8, 7, 6, 5, 4, 3, 2, 1, 0}
points allocated to each act
– Tele-voting established in 1998 — current system a mixture of tele-voting and
“expert” juries
• Especially with tele-voting, accusations of bias in the voting system brought
forward by several commentators
– Famously, Sir Terry Wogan in 2008 quit as the BBC commentator
– Before the last edition, UKIP’s leader Nigel Farage said in a radio interview
that he “absolutely hate” the ESC and thought Britain would never win
because of European prejudice
• Surprisingly (or not?), there is a relatively large literature on statistical
modelling of the ESC voting patterns
– Broadly speaking, clustering to detect “bloc” or “tactical” voting
– All in all, evidence seems to suggest specific voting patterns
– But is this proof of bias? Favouritism or discrimination?
G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 3 / 15Data
• Data on tele-voting available from the ESC website (www.eurovision.tv)
– We consider the period 1998-2012 and all countries that have voted in the
final round, in this period
– yvpt = Number of points from voter v to performer p on occasion (year) t
G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 4 / 15Data
• Data on tele-voting available from the ESC website (www.eurovision.tv)
– We consider the period 1998-2012 and all countries that have voted in the
final round, in this period
– yvpt = Number of points from voter v to performer p on occasion (year) t
• Covariates
– x∗1t = Year of the contest (current year−1998; accounts for “external factors”)
– x2pt = Song language (English, own language, mixture)
– x3pt = Gender & type of performance (group, solo male, solo female)
G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 4 / 15Data
• Data on tele-voting available from the ESC website (www.eurovision.tv)
– We consider the period 1998-2012 and all countries that have voted in the
final round, in this period
– yvpt = Number of points from voter v to performer p on occasion (year) t
• Covariates
– x∗1t = Year of the contest (current year−1998; accounts for “external factors”)
– x2pt = Song language (English, own language, mixture)
– x3pt = Gender & type of performance (group, solo male, solo female)
• NB: We are not particularly interested in the “effect” of these covariates on
the scores
– Our focus is not on predicting the actual votes for next instance of the
contest, given the covariates
– Rather, we use them to balance the data and account for potentially different
baseline characteristics
G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 4 / 15Bayesian hierarchical model
• Model: yvpt ∼ Categorical(πvpt ) [v = 1, . . . , 48, p = 1, . . . , 43, t = 1, . . . , Tvp ]
– πvpt = (πvpt1 , . . . , πvptS ) [S = 11 = number of elements of S]
– πvpts = Pr(yvpt = s) = Pr(v scores p exactly s votes in year t) for s ∈ S
G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 5 / 15Bayesian hierarchical model
• Model: yvpt ∼ Categorical(πvpt ) [v = 1, . . . , 48, p = 1, . . . , 43, t = 1, . . . , Tvp ]
– πvpt = (πvpt1 , . . . , πvptS ) [S = 11 = number of elements of S]
– πvpts = Pr(yvpt = s) = Pr(v scores p exactly s votes in year t) for s ∈ S
• Model the cumulative probabilities: ηvpts = Pr(yvpt ≤ s) = logit−1 (λs − µvpt )
– λ = (λ1 , . . . , λS ) set of random cut-off points: λs ∼ Normal(0, h2 ) + ordering
constraint so that λ1 ≤ λ2 ≤ . . . ≤ λS
– µvpt = linear predictor
G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 5 / 15Bayesian hierarchical model
• Model: yvpt ∼ Categorical(πvpt ) [v = 1, . . . , 48, p = 1, . . . , 43, t = 1, . . . , Tvp ]
– πvpt = (πvpt1 , . . . , πvptS ) [S = 11 = number of elements of S]
– πvpts = Pr(yvpt = s) = Pr(v scores p exactly s votes in year t) for s ∈ S
• Model the cumulative probabilities: ηvpts = Pr(yvpt ≤ s) = logit−1 (λs − µvpt )
– λ = (λ1 , . . . , λS ) set of random cut-off points: λs ∼ Normal(0, h2 ) + ordering
constraint so that λ1 ≤ λ2 ≤ . . . ≤ λS
– µvpt = linear predictor
X
C2
(c)
X
C3
(c)
• Model: µvpt = β1 x∗1t + β2c x2pt + β3c x3t + αvp
c=2 c=2
iid
– β = (β1 , β22 , β23 , β32 , β33 ) ∼ Normal(0, q 2 ) — flat independent prior on the
covariates “effects”
– αvp ∼ Normal(θvp , σα2 ): main parameter in the analysis — represents a
structured effect, accounting for clustering at the voter-performer level
G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 5 / 15Bayesian hierarchical model (cont’d)
• Model the mean of the structured effect as
θvp = γ + ψwvp + φzvp I(zvp ) + δRv p
2
– γ ∼ Normal(0, q ) = overall intercept
– wvp = 1 if countries v and p share a geographic border and 0 otherwise
⇒ ψ ∼ Normal(0, q 2 ) = “geographic” effect
– zvp = estimate of migration intensity from country v to country p
⇒ φ ∼ Normal(0, q 2 ) = “migration” effect
G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 6 / 15Bayesian hierarchical model (cont’d)
• Model the mean of the structured effect as
θvp = γ + ψwvp + φzvp I(zvp ) + δRv p
2
– γ ∼ Normal(0, q ) = overall intercept
– wvp = 1 if countries v and p share a geographic border and 0 otherwise
⇒ ψ ∼ Normal(0, q 2 ) = “geographic” effect
– zvp = estimate of migration intensity from country v to country p
⇒ φ ∼ Normal(0, q 2 ) = “migration” effect
• Assume that voters implicitly cluster in K (fixed number of) “regions”
– Accounts for similarities in voters’ propensity towards p, over and above
geographic and migratory aspects
– Rv ∼ Categorical(ζ), where ζ = (ζ1 , . . . , ζK ) ∼ flat Dirichlet = vector of
probabilities for clusters membership
– δkp ∼ Normal(0, σδ2 ) are set of structured common residual for each
combination of macro-area and p, which describe the “cultural” effect
G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 6 / 15Bayesian hierarchical model (cont’d)
• Model the mean of the structured effect as
θvp = γ + ψwvp + φzvp I(zvp ) + δRv p
2
– γ ∼ Normal(0, q ) = overall intercept
– wvp = 1 if countries v and p share a geographic border and 0 otherwise
⇒ ψ ∼ Normal(0, q 2 ) = “geographic” effect
– zvp = estimate of migration intensity from country v to country p
⇒ φ ∼ Normal(0, q 2 ) = “migration” effect
• Assume that voters implicitly cluster in K (fixed number of) “regions”
– Accounts for similarities in voters’ propensity towards p, over and above
geographic and migratory aspects
– Rv ∼ Categorical(ζ), where ζ = (ζ1 , . . . , ζK ) ∼ flat Dirichlet = vector of
probabilities for clusters membership
– δkp ∼ Normal(0, σδ2 ) are set of structured common residual for each
combination of macro-area and p, which describe the “cultural” effect
• Independent vague priors on the log standard deviation scale
iid
– log(σα ), log(σδ ) ∼ Uniform(−3, 3)
G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 6 / 15Bayesian hierarchical model (cont’d)
• For voters v1 and v2 and performer p, αv1 p and αv2 p determine ηvpts , all
other covariates being equal
– αv1 p > αv2 p ⇒ the chance that v1 scores p more than s points is greater than
the chance that v2 will, for any possible score s
G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 7 / 15Bayesian hierarchical model (cont’d)
• For voters v1 and v2 and performer p, αv1 p and αv2 p determine ηvpts , all
other covariates being equal
– αv1 p > αv2 p ⇒ the chance that v1 scores p more than s points is greater than
the chance that v2 will, for any possible score s
• In this sense, can use αvp to quantify the presence of “favouritism” or
“discrimination”
– αvp > 0 ⇒ systematic pattern in which v scores p higher votes than other
voters
G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 7 / 15Bayesian hierarchical model (cont’d)
• For voters v1 and v2 and performer p, αv1 p and αv2 p determine ηvpts , all
other covariates being equal
– αv1 p > αv2 p ⇒ the chance that v1 scores p more than s points is greater than
the chance that v2 will, for any possible score s
• In this sense, can use αvp to quantify the presence of “favouritism” or
“discrimination”
– αvp > 0 ⇒ systematic pattern in which v scores p higher votes than other
voters
• NB: Difficult to give this a proper “causal” interpretation
– Cannot establish deliberate intervention from the available data
– Nevertheless, can interpret αvp as at least indicative of the underlying voting
patterns
G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 7 / 15Results — clustering
Posterior probability of membership in ’region 1’ Posterior probability of membership in ’region 2’
(0.9,1]
(0.9,1] (0.6,0.7]
(0.5,0.6] (0.5,0.6]
(0.4,0.5] (0.4,0.5]
(0.3,0.4] (0.2,0.3]
[0,0.1] (0.1,0.2]
[0,0.1]
G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 8 / 15Results — clustering
Posterior probability of membership in ’region 3’ Posterior probability of membership in ’region 4’
(0.5,0.6]
(0.4,0.5] (0.4,0.5]
(0.3,0.4] (0.3,0.4]
(0.2,0.3] (0.2,0.3]
[0,0.1] (0.1,0.2]
[0,0.1]
G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 9 / 15Results — structured effects
αvp − ᾱ
• Compute “standardised” effects: α∗vp = ≈ Normal(0, 1)
sα
– α∗vp > 1.96 ⇒ “substantial” positive bias (“favouritism”) from v to p
– α∗vp < −1.96 ⇒ “substantial” negative bias (“discrimination”) from v to p
1
Probability of ’’favouritism’’
0.5
0
Probability of ’’discrimination’’
0.5
1
0 500 1000 1500 2000
Voter−Performer combination
G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 10 / 15Results (selected performers)
Propensity to vote for Sweden Propensity to vote for Greece
−3 −2 −1 0 1 2 3 4 −3 −2 −1 0 1 2 3 4
DNK CYP
NOR ALB
FIN BGR
EST ROU
ISL DEU
GBR ARM
MLT BEL
IRE SRB
SVK SMR
ESP SCG
LVA GBR
MCO NLD
HUN GEO
LTU AZE
ISR CZE
BIH HUN
AND MKD
POL RUS
BEL TUR
TUR MDA
SRB BIH
FRA ITA
AUT FRA
SMR ESP
PRT ISR
SCG BLR
MKD CHE
MDA MLT
AZE SWE
GEO UKR
ROU AND
SVN MNE
BLR HRV
NLD MCO
ALB POL
CYP ISL
ARM SVN
DEU AUT
MNE DNK
UKR PRT
ITA FIN
HRV IRE
CHE SVK
RUS LTU
GRC NOR
CZE EST
BGR LVA
G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 11 / 15Results (selected performers)
Propensity to vote for Turkey Propensity to vote for Albania
−3 −2 −1 0 1 2 3 4 −3 −2 −1 0 1 2 3 4
DEU MKD
AZE SCG
FRA MNE
NLD CHE
BEL GRC
ALB HRV
BGR ITA
ROU TUR
MKD AUT
AUT BIH
CHE SRB
BIH SVN
SMR DEU
ITA SMR
GEO BEL
DNK ROU
GBR FRA
HRV SWE
NOR HUN
GRC AZE
SRB MCO
SCG MLT
SWE SVK
MNE DNK
MCO GBR
ARM GEO
HUN NLD
FIN FIN
ESP ISL
UKR BGR
ISR EST
BLR LVA
SVN NOR
MLT IRE
SVK POL
CYP ESP
CZE PRT
AND CYP
RUS AND
LTU CZE
PRT MDA
ISL BLR
POL RUS
MDA ISR
IRE UKR
LVA LTU
EST ARM
G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 12 / 15What’s up with the UK?
Propensity to vote for the UK Propensity to vote from the UK
−3 −2 −1 0 1 2 3 4 −3 −2 −1 0 1 2 3 4
IRE IRE
MLT SWE
ITA ITA
LVA GRC
CYP
DNK
SMR
TUR MLT
DNK LVA
LTU EST
ISR TUR
GRC LTU
EST DEU
POL NOR
PRT
AUT
HRV
AZE BGR
MKD SCG
SVK ISL
ARM CYP
CZE UKR
NOR NLD
AND BEL
DEU RUS
HUN
ESP ISR
ALB AZE
NLD FRA
BLR HUN
FRA ESP
BGR FIN
SVN POL
FIN CHE
BEL
MDA
SRB
SWE ALB
GEO ROU
MDA SRB
ROU GEO
RUS BLR
CHE SVN
BIH BIH
AUT
PRT
SCG
MCO HRV
ISL SVK
UKR ARM
MNE MKD
G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 13 / 15Conclusions
• No voter seems to show “substantial” negative propensity towards the
UK acts
– In fact no evidence of negative bias is found for any of the performers
– No need to leave the EU, just yet!
G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 14 / 15Conclusions
• No voter seems to show “substantial” negative propensity towards the
UK acts
– In fact no evidence of negative bias is found for any of the performers
– No need to leave the EU, just yet!
• Weak evidence of positive bias
– Clusters of countries that systematically tend to score a performer highly
– Not a “game-changer” — the effects are usually not very large
G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 14 / 15Conclusions
• No voter seems to show “substantial” negative propensity towards the
UK acts
– In fact no evidence of negative bias is found for any of the performers
– No need to leave the EU, just yet!
• Weak evidence of positive bias
– Clusters of countries that systematically tend to score a performer highly
– Not a “game-changer” — the effects are usually not very large
• Migration stocks play a major role in determining the voting patterns
– Eg the Turkish act typically very popular among the German voters (ie: voters
in Germany)
G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 14 / 15Conclusions
• No voter seems to show “substantial” negative propensity towards the
UK acts
– In fact no evidence of negative bias is found for any of the performers
– No need to leave the EU, just yet!
• Weak evidence of positive bias
– Clusters of countries that systematically tend to score a performer highly
– Not a “game-changer” — the effects are usually not very large
• Migration stocks play a major role in determining the voting patterns
– Eg the Turkish act typically very popular among the German voters (ie: voters
in Germany)
• Unmeasured important factors?
– Media coverage: in the days prior to the final, one entry is usually suggested
as the strong favourite
– May be based on objective qualities of the act, but also hangs on political
reasons, eg the willingness to take up the expensive organisation of the next
edition
G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 14 / 15Thank you! G Baio & M Blangiardo ( UCL & ICL) Evidence of bias in the Eurovision song contest RSS2014, Sheffield, 3 Sep 2014 15 / 15
You can also read
