Measuring Societal Biases from Text Corpora with Smoothed First-Order Co-occurrence
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Measuring Societal Biases from Text Corpora with Smoothed First-Order
Co-occurrence
Navid Rekabsaz,1 Robert West,2 James Henderson,3 Allan Hanbury4
1
Johannes Kepler University, Linz Institute of Technology, AI Lab,
2
EPFL,
3
Idiap Research Institute,
4
TU Wien, Complexity Science Hub Vienna
navid.rekabsaz@jku.at, robert.west@epfl.ch, james.henderson@idiap.ch, allan.hanbury@tuwien.ac.at
arXiv:1812.10424v4 [cs.CL] 27 Apr 2021
Abstract
mi
Fil
wo
ma
dw
Text corpora are widely used resources for measuring societal
ipi
ma
tro
gir
na
ife
biases and stereotypes. The common approach to measuring
n
n
l
such biases using a corpus is by calculating the similarities nurse ... ... ...
between the embedding vector of a word (like nurse) and the
vectors of the representative words of the concepts of inter- cosine
est (such as genders). In this study, we show that, depending
on what one aims to quantify as bias, this commonly-used she ... ... ...
approach can introduce non-relevant concepts into bias mea-
surement. We propose an alternative approach to bias mea-
surement utilizing the smoothed first-order co-occurrence
Figure 1: Word vectors with explicit dimensions. Estimating
relations between the word and the representative concept bias with similarity between the vectors also counts some
words, which we derive by reconstructing the co-occurrence (arguably) irrelevant context-words to the female concept
estimates inherent in word embedding models. We compare (red dashed lines). The example is based on the results in
these approaches by conducting several experiments on the Table 2.
scenario of measuring gender bias of occupational words, ac-
cording to an English Wikipedia corpus. Our experiments
show higher correlations of the measured gender bias with
the actual gender bias statistics of the U.S. job market – on els trained on text corpora as in preceding studies (Lenton,
two collections and with a variety of word embedding mod- Sedikides, and Bruder 2009; Hoyle et al. 2019; Zhou et al.
els – using the first-order approach in comparison with the 2019; Chang and McKeown 2019; Zhao et al. 2019; Garg
vector similarity-based approaches. The first-order approach et al. 2018; Caliskan, Bryson, and Narayanan 2017; Boluk-
also suggests a more severe bias towards female in a few spe- basi et al. 2016). In these studies, the associations of words
cific occupations than the other approaches. to concepts are measured based on some form of vector sim-
ilarity, for instance by using the cosine metric. The present
Introduction study sheds light on and discusses what is captured as bias
by these vector similarity-based approaches, and proposes
Text data has been widely utilized for studying and moni- a complementary bias measurement approach based on a
toring societal phenomena – such as biases and stereotypes smoothed variant of direct (first-order) co-occurrences. Let
– commonly by exploiting co-occurrence statistics of words us first have a closer look at what is measured by vector
in text. In these approaches, a societal bias construct (an un- similarity or more generally by similarity metrics applied to
observable abstraction that we aim to characterize) is quan- distributional representations.
tified using measures of words association. A word such as
In distributional representations, two vectors are more
nurse is considered to be stereotypically biased towards the
similar if the corresponding words both frequently co-occur
female concept, when a significant imbalance is observed
with a set of context-words (second-order co-occurrence).
between the associations of nurse to female versus male con-
Figure 1 elaborates this using a toy example. In the exam-
cept. Each of the concepts is commonly defined by a group
ple, the association of the word nurse to the female con-
of words, referred to as concept-representative words. The
cept, represented by the word she, is calculated using co-
focus of the present work is on the computational methods
sine similarity between their vectors. The word vectors in
for measuring biases from text corpora – a particularly es-
the example are high-dimensional representations, such that
sential component in various social studies.
each dimension of a vector is defined explicitly with a spe-
The common approach to calculate words associations
cific context-word, and each value of the vector represents
for bias measurement is by adopting word embedding mod-
the first-order co-occurrence relation between the word and
Copyright © 2021, Association for the Advancement of Artificial the corresponding context-word. We refer to such vectors as
Intelligence (www.aaai.org). All rights reserved. explicit representations. As shown, nurse and she are simi-lar since they co-occur with several common context-words, The proposed explicit variants exploit the word and con-
depicted with the blue circles in both vectors. text embedding vectors of word2vec and GloVe to estimate
Let us assume that in this example our objective of gen- forms of the co-occurrence relations. Such co-occurrence re-
der bias measurement can be formulated as “to quantify the lations, achieved from the reconstruction of explicit vectors
extent to which nurse is perceived as female versus male”. from low-dimensional embeddings, provide smoothed vari-
Considering this objective, we observe that the nurse-to- ants of the count-based co-occurrence estimations.
she similarity is influenced by the common context-words We use the discussed word representations, trained on
such as woman and girl, which are typically considered as an English Wikipedia corpus to study the characteristics of
good representatives of the female concept (Garg et al. 2018; the first- and high-order bias measurement approaches. We
Bolukbasi et al. 2016). However, this similarity is also af- conduct several experiments on the gender bias of occupa-
fected by several other context-words, such as midwife and tions. We first revisit the experiments conducted in previous
matron. According to the literal definitions of these words studies (Garg et al. 2018; Caliskan, Bryson, and Narayanan
(as defined in a dictionary), midwife is gender-neutral, and 2017) on the correlations of the gender bias of some occu-
matron is a mixture of the female concept and the concept pations, measured using the discussed methods, to the actual
of “being in charge of medical arrangements”. Considering statistics of gender bias in the U.S. job market. We use two
this, one can argue that such common context-words can in- collections, provided by Zhao et al. (2018a) and Garg et al.
troduce partially or completely irrelevant concepts to female (2018). We observe that, in all studied word representation
into the measured association, and hence into the calculated models and the two collections, the results of our proposed
gender bias. first-order bias measurement shows higher correlations in
While we discuss this issue on explicit representations, comparison with the high-order approaches.
it is also present in low-dimensional embeddings, although Next, we analyze the measured gender bias of around 500
such an explicit dividing of dimensions into groups is not occupations using first- and high-order approaches, observ-
per se possible. Another difference between explicit repre- ing several cases of the influence of non-relevant context-
sentations and embeddings is that, since word embeddings words on the results of the high-order bias measurement
are defined in low dimensions, the similarity of embedding method. Overall, our results suggest the existence of a more
vectors does not only capture second-order co-occurrences, severe degree of bias towards female in the underlying cor-
but other orders of co-occurrence, such as first-order as well pus, previously undetected by high-order approaches.
as higher orders.1 We therefore refer to the approaches that Finally, we study how each bias measurement approach
use word embeddings for quantifying bias as high-order bias reacts to (hypothetical) changes in the corpus, particularly
measurements. when the corpus moves towards a more balanced repre-
We approach the discussed issue in high-order bias mea- sentation of genders. To this end, we manipulate the cor-
surement by revisiting the utilization of first-order co- pus using the Counterfactual Data Augmentation (CDA)
occurrences for measuring bias. Our proposed approach, re- method (Zhao et al. 2018a; Lu et al. 2018; Zmigrod et al.
ferred to as first-order bias measurement, estimates the as- 2019), such that the genders are represented in the aug-
sociation of a concept and a word by averaging the first- mented corpora in a more balanced way. Our observations
order co-occurrences between the word and the concept- show that, while both bias measurement methods report a
representative context-words. The first-order bias measure- decrease of gender bias, the first-order bias measurement is
ment introduces an alternative to high-order approach, and more sensitive (reacts faster) to the changes in the corpus.
has the advantage of only taking into account the context- Limitations of the study. Gender is treated in this study as
words which are strongly related to the concept of interest. a binary construct, and the definition of gender bias is lim-
First-order co-occurrence of words has been widely used ited to the disparity between female and male. We acknowl-
to calculate societal phenomena particularly through count- edge that this choice neglects the broad meaning of gender,
ing and weighting words (Monroe, Colaresi, and Quinn but the decision is necessary for taking practical steps. Our
2008; Kirchler 1992; Rekabsaz et al. 2017). Among vari- study is also limited to the English language. Our introduced
ous metrics, the ones based on Pointwise Mutual Informa- method is however generic and can be applied to other lan-
tion (PMI) (Church and Hanks 1990) are commonly used guages as well as other forms of societal biases such as re-
to measure words co-occurrence in local contexts. As men- lated to race, age and ethnicity.
tioned by Lenton, Sedikides, and Bruder (2009), a draw back Outline of the paper. We first discussed related work, fol-
of such count-based co-occurrence metrics is the high spar- lowed by the relevant previous methods. Our bias measure-
sity of their resulting vectors, as many related words never ment approach is introduced next. Finally, the gender bias
appear in the same local context. experiments are described, and whose results are presented.
We address the issue of sparsity in count-based metrics by
proposing two novel explicit representations, created from Related Work
pre-trained word2vec Skip-Gram (Mikolov et al. 2013), and
GloVe (Pennington, Socher, and Manning 2014) models. Various aspects of word embeddings and distributional rep-
resentations in areas such as social sciences and psychology
1
For instance, Kontostathis and Pottenger (2006) found that La- are studied in previous work. Lenton, Sedikides, and Bruder
tent Semantic Analysis (Deerwester et al. 1990) can take into ac- (2009) discuss the use of Latent Semantic Analysis (Deer-
count up to fifth-order co-occurrences. wester et al. 1990) for measuring gender bias, highlightingthe sparsity issue of the first-order method based on count- High-Order Bias Measurements
based metrics. The present work complements this study by We define bias as the discrepancy between the associations
exploring the benefits of a smoothed first-order bias mea- of a concept and its counterpart concept to a word. High-
surement approach as an alternative to previous approaches. order bias measurement methods use vector similarity to
In a recent study, Günther, Rinaldi, and Marelli (2019) dis- quantify the associations. We define the concept Z (and
cuss the applications and common misconceptions of distri- similarly its counterpart concept Z 0 ) as a set, containing a
butional semantic models in psychology. group of representative words. In general, three approaches
to high-order bias measurement are proposed in the litera-
Several pieces of work exploit word embeddings to study
ture, explained in what follows.
societal aspects. Garg et al. (2018) investigate the changes in
D IRECTIONAL: In this method, first a matrix of
gender- and race-related stereotypes over decades using his-
directional vectors D is created using a set of word
torical text data. Caliskan, Bryson, and Narayanan (2017)
pairs PZ,Z 0 = {(x, x0 )|x ∈ Z, x0 ∈ Z 0 }, such that
and more recently Chang and McKeown (2019) study the
D = {vx − vx0 |(x, x0 ) ∈ PZ,Z 0 }, where vx is the vec-
patterns of language use, indicating accurate imprints of his-
tor of the word x. Using D, the first principle component of
torical biases. Bolukbasi et al. (2016) show the reflection
the directional vectors is then calculated, which we refer to
of gender stereotypes in word analogies derived from word
as vd .
embeddings. Zhou et al. (2019) propose methods to mea-
Bolukbasi et al. (2016) define the bias of the word w using
sure societal biases in languages with grammatical gender.
the cosine similarity of vw and vd . Ethayarajh, Duvenaud,
Our work directly contributes to these studies by proposing
and Hirst (2019b) propose to normalize only vd to avoid the
a more accurate approach for measuring bias in corpora. In
overestimation of the association degree, resulting in the fol-
this line, Hoyle et al. (2019) propose a method to measure
lowing bias measurement:
the differences between descriptions of men and women. In
contrast to our bag-of-words-based method, they measure vd vw
ψ(w) = (1)
bias using a parsed corpus. kvd k
Gender bias is also studied in various downstream tasks, where ψ(w) denotes the degree of bias of the word w, and its
such as sentiment analysis (Kiritchenko and Mohammad sign defines whether the word is biased towards the concept
2018), visual semantic role labeling (Zhao et al. 2017), Z or Z 0 . This definition is applicable to all the bias measure-
coreference resolution (Zhao et al. 2018a; Rudinger et al. ments, discussed through the paper.
2018), information retrieval (Rekabsaz and Schedl 2020) C ENTROID: This method, used in several studies such
text, and classification (Dixon et al. 2018; Barrett et al. 2019; as Garg et al. (2018) and Dev and Phillips (2019), first de-
Elazar and Goldberg 2018; De-Arteaga et al. 2019), as well fines the representative vector vZ as the mean of the embed-
as in language generation models (Sheng et al. 2019). dings of the representative words:
X vx
Mitigating the existence of societal biases in data and vZ = (2)
|Z|
models has been the topic of several studies. Some pieces x∈Z
of work propose the debiasing of word embeddings by iden- The association of the concept Z to the word w is then de-
tifying and removing gender subspace (Ethayarajh, Duve- fined using the cosine metric of vZ and vw . Finally, the bias
naud, and Hirst 2019b; Bolukbasi et al. 2016; Kaneko and is calculated as follows:
Bollegala 2019; Zhao et al. 2018b). As pointed out by Gonen
and Goldberg (2019), these methods successfully remove ψ(w) = cosine(vZ , vw ) − cosine(vZ 0 , vw ) (3)
the explicit bias, while the implicit bias, examined through
the ability of a classifier or a clustering algorithm to retrieve AVERAGE H IGH : Caliskan, Bryson, and Narayanan (2017)
the gender of vectors after debiasing, still remains. Recently, introduce Word Embedding Association Test (WEAT), a sta-
Lauscher et al. (2020) propose a general framework for mit- tistical test to examine the existence of bias using vector
igating both forms of bias. similarity. Ethayarajh, Duvenaud, and Hirst (2019b) criti-
cizes WEAT, showing that the conclusion of the test can be
Another approach to address bias, and related to this pa- manipulated by swapping gender-related concept words. We
per, is bias reduction in corpus via data augmentation. Coun- therefore only study the method used by Caliskan, Bryson,
terfactual Data Augmentation (CDA) is a common method, and Narayanan (2017) to measure the associations of words
which, in its basic form, extends a corpus by adding new to concepts. This method calculates the average of the co-
sentences, achieved from swapping the indicative words of sine similarities between the vector of the target word and
the concept of interest in the corpus. Zhao et al. (2018a) the vectors of the concept words. We refer to this method as
use CDA in the context of coreference resolution, Lu et al. AVERAGE H IGH , formulated as follows:
(2018) show the effectiveness of combination of CDA and 1 X
embedding debiasing, Zmigrod et al. (2019) and later Maud- AVERAGE H IGH (w, Z) = cosine(vx , vw ) (4)
|Z|
slay et al. (2019) extend CDA to address bias in morpholog- x∈Z
ically rich languages, as well as names. In this work, we use The bias using AVERAGE H IGH is calculated as follows:
the basic CDA method to study the reaction of the bias mea-
surement methods to the changes in the corpus. ψ(w) = AVERAGE H IGH (w, Z) − AVERAGE H IGH (w, Z 0 )Novel Bias Measurement w.2
We should note that the eSG representation is consider-
As discussed in Introduction, the first-order bias mea- ably different from the shifted PMI representation (Levy
surement requires the estimation of co-occurrence rela- and Goldberg 2014): shifted PMI assumes very high em-
tions, which we provide using explicit representations. bedding dimensions during the model training, while eSG
A well-known method to create explicit representations draws the co-occurrence relations after the model is trained
is based on the Point Mutual Information (PMI) met- on low-dimensional embeddings. In fact, as SG is an im-
ric. The PMI representation uses the count-based proba- plicit factorization of shifted PMI (Levy and Goldberg 2014;
bilities, where the co-occurrence relation between a word Ethayarajh, Duvenaud, and Hirst 2019a), eSG provides a
and a context-word in the PMI representation is calcu- smoothed variation of shifted PMI.
lated by log (p(w, c)/p(w)p(c)). Positive PMI (PPMI) is
a commonly-used variation, where negative values are re- explicit GloVe (eGloVe) The GloVe model first defines
placed with zero. Levy and Goldberg (2014) draw the re- an explicit matrix (size |V| × |V|), where the correspond-
lation between word2vec SkipGram (SG) embeddings and ing co-occurrence value of each word and context-word is
PMI representations, showing that SG can be seen as a fac- set to log p(w|c). This log probability is calculated based
torization of the PMI matrix shifted by log k. Based on this on the number of co-occurrences (denoted by #h, i), such
idea, they propose the Shifted Positive PMI (SPPMI) repre- that log p(w|c) = log #hw, ci − log #h·, ci. We refer to this
sentation by subtracting log k from PMI vector representa- sparse explicit matrix as initGlove.
tions and setting the negative values to zero. The GloVe model then implicitly factorizes the initGlove
In the following of this section, we first explain our ap- matrix, achieving two low-dimensional matrices of size
proach to creating the explicit variations of the SG and |V|×d, as well as two bias vectors of size |V|, where one as-
GloVe vectors, referred to as explicit Skip-Gram (eSG) and sign a bias value to each word, and the other to each context-
explicit GloVe (eGloVe). Our approach reconstructs explicit word. Using the same notation as SG, the factorization is
representations from embedding vectors, and is related to done such that the dot products of the vectors of the matri-
previous studies such as Ethayarajh, Duvenaud, and Hirst ces V and U plus the corresponding bias values estimate
(2019a), and Levy and Goldberg (2014). We then describe the logarithm of the co-occurrence values, as defined in the
our first-order bias measurement method, defined based on following:
any explicit vector.
vw u >
c + bw + b̃c ≈ log #hw, ci (6)
Smoothed Explicit Representations where bw and b̃c denote the bias value of word w and
explicit Skip-Gram (eSG) The original SG model con- context-word c, respectively.
sists of two parameter matrices: word (V ) and context (U ) Similar to eSG, the eGloVe representation estimates the
matrices, both of size |V| × d, where V is the set of words in co-occurrence relations using the word and context vectors,
the collection and d is the embedding dimensionality. Given after training the GloVe model. Considering Eq. 6, we define
the word c, appearing in a context of word w, the model the co-occurrence relations of eGloVe as the dot product of
calculates p(y = 1|w, c) = σ(vw u> the word and context vectors,3 shown as follows:
c ), where vw is the
vector representation of w, uc context-vector of c, and σ ew:c = vw u>
c , ew = vw U > ∈ R|V| (7)
denotes the sigmoid function. The SG model is optimized
by maximizing the difference between p(y = 1|w, c) and The eGloVe representation in fact reconstructs initGlove,
p(y = 1|w, č) for k negative samples č, randomly drawn providing a smoothed variation.
from a noisy distribution N . The N distribution is set to the
unigram distribution of the corpus, while downsampled by First-Order Bias Measurement
the context distribution smoothing parameter. The main difference between the first-order bias measure-
The p(y = 1|w, c) term in the SG model measures the ment method and the high-order approaches is in the estima-
probability that the co-occurrence of two words w and c tion of the associations of a word to the concepts. We define
comes from the genuine co-occurrence distribution, derived our bias measurement method based on the AVERAGE H IGH
from the training corpus. The model uses this probability method, by replacing the cosine metric with co-occurrence
to learn the embedding vectors, by separating these genuine relations, and therefore refer to it as AVERAGE F IRST . Given
co-occurrence relations from the sampled negative ones. We an explicit vector denoted as e, AVERAGE F IRST is defined as
therefore use this estimation of the co-occurrence relations 2
An alternative formulation of eSG is to normalize it by di-
to define the vectors of the eSG representation, resulting to viding its values with the square root of the expectations of the
the following definition of eSG vector: co-occurrence relations for each word and context-word, namely
Ec0 ∼N σ(vw u> >
c0 ) and Ew0 ∼N σ(vw0 uc ). We study this variation
in pilot experiments, observing similar results to the introduced
ew:c = σ(vw u>
c ), ew = σ(vw U > ) ∈ R|V| (5) variation. We therefore stay with the less complex formulation
(Eq. 5).
where ew:c denotes the value of the corresponding dimen- 3
As in eSG, eGloVe does not need extra normalization, since
sion of the vector of w to the context-word c, and ew in |V| the bias terms bw and b̃c in Eq. 6, learned during training, act as
dimensions is the explicit variation of the SG vector of word normalizers to the co-occurrence estimation log #hw, ci.the mean of the co-occurrence values of w with the repre- Gender-Representative Words. In all bias measurements,
sentative words Z, formulated as follows: the concepts Z and Z 0 are assumed as female and male, re-
spectively. Therefore, a positive bias value indicates the in-
1 X
AVERAGE F IRST (w, Z) = ew:c (8) clination towards female, and a negative one towards male.
|Z| The concepts are defined using two sets, each with 28 words,
c∈Z
containing words like she, her, woman for the female and he,
The bias toward Z is then defined as the differences be- his, man for the male concept. These sets are taken from pre-
tween the associations of w to Z and Z 0 : vious studies (Bolukbasi et al. 2016; Garg et al. 2018). From
ψ(w) = AVERAGE F IRST (w, Z) − AVERAGE F IRST (w, Z 0 ) the same sets, we form the gender pairs, used in the CDA
method, and listed in Supplemental Material.
As shown, AVERAGE F IRST only considers the context-words Occupational Words. We provide a list of 496 occupa-
related to the Z and Z 0 concepts. This avoids the influence tions, among which 17 are female-specific (e.g. congress-
of other non-relevant concepts as in the high-order bias mea- woman), 9 male-specific (e.g. congressman), and the rest are
surements. gender-neutral (e.g. nurse and dancer). The set and assigned
Let us review the required calculations for genders are listed in Supplemental Material.
AVERAGE F IRST (w, Z) when using the introduced smoothed Job market statistics. We study the correlation of the cal-
explicit representations, namely eSG or eGloVe. In this culated gender bias values with the statistics of the gender
setting, the main computation of the bias measure is the bias of a set of occupations, obtained from the U.S. job mar-
dot products of the vector of w to the context-vectors of ket. These statistics are provided by two collections, where
the words in Z. We should note that the computational in each the bias for an occupation is the percent of people
complexity of this calculation is the same as the one of in the occupation who are reported as female (e.g. 90% of
AVERAGE H IGH (w, Z) on SG/GloVe. In fact, considering nurses are women). The first collection uses the data pro-
the dot product of two embedding vectors as computation vided by Zhao et al. (2018a). The collection contains the
unit, the complexity of both bias measurement methods is statistics of 40 occupations, gathered from the U.S. Depart-
O(|Z|). ment of Labor. We refer to the collection as Labor Data. The
Finally, a practical consideration in calculating eSG and second is provided by Garg et al. (2018) using the U.S. cen-
eGloVe is that this computation requires the context vectors sus data. From the provided data, we use the gender bias
in addition to word vectors. These context vectors are com- statistics of the year 2015 – the most recent year in the col-
monly stored in the libraries used for SG and GloVe embed- lection, resulting in a list of 96 occupations. We refer to this
dings alongside the word vectors, mainly for the purpose of as Census Data. We use Spearman ρ and Pearson’s r corre-
continuous training. In eSG/eGloVe, these context vectors lations.
are exploited to estimate smoothed first-order relations.
Experiments and Results
Gender Bias Experiment Design In this section, we first present the results of correlation stud-
In this section, we explain the design of our experiment and ies with job market statistics, followed by analysing the re-
the resources. Our source code together with all resources, sults of the high-order bias measurement and discussing the
including the lists of occupational and gender-representative characteristics of the two approaches. We then visualize and
words are publicly available.4 . compare the gender bias of occupational words when mea-
Word Representations. We conduct our experiments on sured with the first-order versus high-order approach, and fi-
the PMI, PPMI, SPPMI, SG, eSG, Glove, and eGlove rep- nally study the reaction of the bias measurement approaches
resentations. In addition, we create the low-dimensional to bias reduction in the corpus.
vectors of the the PMI-based representations using Singu-
lar Value Decomposition (SVD), referred to as PMI-SVD, Correlation Results with Job Market Statistics
PPMI-SVD, and SPPMI-SVD. These word representation We calculate the gender bias of the occupations in
models are created on the English Wikipedia corpus of Au- the Labor Data and Census Data collections, using the
gust 2017. We project all characters to lower case, and re- high-order bias measurements (D IRECTIONAL, C ENTROID,
move numbers and punctuation marks. For all models, we and AVERAGE H IGH ) on both low-dimensional and ex-
use the window size of 5, and filter the words with frequen- plicit representations, and the first-order bias measurement
cies lower than 200, resulting in 197,549 unique words. The (AVERAGE F IRST ) on explicit vectors.
number of dimensions of the embeddings are set to 300. The Table 1 (next page) shows the results of the correlation be-
rest of the parameters are set using the default parameter tween the calculated gender bias, and the gender bias statis-
setting of the word2vec Skip-Gram model in the Gensim li- tics, provided by the Labor Data and Census Data collec-
brary (Rehurek and Sojka 2010), and the GloVe model in the tions. Each section of the table is assigned to a family of the
provided tool by its authors. As suggested by (Levy, Gold- models, namely PMI, PPMI, SPPMI, GloVe, and SG. The
berg, and Dagan 2015), we apply subsampling and context highest correlation results of each section are shown in bold,
distribution smoothing on all PMI-based models with the and the highest overall correlations are indicated with under-
same parameter values as the SG model. lines.
In all families of representations, the first-order bias mea-
4
https://github.com/navid-rekabsaz/SmoothedFirstOrderBias surement shows higher correlations than the high-orderLabor Data Census Data
Order Representation Method
Spearman ρ Pearson’s r Spearman ρ Pearson’s r
D IRECTIONAL 0.28 0.07 0.18 0.02
PMI C ENTROID 0.14 0.21 0.35 0.40
AVERAGE H IGH 0.33 0.24 0.27 0.19
High-Order
D IRECTIONAL 0.05 0.07 0.00 0.00
PMI-SVD C ENTROID 0.41 0.47 0.46 0.53
AVERAGE H IGH 0.41 0.49 0.49 0.56
First-Order PMI AVERAGE F IRST 0.53 0.51 0.57 0.62
D IRECTIONAL 0.45 0.49 0.39 0.47
PPMI C ENTROID 0.43 0.46 0.45 0.50
AVERAGE H IGH 0.43 0.46 0.45 0.52
High-Order
D IRECTIONAL 0.05 0.07 0.00 0.00
PPMI-SVD C ENTROID 0.41 0.47 0.46 0.53
AVERAGE H IGH 0.41 0.49 0.49 0.56
First-Order PPMI AVERAGE F IRST 0.59 0.58 0.64 0.64
D IRECTIONAL 0.26 0.37 0.26 0.28
SPPMI C ENTROID 0.39 0.45 0.45 0.48
AVERAGE H IGH 0.32 0.40 0.44 0.48
High-Order
D IRECTIONAL 0.17 0.29 0.11 0.03
SPPMI-SVD C ENTROID 0.28 0.35 0.39 0.43
AVERAGE H IGH 0.26 0.38 0.36 0.46
First-Order SPPMI AVERAGE F IRST 0.57 0.49 0.52 0.48
D IRECTIONAL 0.53 0.56 0.34 0.46
High-Order GloVe C ENTROID 0.58 0.60 0.39 0.51
AVERAGE H IGH 0.60 0.60 0.39 0.51
initGlove 0.38 0.42 0.40 0.51
First-Order AVERAGE F IRST
eGloVe 0.56 0.57 0.42 0.52
D IRECTIONAL 0.50 0.54 0.58 0.64
High-Order SG C ENTROID 0.55 0.57 0.60 0.65
AVERAGE H IGH 0.55 0.57 0.59 0.65
First-Order eSG AVERAGE F IRST 0.66 0.61 0.67 0.70
Table 1: Spearman ρ and Pearson’s r correlation results of the gender bias values, calculated with word representations, to the
statistics of the portion of women in occupations
methods, on both collections and correlation metrics (with tions of the text-based quantities with various external indi-
the exception of GloVe on Labor Data). Overall, eSG with cators (such as job market statistics) Caliskan, Bryson, and
AVERAGE F IRST shows the highest correlations across the Narayanan (2017); Garg et al. (2018), as in these work the
combinations. Interestingly, PPMI, as a simple count-based quantities are only calculated based on the high-order bias
method, when combined with AVERAGE F IRST shows higher measurements.
correlations in comparison with any high-order bias mea- Comparing the correlation results of high-order methods,
surement combined with SG or GloVe, on Census Data col- overall, AVERAGE H IGH and C ENTROID show similar results,
lection. which are slightly higher than the ones of D IRECTIONAL.
Considering the results of this study, in the following, we
The results indicate that the calculated bias values us- focus on analyzing the eSG and SG representations, and
ing the first-order method more closely follows the distri- AVERAGE H IGH among the three high-order measurements.
butions of the gender bias indicators for occupations. This
emphasizes the importance of exploiting the first-order bias Analysis and Discussion
measurement approach, as an alternative to the high-order
approaches. The results of these experiments are particu- In this section, we first diagnose the results of the high-order
larly applicable to the studies, which explore the correla- bias measurement, and then discus and compare the charac-manicurist: businesswoman, nurse, Filipina, In these representative samples, we can observe the ex-
seamstress, matron istence of various groups of context-words: several gender-
midwife: midwife, nurse, feminist, matron, suffragist neutral context-words like nurse, commanded, and appren-
nurse: midwife, nurse, matron, nursing, Filipina ticed shown with underlines; several gender-specific words,
socialite: businesswoman, Filipina, suffragist, which also contain other concepts such as an occupation like
feminist, hostess matron and actress; and finally person names such as Ernest
housekeeper: matron, midwife, nurse, maid, and Cyril.
governess As discussed in Introduction, some of these context-
words are arguably irrelevant to what we aim to character-
captain: commanded, capt, quartermaster, enlisted, ize, namely the extent to which a word is perceived as fe-
Hugh male/male. For instance, the association of manicurist with
colonel: commanded, Hugh, Ernest, guards, female is influenced by context-words such as nurse and
quartermaster businesswoman, or in the association of captain with male is
mechanician: apprenticed, Cyril, Ernest, Messrs, affected by commanded and enlisted. This issue is avoided
surveyor in AVERAGE F IRST , as the method calculates the associations
lieutenant: commanded, Ernest, Hugh, enlisted, by only considering a pre-defined set of highly-relevant
quartermaster context-words.
engineer: Jagmal, surveyor, apprenticed, Finally, let us put forward the question of which approach
draughtsman, engineer should be applied to bias measurement using text corpora?
Indeed answering this question, foremost, requires a clear
Table 2: Context-words with the highest effects on the cal- definition of what is aimed to be captured as bias. Our study
culated gender bias with the AVERAGE H IGH method. Gender- proposes AVERAGE F IRST , as a complementary to the exist-
neutral context-words are shown with underlines. ing high-order approaches, which exploits smoothed first-
order co-occurrence relations of the highly-relevant context-
words. This characteristic makes AVERAGE F IRST particularly
appropriate for bias measurement scenarios, in which cap-
teristics of the two bias measurement approaches. turing the direct relations between the words and the con-
We start by exploring which context-words in practice cept of interest is favorable. However, we should note that,
contribute the most to the calculated gender bias of the oc- depending on the objective of bias measurement, utilizing
cupational words with AVERAGE H IGH . To this end, we ex- second- or higher-order co-occurrences can be also highly
amine the measured gender bias of the occupational words important. For instance, if the aim of a study is to measure
using AVERAGE H IGH , applied on the eSG vectors (instead of the stereotypical bias of girls and boys towards the color
the SG vectors). We use eSG since its explicit vectors en- pink, it might be favorable to take into account the second-
able the diagnosis of the results, particularly by looking at order co-occurrences, for example the ones through cloth.
the context-words with the highest contributions. However, even in this scenario, we should still be aware
To diagnose the measured bias of AVERAGE H IGH , we first that utilizing the discussed high-order bias measurements
create the explicit variations of each female- and male- might yet not be an ideal approach, as these methods are
representative word vector, denoted as ez and ez0 (see sec- still affected by other non-relevant second- or higher-order
tion Smoothed Explicit Representations). Given the occupa- context-words. An optimal bias measurement method for
tion w with explicit vector ew , we calculate the high-order such a scenario still remains an open question, and study-
bias with AVERAGE H IGH , and provide its element-wise re- ing it is a potential direction for future work.
sults by removing the summation of the cosine function, for-
mulated as follows: Visualization of Gender Bias Results
1 X ez ew 1 X ez0 ew In this section, we continue our analyses by visualiz-
eψ = − 0
|Z| kez kkew k |Z | 0 0 kez0 kkew k ing the gender bias of all occupations, measured using
z∈Z z ∈Z
AVERAGE H IGH on SG, and AVERAGE F IRST on eSG.
where denotes the element-wise product, and eψ ∈ R|V| The results of the high- and first-order methods are shown
is the gender bias results of w for all context-words, each in Figures 2a and 2b, respectively. To make the results vi-
correspond to one dimension. sually comparable, we apply Min-Max normalization to the
To select a representative set of words for the analysis, measured associations of each approach, where the min/max
we first calculate the gender bias of all target words us- values are calculated over the measured associations of all
ing AVERAGE H IGH on SG vectors. We then select the top 5 words in vocabulary to male and female.
gender-neutral occupations with the highest bias towards fe- In both bias measurement methods, we are interested in
male, and the same towards male. These top female and male distinguishing between the words with significantly higher
words are shown in the top and bottom part of Table 2, re- bias values and any random word with low bias values. To
spectively. For each word, we show the top 5 context-words do this, we define a threshold for each plot, below which the
with the largest influence on the results of the gender bias words are considered as unbiased. To find the thresholds for
measurement, namely the highest absolute values of the di- distinguishing between the words with significantly higher
mensions in eψ . bias values and any random word with low bias values, since(a) High-order measurement (AVERAGE H IGH ) (b) First-order measurement (AVERAGE F IRST ) Figure 2: The associations of the occupations to the female and male concepts, indicating their inclinations towards the genders. The ones in the gray area are considered as unbiased. The occupations, inclined towards female and male are shown in purple and orange, respectively, where among them the gender-neutral ones, namely the occupations with stereotyped biases, have darker colors. the number of biased words to a concept are limited, we assume that there is a high probability that any randomly sampled word is unbiased. We therefore define the threshold for each bias measurement method as the mean of the abso- lute bias values of all words in vocabulary. These thresholds for AVERAGE H IGH with SG and AVERAGE F IRST with eSG are 0.036 and 0.047, respectively. In each plot, the area where the distance from the diagonal is less than the corresponding threshold value, is referred to as the unbiased area, and shown in gray. The occupations (a) AVERAGE H IGH (b) AVERAGE F IRST located in the unbiased areas are considered as unbiased (no significant stereotypical bias is observed). Figure 3: Histograms of the gender bias of the occupations, A gender-neutral occupation is considered to be stereo- measured using AVERAGE H IGH and AVERAGE F IRST . typically biased to either female or male, when it is inclined towards the female/male associations, namely when it is lo- cated below/above the unbiased area, shown in dark pur- AVERAGE F IRST . Similar to Figure 2, the gray color shows the ple and dark orange, respectively. The plot also depicts the number of unbiased occupations, and the purple and orange gender-specific occupations (e.g. actress, sportman), while colors indicate the number of biased ones towards female we should consider that these occupations are not stereotyp- and male in each bin, among which the gender-specific ones ically biased, but are expected to be inclined towards their are shown with light colors. respective genders. We show these gender-specific occupa- In both measurement methods, a larger number of occupa- tions in light colours, namely light purple for female, and tions are biased to male. However, the first-order bias mea- light orange for male. As expected, all gender-specific occu- surement captures a larger degree of bias towards female, pation words are on the correct side of the diagonal for both suggesting the existence of a more severe degree of female measures. bias, previously undetected by the high-order approach. Both figures show the existence of significant gender bias in several occupations. However, Figure 2a and 2b provide Sensitivity to Changes in Corpus considerably different perspectives on the extents of the as- How do the bias measurement methods react to (hypothet- sociations of the occupations to genders. In particular, the ical) future changes, especially when the provided corpora AVERAGE F IRST method shows relatively larger degrees of move towards more balanced representations of genders? bias towards female, specially for some gender-neutral oc- To explore this question, we use the basic form of the CDA cupations such as nurse, and housekeeper. method (see Related Work) to expand the corpus with syn- To have a better view on the distribution of the bias values, thetic augmented data. Using the list of gender words pairs, Figure 3 shows the histogram of the occupations over the for every sentence in the corpus, we swap each female word range of the bias values, measured with AVERAGE H IGH and with its counter-part male word, and vice versa. We create
Corpus Augmentation AVERAGE H IGH AVERAGE F IRST
with SG with eSG
First Step 0.037 0.044
Second Step 0.012 0.016
Table 3: Mean of the absolute values of bias changes in each
step of corpus augmentation experiments, discussed
higher sensitivity of the AVERAGE F IRST approach to the
changes in the corpus in comparison with AVERAGE H IGH ,
which is a valuable characteristic of the first-order method
particularly in the study of social changes using text data.
Figure 4: Changes in the gender bias values, measured on
the original corpus and two augmented ones with the CDA Conclusion
method. The black and red arrows indicate the results of We highlight a potential issue of the commonly-used high-
AVERAGE F IRST and AVERAGE H IGH , respectively. order bias measurement approaches, and propose a novel
approach based on the smoothed first-order co-occurrence
relations between words and their context words. To esti-
two new augmented corpora: the first one expands the origi- mate these relations, we reconstruct explicit vectors from
nal corpus with only one half of the new sentences (selected pre-trained word2vec Skip-Gram, and GloVe embeddings,
randomly), while the second one adds all the new sentences proposing smoothed first-order bias measurement method.
to the original corpus. Let us visit again the question of “which approach should
We create the SG models, and consecutively the eSG we use for measuring bias (or any societal phenomena) us-
representations, based on these two new corpora. We ing text corpora?” To answer this question, we recommend
then measure gender bias of the corpora using SG with the reader to first contemplate: “what construct/quality is
AVERAGE H IGH , and eSG vectors with AVERAGE F IRST . Fig- aimed to be captured as bias”. As an example, measur-
ure 4 shows the results of the changes in the bias values ing “to what extent different gender-neutral occupations are
of 5 randomly-selected gender-neutral occupations for each perceived differently” – as pursued in our experiments – ar-
gender, which are perceived as biased in the original cor- guably looks for a different construct/realization of bias in
pus by both bias measurements. Each arrow shows the di- comparison with measuring “what the stereotypical rela-
rection of the changes, where the starting point indicates the tion of color pink is towards girls and boys.”. The smoothed
measured bias in the original corpus, and the middle and first-order approach offers an alternative to high-order ap-
final points are calculated based on the first and second aug- proaches, and particularly has the advantage of narrowing
mented corpora. The black and red arrows show the results down the measurement to the effect of only a specific set of
of AVERAGE F IRST and AVERAGE H IGH , respectively. The gray representative words, appearing in the vicinity of the word
areas, bordered with the black and red vertical dashed lines under study.
show the unbiased areas of the results of AVERAGE F IRST and To have a better understanding of the differences of these
AVERAGE H IGH , respectively. As mentioned in previous sub- two approaches, our experiments examine and compare the
section, since in each method the values are normalized over measured gender bias of a set of occupational words, calcu-
all words, the bias values across the methods are reasonably lated on a Wikipedia text corpus. We observe that the gender
comparable. bias calculated with the proposed first-order approach corre-
As shown, in both methods the gender bias of all occupa- lates higher with the gender-related statistics of the job mar-
tions consistently decrease (move towards zero). The gender ket in comparison to the high-order methods. Our analyses
bias values measured on the second augmented corpus, for provide showcases of the existence of non-relevant context-
all the occupations, reach the corresponding unbiased area words when using high-order measurements, and suggests
of each method, indicating that these occupations in this the existence of a more severe degree of female-bias regard-
corpus are considered as unbiased. Comparing the bias mea- ing some jobs. While our experiments are conducted on gen-
surement methods, for these occupations, the AVERAGE F IRST der bias, the introduced method is generic and can be ex-
approach indicates the existence of a higher degree of bias tended to study other forms of societal biases such as related
in the initial corpus in comparison with AVERAGE H IGH , and to race, age and ethnicity.
also demonstrates a larger degree of bias reduction in each
step. The aggregated degrees of changes over all gender- Supplemental Material
neutral occupations are reported Table 3. As shown, in both
steps the AVERAGE F IRST demonstrate larger changes in com- Gender Definitional Words
parison with AVERAGE H IGH . Female: girl, girls, sister, sisters, mom, moms, mother, mothers,
These larger changes in the measured bias indicate the fiancee, grandmother, grandma, granddaughter, granddaughters,she, her, herself, hers, gal, gals, female, females, woman, women, hardener, harpooner, hatter, headmaster, healer, herbalist, histo-
madam, daughter, daughters, stepmother, stepdaughter rian, homemaker, hooker, housekeeper, hydrologist, illustrator, in-
dustrialist, infielder, inspector, instructor, insulator, interpreter,
inventor, investigator, janitor, jeweler, jeweller, joiner, journal-
Male: boy, boys, brother, brothers, dad, dads, father, fathers, ist, judge, jurist, keeper, knitter, laborer, labourer, lacemaker,
fiance, grandfather, grandpa, grandson, grandsons, he, him, him- lawmaker, lawyer, lecturer, legislator, librarian, lieutenant, life-
self, his, lad, lads, male, males, man, men, sir, son, sons, stepfather, guard, lithographer, lumberjack, lyricist, machinist, maestro, ma-
stepson gician, magistrate, maltster, manager, manicurist, marksman, mar-
shal, mason, masseur, master, mathematician, mechanic, mechani-
Pairs: (boy, girl), (boys, girls), (brother, sister), (brothers, cian, mediator, medic, melter, merchandiser, metallurgist, met-
sisters), (dad, mom), (dads, moms), (father, mother), (fathers, alworker, meteorologist, metrologist, microbiologist, midfielder,
mothers), (fiance, fiancee), (grandfather, grandmother), (grandpa, midwife, miller, millwright, miner, minister, missionary, mob-
grandma), (grandson, granddaughter), (grandsons, granddaugh- ster, model, modeller, mover, musician, musicologist, nanny, nar-
ters), (he, she), (him, her), (himself, herself ), (his, hers), (lad, gal), rator, naturalist, negotiator, neurologist, neurosurgeon, novelist,
(lads, gals), (male, female), (males, females), (man, woman), (men, nurse, nutritionist, observer, obstetrician, officer, official, oper-
women), (sir, madam), (son, daughter), (sons, daughter), (stepfa- ator, optician, optometrist, organiser, organist, orthotist, owner,
ther, stepmother), (stepson, stepdaughter) packer, paediatrician, painter, palmists, paperhanger, paralegal,
paramedic, parishioner, parliamentarian, pastor, pathologist, pa-
Target Words trolman, patternmaker, paver, pawnbroker, pediatrician, pedi-
curist, performer, pharmacist, philanthropist, philosopher, pho-
Female-specific occupations: actress, ballerina, barmaid, tographer, photojournalist, physician, physicist, physiotherapist,
businesswoman, chairwoman, chambermaid, congresswoman, pianist, pilot, planner, plasterer, playwright, plumber, poet, po-
forewoman, housewife, landlady, maid, masseuse, matron, mis- lice, politician, pollster, porter, potter, poulterer, preacher, presi-
tress, policewoman, sportswoman, stewardess, stuntwoman, ush- dent, priest, principal, prisoner, producer, professor, programmer,
erette, waitress projectionist, promoter, prompter, proprietor, prosecutor, pros-
thetist, protagonist, protege, protester, provost, psychiatrist, psy-
Male-specific occupations: barman, businessman, foreman, chologist, psychotherapist, publicist, publisher, pundit, radiogra-
landlord, policeman, salesman, serviceman, sportsman, waiter pher, radiologist, radiotherapist, ranger, realtor, receptionist, ref-
eree, refiner, registrar, repairer, reporter, representative, rescuer,
researcher, restaurateur, retoucher, rigger, roaster, roofer, sailor,
Gender-neutral occupations: accountant, actor, adminis- saint, sales, salesperson, sausagemaker, saxophonist, scaffolder,
trator, adventurer, adviser, advocate, agent, aide, ambassador, scholar, scientist, screenwriter, scriptwriter, sculptor, seaman, sec-
analyst, animator, announcer, anthropologist, apprentice, archae- retary, senator, sergeant, servant, setter, sewer, shepherd, sher-
ologist, archeologist, architect, archivist, artist, artiste, assas- iff, shoemaker, shopkeeper, shunter, singer, skipper, smith, so-
sin, assessor, assistant, astrologer, astronaut, astronomer, ath- cialite, sociologist, soldier, solicitor, soloist, songwriter, special-
lete, attendant, attorney, auctioneer, auditor, author, bailiff, baker, ist, spinner, sportswriter, staff, statesman, statistician, steelworker,
ballplayer, banker, barber, bargee, baron, barrister, bartender, steeplejack, steward, stockbroker, stonecutter, stonemason, store-
basketmaker, beautician, beekeeper, bibliographer, biochemist, bi- keeper, strategist, student, stuntman, stylist, substitute, superinten-
ologist, biotechnologist, blacksmith, boatman, bodyguard, boil- dent, supervisor, surgeon, surveyor, sweep, swimmer, tailor, tamer,
erfitter, boilermaker, bookbinder, bookkeeper, bookmaker, boot- tanner, tannery, teacher, technician, technologist, telecaster, teller,
maker, boss, botanist, boxer, breeder, brewer, bricklayer, broad- therapist, tinsmith, toolmaker, tracklayer, trader, trainer, train-
caster, broker, bureaucrat, butcher, butler, buyer, cabbie, cabi- man, translator, traveller, treasurer, trooper, trucker, trumpeter,
netmaker, cameraman, campaigner, captain, cardiologist, care- tuner, turner, tutor, tycoon, typesetter, tyrefitter, undersecretary,
taker, carpenter, cartographer, cartoonist, cashier, cellist, ceo, ce- understudy, upholsterer, usher, valedictorian, valuer, varnisher,
ramicist, chairman, chancellor, chaplain, character, chef, chemist, vendor, veterinarian, viniculturist, violinist, vocalist, warden, war-
chief, chiropractor, choreographer, cinematographer, citizen, rior, washer, weaver, weigher, welder, whaler, wigmaker, worker,
cleaner, clergy, cleric, clerical, clerk, coach, collector, colonel, wrestler, writer, zookeeper
columnist, comedian, comic, commander, commentator, commis-
sioner, composer, compositor, concreter, conductor, confectioner, References
confesses, congressman, conservationist, conservator, constable,
consultant, cook, cooper, cop, coremaker, correspondent, coun- Barrett, M.; Kementchedjhieva, Y.; Elazar, Y.; Elliott, D.;
cilman, councilor, counsellor, counselor, courtier, critic, crooner, and Søgaard, A. 2019. Adversarial Removal of Demo-
croupier, crusader, crusher, curator, custodian, cutler, dancer, graphic Attributes Revisited. In Proceedings of the Confer-
dean, decorator, dentist, deputy, dermatologist, designer, detec- ence on Empirical Methods in Natural Language Process-
tive, developer, dietician, dietitian, digger, diplomat, director, dis- ing.
patcher, diver, doctor, doorkeeper, draughtsman, draughtsperson,
dresser, dressmaker, driller, driver, drummer, drycleaner, dyer, Bolukbasi, T.; Chang, K.-W.; Zou, J. Y.; Saligrama, V.; and
ecologist, economist, editor, educator, electrician, embroiderer, Kalai, A. T. 2016. Man is to computer programmer as
employee, engineer, engraver, entertainer, entrepreneur, environ- woman is to homemaker? debiasing word embeddings. In
mentalist, envoy, epidemiologist, ergonomist, ethnographer, evan- Proceedings of the 30th International Conference on Neural
gelist, expert, farmer, farrier, filmmaker, financier, firebrand, fire- Information Processing Systems.
fighter, fisherman, fitter, footballer, furrier, gangster, gardener, ge-
neticist, geographer, geologist, geophysicist, gilder, glassmaker, Caliskan, A.; Bryson, J. J.; and Narayanan, A. 2017. Se-
glazier, goalkeeper, goatherd, goldsmith, gravedigger, grinder, mantics derived automatically from language corpora con-
guard, guide, guitarist, gunsmith, hairdresser, handler, handyman, tain human-like biases. Science .Chang, S.; and McKeown, K. 2019. Automatically Infer- Kaneko, M.; and Bollegala, D. 2019. Gender-preserving De-
ring Gender Associations from Language. In Proceedings of biasing for Pre-trained Word Embeddings. In Proceedings
the Conference on Empirical Methods in Natural Language of the Annual Meeting of the Association for Computational
Processing. Linguistics.
Church, K. W.; and Hanks, P. 1990. Word association norms, Kirchler, E. 1992. Adorable woman, expert man: Changing
mutual information, and lexicography. Computational lin- gender images of women and men in management. Euro-
guistics . pean Journal of Social Psychology .
De-Arteaga, M.; Romanov, A.; Wallach, H.; Chayes, J.; Kiritchenko, S.; and Mohammad, S. 2018. Examining Gen-
Borgs, C.; Chouldechova, A.; Geyik, S.; Kenthapadi, K.; and der and Race Bias in Two Hundred Sentiment Analysis Sys-
Kalai, A. T. 2019. Bias in bios: A case study of semantic tems. In Proceedings of the Joint Conference on Lexical and
representation bias in a high-stakes setting. In Proceedings Computational Semantics.
of the Conference on Fairness, Accountability, and Trans-
parency. ACM. Kontostathis, A.; and Pottenger, W. M. 2006. A framework
for understanding Latent Semantic Indexing (LSI) perfor-
Deerwester, S.; Dumais, S. T.; Furnas, G. W.; Landauer, mance. Information Processing & Management .
T. K.; and Harshman, R. 1990. Indexing by latent semantic
analysis. Journal of the American society for information Lauscher, A.; Glavaš, G.; Ponzetto, S. P.; and Vulić, I. 2020.
science . A general framework for implicit and explicit debiasing of
distributional word vector spaces. In Proceedings of the
Dev, S.; and Phillips, J. 2019. Attenuating Bias in Word AAAI Conference on Artificial Intelligence.
vectors. In The International Conference on Artificial Intel-
ligence and Statistics. Lenton, A. P.; Sedikides, C.; and Bruder, M. 2009. A la-
tent semantic analysis of gender stereotype-consistency and
Dixon, L.; Li, J.; Sorensen, J.; Thain, N.; and Vasserman, narrowness in American English. Sex Roles .
L. 2018. Measuring and mitigating unintended bias in text
classification. In Proceedings of the AAAI/ACM Conference Levy, O.; and Goldberg, Y. 2014. Neural word embedding
on AI, Ethics, and Society. as implicit matrix factorization. In Proceedings of the In-
ternational Conference on Neural Information Processing
Elazar, Y.; and Goldberg, Y. 2018. Adversarial Removal of Systems.
Demographic Attributes from Text Data. In Proceedings of
the Conference on Empirical Methods in Natural Language Levy, O.; Goldberg, Y.; and Dagan, I. 2015. Improving dis-
Processing. tributional similarity with lessons learned from word embed-
dings. Transactions of the Association for Computational
Ethayarajh, K.; Duvenaud, D.; and Hirst, G. 2019a. Towards Linguistics .
Understanding Linear Word Analogies. In Proceedings of
the Annual Meeting of the Association for Computational Lu, K.; Mardziel, P.; Wu, F.; Amancharla, P.; and Datta, A.
Linguistics. Association for Computational Linguistics. 2018. Gender bias in neural natural language processing.
arXiv preprint arXiv:1807.11714 .
Ethayarajh, K.; Duvenaud, D.; and Hirst, G. 2019b. Un-
derstanding Undesirable Word Embedding Associations. In Maudslay, R. H.; Gonen, H.; Cotterell, R.; and Teufel, S.
Proceedings of the Conference of the Association for Com- 2019. It’s All in the Name: Mitigating Gender Bias with
putational Linguistics. Name-Based Counterfactual Data Substitution. In Proceed-
ings of the Conference on Empirical Methods in Natural
Garg, N.; Schiebinger, L.; Jurafsky, D.; and Zou, J. 2018. Language Processing.
Word embeddings quantify 100 years of gender and ethnic
stereotypes. Proceedings of the National Academy of Sci- Mikolov, T.; Sutskever, I.; Chen, K.; Corrado, G. S.; and
ences . Dean, J. 2013. Distributed representations of words and
phrases and their compositionality. In Proceedings of the
Gonen, H.; and Goldberg, Y. 2019. Lipstick on a Pig: Debi- International Conference on Neural Information Processing
asing Methods Cover up Systematic Gender Biases in Word Systems.
Embeddings But do not Remove Them. Proceedings of the
North American Chapter of the Association for Computa- Monroe, B. L.; Colaresi, M. P.; and Quinn, K. M. 2008.
tional Linguistics . Fightin’words: Lexical feature selection and evaluation for
identifying the content of political conflict. Political Analy-
Günther, F.; Rinaldi, L.; and Marelli, M. 2019. Vector-space
sis .
models of semantic representation from a cognitive perspec-
tive: A discussion of common misconceptions. Perspectives Pennington, J.; Socher, R.; and Manning, C. 2014. Glove:
on Psychological Science . Global vectors for word representation. In Proceedings of
the Conference on Empirical Methods in Natural Language
Hoyle, A.; Wallach, H.; Augenstein, I.; and Cotterell, R.
Processing.
2019. Unsupervised Discovery of Gendered Language
through Latent-Variable Modeling. In Proceedings of the Rehurek, R.; and Sojka, P. 2010. Software framework for
Conference of the Association for Computational Linguis- topic modelling with large corpora. In In Proceedings of the
tics. LREC Workshop on New Challenges for NLP Frameworks.You can also read
