California Health Interview Survey Sexual Orientation and Gender Identity Working Group Summary and Final Recommendations June 29, 2018
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
California Health Interview Survey
Sexual Orientation and Gender Identity Working Group
Summary and Final Recommendations
June 29, 2018
In May 2018, the California Health Interview Survey (CHIS) Sexual Orientation and Gender Identity (SOGI)
Working Group convened twice to discuss and adopt a set of recommendations for changes to be made to
the SOGI questions included in the 2019-2020 cycles of CHIS. The first meeting occurred on Tuesday, May
8, 2018. The second was held two weeks later, on Tuesday, May 22, 2018. This document summarizes the
discussion and final recommendations that were made by the SOGI Working Group during these meetings.
SECTION 1: Measurement of Respondent Sex and Gender Identity
SECTION 1A. BACKGROUND
Measures of gender identity were first introduced to the CHIS survey in the final quarter of CHIS 2014 as a
field experiment designed to test the effectiveness of different question wording. A random sample of
approximately 500 CHIS respondents were selected to receive one of several potential gender identity
questions or sequences of questions. The results of this experiment were presented to the SOGI Working
Group at the previous Working Group meeting in June 2016. Based on these findings, the two-step gender
identity method was identified as the best performing alternative of those tested within CHIS. This method
produced the lowest non-response rates and less confusion among cisgender respondents, while identifying
transgender individuals at similar rates to those expected based on other national population estimates.
After this initial experiment, the gender identity questions were expanded to include all CHIS respondents in
the 2015-2016 cycle. Due to a vendor programming error, respondents over the age of 70 were skipped out
of these questions for most of CHIS 2015. During the 2015-2016 cycle, CHIS tested two ways of wording the
first step of the two-step method using a split-sample design. The first SOGI Working Group Meeting in
May 2018 provided an opportunity for the CHIS team to give the group a progress update on the results of
this experiment, as well as elicit feedback from these content experts regarding other potential changes to this
sequence for future cycles. These potential changes included replacing the current question collecting
sex/gender information with the two-step gender identity sequence for adults; potential modifications to
question wording to reflect the addition of a “nonbinary” category in California state documents; replacing
the current sex/gender question with the two-step gender identity sequence with age-appropriate
modifications for adolescents; replacing the current sex/gender question with the first step of the two-step
gender identity sequence for children; and adding the potential “nonbinary” response option to reflect
changes to California state law that will allow “nonbinary” as an option for recording infant sex on original
birth certificates.
CHIS Update on Results of Two-Step Gender Identity Experiment
During the 2015-2016 cycle, CHIS tested two ways of wording the initial step of the two-step gender identity
sequence. Both versions asked respondents to provide the sex reported on their original birth certificate, but
they differed in the placement of the birth certificate mention and the explicit listing of response options. The
first, referred to as the Indirect Method, places the mention of the birth certificate at the end of the question and
does not state the potential response options. The second version, referred to as the Direct Method, places the
birth certificate mention at the beginning of the question, and explicitly references the “male” and “female”
1response options. The full wording of the two versions of the question can be found in Table 1. CHIS
respondents had an equal probability of being randomly assigned to each of the two versions of the question.
Table 1: Gender Identity Two-Step Method Experimental Testing of First Step Question Wording
Experimental Condition 1: Indirect Method
AD65: What sex were you assigned at birth, on your original birth certificate?
Experimental Condition 2: Direct Method
AD65A: On your original birth certificate, was your sex assigned as male or female?
Final Proposed Two-Step Gender Identity Question Wording
AD65A: On your original birth certificate, was your sex assigned as male or female?
AD66: Do you currently describe yourself as male, female, or transgender?
If response is “None of these” then AD67: What is your current gender identity?
The quality of the two versions of the first step question was assessed in two ways. First, the non-response
rates of the two versions were compared. The results of this comparison can be found in Table 2 below. Due
to a skip error in 2015, the gender identity questions were only asked of respondents who were ages 70 and
younger in this year. To more easily compare 2016 to 2015 and assess the effects of this skip error, the non-
response rates for 2016 are presented both for this subsample of adult respondents only, as well as for all
adult respondents. The rate of respondents providing a “Don’t Know” response was comparable between the
two versions, though the Direct Method elicited a slightly higher rate of refusals than the Indirect Method. Based
on this criterion, the Indirect Method was slightly preferred; however, both versions resulted in a very low rate
of non-response among adult CHIS respondents.
Table 2: Non-Response Rates by Type for Gender Identity Question Wording Experiment
AD65: AD65A:
Indirect Method Direct Method
Don’t Know
2015: Adults Ages 70 and Under 0.04% 0.12%
2016: Adults Ages 70 and Under 0.09% 0.09%
2016: All Adults 0.13% 0.10%
Refusal
2015: Adults Ages 70 and Under 0.15% 0.29%
2016: Adults Ages 70 and Under 0.16% 0.25%
2016: All Adults 0.17% 0.25%
Note. Unweighted percentages.
The second method of assessing the quality of the two versions of the first step question was qualitative; it
was based on monitoring the administration of the gender identity sequence to respondents by interviewers.
During these monitored sessions, evaluators found that CHIS respondents found the Indirect Method more
difficult to understand and respond to. Respondents asked the interviewer to repeat the question more
frequently, leading to longer interview times. After this evaluation, the CHIS team decided that the slightly
lower refusal rate was not sufficiently beneficial to outweigh the additional costs associated with respondent
confusion associated with the Indirect Method and so the Direct Method became the final version of the two-step
gender identity question included in subsequent cycles of CHIS (as indicated in Table 1).
2CHIS Update on Previous SOGI Working Group Recommendation to replace original sex question
with two-step gender identity sequence
In the previous SOGI Working Group meeting in June 2016, the Working Group recommended that CHIS
consider replacing the current method of collecting adult respondent sex (AA3: Are you male or female?)
with the two-step gender identity sequence (bottom section of Table 1). Though there were initial concerns
that such a change could lead to an increase in respondent refusals or break-offs early in the questionnaire,
because AA3 is the second question, asked immediately following age, the results of the 2014 experiment and
2015-2016 CHIS cycle suggests that the risk of interview refusals and break-off is low.
With the completion of the 2015-2016 CHIS cycle, the CHIS team was able to further evaluate the
implications of the removal of AA3 by comparing responses to AA3 with those for the two-step gender
identity sequence. The results of this comparison can be found in Table 3. Among the vast majority of
respondents, including among those who are transgender, the response to the original sex question matches
their response to the first step of the two-step gender identity sequence; therefore, it does not seem likely that
replacing AA3 with the gender identity sequence will introduce additional error in measurement. Based on
these results, the CHIS team decided to move forward with replacing the original sex question (AA3) with the
gender identity sequence (AD65A/AD66) on the adult questionnaire.
Table 3: Comparison of Original Sex Question and Gender Identity Sequence in 2015-2016 CHIS
AA3: Male AA3: Female
AD65NEW: Gender AD65NEW: Gender
on Birth Certificate on Birth Certificate
AD66: Currently describe self as… Male Female Male Female
Male 99.49% 0.05% 0.01% 0.01%
Female 0.08% 0.19% 0.03% 99.69%
Transgender 0.14% 0.02% 0.02% 0.13%
None of the Above 0.03% 0.00% 0.00% 0.12%
Total 99.74% 0.26% 0.06% 99.94%
Note. Red fields indicate respondents that are considered “Transgender” on the CHIS transgender construct.
In one important way, the switch to the two-step gender identity method may improve our measurement of
respondent sex. In 2016, for 0.20% of all adult respondents, the interviewer recorded a value for AA3 that is
not consistent with the respondent’s report for either step of the gender identity sequence. The CHIS team
suspects that these are cases in which the interviewer did not explicitly ask AA3, but recorded the
respondent’s sex based on the respondent’s voice. In these cases, the interviewer recorded the wrong sex for
the respondent.
The change to replace the original sex question has implications for a number of other questions that have
sex- or gender-specific skip conditions or fill-in text that are included in CHIS. The list of these measures can
be found in Table 4. The CHIS team raised the question of which measure from the two step gender identity
sequence should be used to determine eligibility for questions with sex-/gender-based skip conditions or to
fill-in sex-/gender-specific text throughout the questionnaire. In the previous SOGI Working Group meeting
in June 2016, the group provided recommendations regarding which of the sex assigned on the respondent’s
birth certificate or current gender identity should be used for each of the sex-/gender-specific questions.
During the first meeting of the 2018 SOGI Working Group, the CHIS team provided the group with the list
of recommendations made by the previous Working Group. They asked participants to review these
recommendations and confirm that they agreed with these initial recommendations.
3Table 4: CHIS Questions with Sex-/Gender-Specific Skip Conditions or Fill-In Text
CHIS Question Topic or Questionnaire Section GI Variable to Use
Sex-/Gender-Specific Skip Conditions
Section E: Women’s Health (includes current pregnancy status) Sex at birth (AD65A)
Received Prenatal Care in Past Year (AH77) Sex at birth (AD65A)
Section J: Family Planning subsection Sex at birth (AD65A)
WIC Eligibility (AL7) Sex at birth (AD65A)
Binge drinking Sex at birth (AD65A)
Sex-/Gender-Specific Fill-In Text in Question(s)
Diabetes (AB22, AB99, AB81, e.g., gestational diabetes) Sex at birth (AD65A)
Weight (AE18) Sex at birth (AD65A)
Sexual Orientation (AD46B) Gender Identity (AD66)
Pre-exposure Prophylaxis (AD79, AD80, AD81, AD82) Gender Identity (AD66)
The 2018 SOGI Working Group agreed with the gender-specific skip pattern recommendations of the 2016
SOGI Working Group 1. The CHIS team raised the question of whether the Working Group believed that
further testing would be needed to determine how changes in the skip conditions or the fill-in text might
affect survey responses. The Working Group expressed minimal concern regarding the need for testing of
these changes prior to full implementation.
The Passage of California Senate Bill 179 (SB-179), California’s Gender Recognition Act
In October 2017, the California State legislature passed and Governor Brown signed into law a new bill, SB-
179, California’s Gender Recognition Act. This bill will go into effect on January 1, 2019 and mandates that
the State of California “…provide three equally recognized gender options on state-issued identification
documents – female, male, and nonbinary – and an efficient and fair process for people to amend their
gender designation on state-issued identification documents.” Because they are the primary forms of
identification provided by the state of California, SB-179 specifically applies to birth certificates and driver’s
licenses.
This change raised two primary concerns for the CHIS staff. First, the current version of the CHIS two-step
gender identity sequence specifically refers to sex on the respondent’s original birth certificate. It is possible
that because SB-179 allowed individuals to change the sex listed on their birth certificate, this would create
confusion and misreporting of transgender status, particularly for those whose current birth certificates list
their sex as “nonbinary”.
A second concern was that though SB-179 introduces the category of “nonbinary” as a legal category, the
current version of the CHIS two-step gender identity question does not include the term “nonbinary”. For
this reason, the current language may not adequately reflect the current legal landscape in California. While
use of the term “nonbinary” is rare in California at the current time, we expect that familiarity with and use of
the term will increase over time due to implementation of SB-179 on January 1, 2019.
1 There was a potential exception to the question on current WIC enrollment related to the passing of Senate Bill 179
(SB-179, which is covered in detail in the next section) and how those who had formally requested a change of gender
would be impacted in relation to WIC. Upon inquiry to the California Department of Public Health’s WIC Division, we
were informed that, while they are currently changing their system to be in compliance with the new law, the eligibility
criterion will remain the same which is “based on income eligibility, nutritional risk and categorical eligibility: are they
pregnant, recently had a baby and breastfeeding or non-breastfeeding, or a child under the age of 5 years. If they meet
[any of] these requirements, they should be eligible to receive services regardless of what gender they identify themselves
as” (personal communication, 6/14/18). Given this insight, we do feel it is appropriate to maintain the originally
recommended GI variable (sex at birth, AD65A) to replace the gender-specific skip condition.
4The CHIS team raised these concerns with the 2018 SOGI Working Group who proposed several
recommendations regarding changes to the gender identity measures in CHIS. These recommendations are
detailed in the next section.
SECTION 1B. RECOMMENDATIONS
Recommendation 1: In the second step of the two-step gender identity sequence (AD66), add an
explicit response option “None of these”.
Recommendation 1A: Use a split panel design to test whether the addition of the new “None of
these” response option leads to changes in responses to the gender identity sequence.
The passage of SB-179 introduced the possibility that a growing number of respondents will legally change
their original birth certificate to reflect their current gender identity. The ability to change one’s birth
certificate to reflect one’s current gender identity could potentially lead some respondents to report that their
sex on their original birth certificate is “nonbinary” or to report their current gender identity rather than the
sex they were originally assigned at birth.
CHIS identifies the transgender population by comparing the sex on the respondent’s original birth certificate
to their current gender identity. If a transgender respondent changes their birth certificate to reflect their
current gender identity and reports the sex on their revised birth certificate rather than the originally assigned
gender, this could lead CHIS to underestimate the size of the transgender population. Because this population
is already such a small group within CHIS, the CHIS team was concerned that even small changes in
reporting could lead to meaningful changes in these estimates.
The CHIS team raised this concern with the SOGI Working Group. The Working Group was not convinced
that the “original birth certificate” terminology would be unclear to respondents who had changed their birth
certificate to reflect their current gender identity. Moreover, due to the fact that only a small subset of the
transgender population is likely to change their birth certificate, the number of respondents who would
potentially be affected by this change would be very small, potentially negligible. Because the process to
change one’s birth certificate requires a bureaucratic process, even after the passage of SB-179 there remain
barriers to changing one’s birth certificate that do not exist for self-identification. For this reason, we might
expect only a small minority of the transgender population to go through the process to modify their birth
certificates. Clarifying language available to the interviewer could be included for those with any confusion
about what “original” birth certificate means.
SB-179 could also lead to an increase in the use of the term “nonbinary” in response to one or both of the
gender identity questions. To the extent that CHIS does not capture this information, it may lose
comparability to State records and estimates, reducing its utility for addressing public policy. However,
because the change to inclusion of a “nonbinary” category on the original birth certificate will not go into
effect until 2019, no adults in California will currently have an original birth certificate with a third gender
category. For these reasons, the SOGI Working Group recommended that no change is necessary for the
first question in the two-step method for adults.
The adoption of the “nonbinary” language by the State of California may have a larger impact on responses
to the second step of the two-step gender identity sequence. Currently, the explicit response categories for the
second step use the term “transgender” and do not reference the State’s preferred “nonbinary” terminology.
The CHIS team asked the SOGI Working Group to weigh in on whether the question should be changed to
reflect this change in terminology (see Table 5).
5Table 5: Proposed Recommendations to Second Step of Adult Gender Identity Sequence (AD66)
Option 1: Maintain the current wording for AD66
Do you currently describe yourself as male, female, or transgender?
Pros Cons
Consistent with previous cycles May underestimate use of new gender identity
terminology
Does not reflect terminology used by the state of
California
Option 2: Add new explicit response category to capture other potential response options
Example 1: Do you currently describe yourself as male, female, transgender, or none of these?
Example 2: Do you currently describe yourself as male, female, transgender or something else?
Example 3: Do you currently describe yourself as male, female, transgender, or another gender?
Pros Cons
Explicitly allows for other gender identities, Potential break in historical trends
including (but not limited to “nonbinary”) Needs testing to understand the impact on
If CHIS moves to web, explicit option will estimates
be needed “Something else” may be too open-ended,
allowing respondents to provide irrelevant
responses
One concern was that the “nonbinary” terminology used by the State of California in SB-179 is not widely
understood by the general population. Because this is a new change to the law and affects only a small
percentage of Californians, replacing the term “transgender” with the term “nonbinary” may lead to greater
confusion for survey respondents who are not aware of the new law, particularly for those who are cisgender
and not aware of this terminology. This change could also lead to difficulties in translating the survey into
other languages, where no similar terminology exists. Instead, the CHIS team proposed adding a more general
response option that would provide the ability to capture a wider array of responses, while also making it
possible to document growth in the “nonbinary” terminology.
It is important to note that in the 2015-2018 waves of CHIS, interviewers have the ability to record a fourth
response option that is not explicitly listed in the question as it is read to respondents. Respondents who tell
the interviewer that they do not fit into any of the three explicitly mentioned response categories (male,
female, and transgender) are coded as “None of these”. Those whose response falls into this fourth, unstated
category are asked a follow-up question: “What is your current gender identity?” Their open response
answers are recorded by the interviewer, to be coded by the CHIS team during data processing. Therefore,
the current question wording does allow respondents who prefer to use the “nonbinary” terminology to use
this term and have it recorded as their preferred gender identity; however, the fact that this option is not
explicitly mentioned to the respondent could discourage reporting of it overall.
The proposed change to the questionnaire would make the availability of a fourth response option explicit to
the respondent. This would have the benefit of matching the response options that would need to be
provided if the CHIS survey moved to implement a web-based survey option in the future. The Working
Group discussed two options for the wording of this additional general response category. The first option,
“None of these,” reflects the current wording of the additional response option as it appears to the
interviewer. The second option, “Something else,” potentially broadens the scope of responses that could fit
into that category. The third option, scheduled to be tested by the Williams Institute, similarly broadens the
scope, but allows for additional clarification that the desired response should be a gender. Regardless of the
final wording of the fourth explicit response option, any respondent who fell into this fourth category would
be asked to provide their current gender identity in an open-response follow-up question.
6The Working Group was concerned that “None of these” would sound strange to the respondent. However,
the CHIS team was concerned that a broader “Something else” response option would encourage more
respondents to provide open-response values that were not relevant to the gender identity sequence or were
uninterpretable or uncodable. The “None of these” response would have the additional benefit of being
consistent with the current interviewer coding of gender identity sequence. Moreover, “None of these” could
discourage additional responses from all but those who are uncomfortable classifying themselves into one of
the three existing categories, reducing the likelihood that the addition of a new response option would
substantially change reporting of gender identity over time. The open-response follow-up question would
allow the CHIS team to monitor the growth of these “other” responses over time and assess whether other
terminology such as “nonbinary” becomes more common. For these reasons, the SOGI Working Group
recommended using the “None of these” language to define the fourth response category.
Explicitly including the “None of these” response option could potentially encourage more respondents to
provide a response that does not fit into one of the three original response categories. This could potentially
affect the identification of transgender respondents within CHIS. For this reason, the SOGI Working Group
recommended testing the effects of this wording change by using a split-sample testing approach. Under this
approach, each respondent would randomly be assigned to two survey conditions. The first would use the
original gender identity sequence question wording without the explicit mention of the “None of these”
response option in the question. The second condition would include “None of these” in the list of response
options provided to the respondent. The CHIS team could then compare the rate at which the “None of
these” category was coded, as well as the open-text responses to the follow-up question, between the two
conditions.
Recommendation 2: Replace the original sex question with the full two-step gender identity
sequence for adolescents.
Recommendation 2A: Modify the second question in the two-step gender identity sequence in an
age-appropriate way to better reflect the greater uncertainty and knowledge about gender identity
among adolescents.
Recommendation 2B: Test modifications of the second step of the gender identity sequence to
ensure they are adequately capturing gender identity among this age group.
The June 2016 SOGI Working Group advanced the recommendation that CHIS introduce measures of
gender identity for children and teens. CHIS has committed to replacing the original sex question with the
gender identity sequence among adults. Because the original sex question is similar across the adult,
adolescent, and child questionnaires, the CHIS team considered also making a similar change to replace the
original sex question with the gender identity sequence on the teen questionnaire. However, there are some
additional complications that arise when making this change for adolescents and children that were not
present for the adults.
First, unlike among adults, CHIS has not previously included measures of gender identity in the adolescent
questionnaire. Instead of being asked about their gender identity, teen respondents have been asked about
their gender expression, specifically, how their peers view them in terms of how they present themselves to
the world. Though gender expression and gender identity are related, they are conceptually distinct. While
gender expression focuses on the perceptions of others, gender identity focuses on internal identification.
This internal identification with a specific gender may or may not match the outward-facing expression of
gender that others perceive.
Another complication is that the adolescent (teen) questionnaire is given to respondents who are ages 12 to
17 years old. These young respondents may not be familiar with the difference between gender identity and
7reported sex or with terminology such as “transgender”, which could lead them to be confused by the second
question of the two-step gender identity sequence. If teens are confused about this terminology, this could
lead to misreporting of transgender status. Further complicating this issue is the fact that many adolescents
are currently in the process of actively questioning and understanding their own gender identity. Their gender
self-identification may be fluid or in flux, making responding to a survey question that provides only well-
defined categories difficult to answer.
Because CHIS has not included measures of gender identity in previous cycles, there is no previous CHIS
data that can be used to evaluate the impact of replacing the original sex question with the gender identity
sequence. Among adults, it was possible to compare how adults responded to the original sex question to
how they responded to the gender identity sequence, specifically, how they responded to the first step of this
sequence. This comparison showed very few differences between these two reports, suggesting that the
change would not appreciably affect the measurement of respondent sex. This comparison is not available for
adolescents; therefore, the impact of this change can only be guessed for this group.
One option the SOGI Working Group considered was to replace the original sex question with only the first
step of the two-step gender sequence. This option would have the benefit of allowing the teen sex question to
mirror that of the adult without introducing the additional issues that would come from asking teens to report
their current gender identity. Though CHIS has not asked adolescents about the sex on their original birth
certificate in the past, the Working Group saw no reason to expect that this age group would experience
greater difficulty answering this question than the adult population.
The SOGI Working Group was concerned that if CHIS was to replace only the first question of the gender
identity sequence, CHIS would lose the opportunity to measure the development of gender identity among
California teens. Several members of the Working Group pointed to other surveys and studies that had been
able to collect gender identity information from adolescents. This change took on greater urgency when
CHIS was unable to find a funder for the teen gender expression questions in the 2019-2020 cycle. Unless
additional funding is acquired, the gender expression questions will be dropped from the 2019-2020
questionnaire. Replacing the original sex question with the gender identity sequence would have the added
benefit of converting the gender identity sequence to CHIS core questionnaire status, insulating this sequence
against being removed from the questionnaire in the future due to funding constraints. For these reasons, the
Working group recommended that CHIS replace the original sex question with the gender identity sequence
in the 2019-2020 cycle.
Because adolescents face different issues in understanding and reporting their current gender identity, the
two-step method used for adults may require modification before it can be applied to adolescents. The CHIS
team raised the possibility of making two types of changes to the two-step gender identity sequence before
inclusion on the teen survey. The first type of change was to incorporate the third “nonbinary” category into
the first step, or birth certificate question. The second type of change was to modify the second step question
to take into account the greater uncertainty and more limited knowledge that adolescents may have about
their own gender identity.
As was the case with adults, the passage of SB-179, California’s Gender Recognition Act, could lead some
respondents to report that the sex recorded on their birth certificate was neither male nor female, but
“nonbinary”. The current version of the first step of the gender identity sequence does not include this third
category as a potential response option. Using the same reasoning for teens as they used for adults, the SOGI
Working Group recommended that CHIS adopt the same first step question for teens as is used for adults.
The change in the law allowing birth certificates to use the “nonbinary” category does not go into effect until
January 2019; none of the adolescents included in CHIS during the 2019-2020 cycle should be recorded as
“nonbinary” on their original birth certificate. For this reason, no change is needed to the question wording
of the first step.
8Table 6: Proposed Recommendations to Replace the Original Sex Question with Gender Identity for
Adolescents
Option 1: Maintain the current sex question wording:
Are you male or female?
Pros Cons
Consistent with previous cycles Would no longer be comparable to the adult
Consistent with other federal and state Does not allow measurement of gender identity
health surveys Not consistent with changes in state law allowing
for a third sex/gender option
Option 2: Replace original sex question with first step of the two-step gender identity sequence:
On your original birth certificate, was your sex assigned as male or female?
Pros Cons
Comparability to adult interview Has not been tested among adolescents
Does not introduce potentially confusing Lack of comparability to other data sources
terminology, such as “transgender” No previous CHIS data to use to assess impact
of change on reporting of sex
Option 3: Replace original sex question with the full two-step gender identity sequence used for
adults:
On your original birth certificate, was your sex assigned as male or female?
Do you currently describe yourself as male, female, or transgender?
If response is “None of these” then: What is your current gender identity?
Pros Cons
Consistent with the adult protocol Has not been tested among adolescents
Would make the gender identity questions Terminology might be confusing to adolescents
part of the CHIS core questionnaire Gender identity may be in flux and not yet
Would provide first estimates of gender defined, especially at the younger ages
identity among teens Sample size of transgender teens likely to be too
small for analysis for an extended period.
Option 4: Replace original sex question with the full two-step gender identity sequence, but use a
modified version that captures age-specific instability in and lack of familiarity with gender
identity:
Example 1:
On your original birth certificate, was your sex assigned as male or female?
Do you currently describe yourself as male, female, transgender, or none of these?
Example 2:
On your original birth certificate, was your sex assigned as male or female?
Do you currently describe yourself as male, female, transgender, none of these, do you not know yet, or do
you not know what this question means?
Pros Cons
Can be used to provide first estimates of Has not been tested among adolescents
gender identity among teens No consensus on best question wording to use
(Depending on final wording) Can (Depending on final wording) Might depart too
explicitly measure uncertainty and much from adult protocol, leading to issues with
confusion with gender identity comparability
The previous SOGI Working Group recommended that if the two-step gender identity method was adopted
for adolescents, the second step of the sequence should be changed to include two additional response
options. The first new response option should capture any potential uncertainty the adolescent currently feels
regarding their gender identity. The second response option should allow the respondent to indicate lack of
knowledge about gender identity, specifically, that they did not understand what the question was asking. The
full question with all response options would read: Do you currently describe yourself as male, female,
transgender, none of these, do you not know yet, or do you not know what this question means?
9The CHIS team asked the current SOGI Working Group to provide feedback on this proposed change. In
general, the Working Group agreed that the two-step sequence that was administered to adults should be
modified to allow teens who were uncertain about their gender identity or were unsure they understood the
concept of gender identity to provide a valid response that reflected their current thinking. However, the
group expressed concern that the additional response options made the question more difficult to follow
and/or understand. Several Working Group members discussed alternative wording that they had included or
seen included in other studies and offered to provide this language to the CHIS team. One common question
to enumerate transgendered individuals is used by BRFSS: “Do you consider yourself to be transgender?” If
the respondent answers “Yes” then the interviewer probes with a follow-up question: “Do you consider
yourself to be male-to-female, female-to-male, or gender non-conforming?”
One concern that the CHIS team had regarding some of these proposed alternatives is that they would
introduce larger differences between the methods used to collect gender identity information from adults and
teens. This could make it more difficult to combine the adult and teen gender identity data together, as well as
lead to a shift in measurement between ages 17 and 18. In addition, some proposed alternatives could not
serve as a substitute for the original sex question in the same way that the first step question from the two-
step method can. Without this ability to substitute the questions, the teen gender identity sequence could not
replace the original sex question as a core CHIS question and would not be insulated from funding shortfalls.
Taken together, the lack of a clear standard for measuring gender identity among children and teens and the
lack of previous data that can be used to evaluate the impact of potential changes mean that any changes to
the adolescent questionnaire will require further testing to assess their impact. The SOGI Working Group
recommended that CHIS test any new gender identity questions for adolescents. Additional gender identity
questions to consider are included in Appendix A.
Recommendation 3: Replace the original sex question with the first step of the two-step gender
identity sequence for children.
Recommendation 3A: Modify the replaced question to include a third, “nonbinary” response option
that is not explicitly read by the interviewer.
Recommendation 3B: If web survey is introduced, include follow-up screen asking respondent
whether child is “nonbinary” if respondent does not provide a response to the child sex question.
Given the proposed recommendation to bring the adult and adolescent gender identity sequences into
harmony with each other, it is natural to ask whether the sex question for children should also be brought
into alignment by asking the first step of the two-step gender identity question. While the move could break
with historic trends consistent with other federal and state health surveys, maintaining the original wording
removes the comparability within CHIS. The Working Group recommended that in order to maintain
consistency within the CHIS, all three household interviews should utilize the same question wording.
Beginning in 2019 in conjunction with the implementation of SB-179, children born in California can be
issued an original birth certificate on which their sex is recorded as neither male nor female, but “nonbinary”.
While this legislation is primarily aimed at adults, it is unclear how this will be used by parents of newborns.
The current sex question only allows two response options: “male” and “female”. Parents of children whose
sex is recorded as “nonbinary” may not be able to classify their children into either of these categories.
Moreover, by not including this as a response option, CHIS would no longer reflect the current data and
policy landscape of California. If the state of California includes the “nonbinary” category in population
projections, CHIS would no longer be able to use these estimates for weighting and benchmarking purposes.
10For this reason, the CHIS team asked the SOGI Working Group whether they should consider changing the
original sex question to allow for a third “nonbinary” response.
Table 7. Proposed modifications to the original sex question on the Child Questionnaire
Option 1: Maintain the current sex question wording:
Is [CHILD] male or female?
Pros Cons
Consistent with previous cycles Would no longer be comparable to the adult
Consistent with other federal and state Does not allow measurement of gender identity
health surveys Not consistent with changes in state law allowing
for a third sex/gender option, potentially causing
disagreement with state estimates
Option 2: Replace the original sex question with the first step of the two-step gender identity
sequence and include a third, “nonbinary” response option that is not read by the interviewer:
On [CHILD]’s original birth certificate, was their sex assigned as male or female?
Pros Cons
Allows parents to report “nonbinary” Potential break in historical trends
children Preserves the normative male/female dichotomy
Does not change the current question without reflecting spirit of SB-179
wording from respondent’s perspective Comparability with web version of survey, which
Unobtrusive, won’t alienate parents early in would require an explicit response option
survey administration
Doesn’t require decision on how to refer to
third category
Option 3: Modify to explicitly include a third, “nonbinary” response option that is read by the
interviewer:
On [CHILD]’s original birth certificate, was their sex assigned as male, female, or nonbinary?
Pros Cons
Allows parents to report “nonbinary” Break in historical trends
children Has not been tested.
Reflects the post SB-179 legal landscape Lack of familiarity with the “nonbinary”
Comparability to state estimates that terminology among California parents
include “nonbinary” sex category Could alienate respondents early in
administration (second question on survey)
The CHIS team had several concerns about making this change. First, it was not clear that many California
parents would be familiar with the “nonbinary” terminology. This could lead to confusion and potentially
misreporting or underreporting. Second, the original sex question appears as the second question on the
CHIS questionnaire. The CHIS team was concerned that raising the “nonbinary” sex issue so early during the
questionnaire’s administration may alienate some respondents and lead them to refuse to complete the
remainder of the survey. One solution for this issue is to include “nonbinary” as a response option that is not
read to the respondent; however, this could lead to a problem if CHIS adopted a web version of the survey.
In order to record “nonbinary” as a response, this response option would have to be presented to the
respondent explicitly.
After consideration of these options, the SOGI Working Group recommended that CHIS include
“nonbinary” as a response option that is not explicitly read to the respondent during the interview. They did
not expect that this change would require additional testing. To address the concern that this could lead to
differences between the phone and web versions of the survey, the Working Group recommended that CHIS
could add a follow-up question about “nonbinary” sex that is asked only when the respondent does not
11provide an answer regarding the child’s sex. These changes would allow CHIS to capture the growth in
reporting of “nonbinary” over time in an unobtrusive way.
SECTION 2: Measurement of Sexual Orientation
SECTION 2A: BACKGROUND
There are several known issues with the CHIS self-identified sexual orientation question (AD46), particularly
when interviewing in languages other than English. These issues are related to two areas: 1) the grammar and
word choice for the items, and 2) the translation of sexual orientation concepts into other languages.
Concerns about these issues are long-standing and were raised with the previous SOGI Working Group;
however, no changes to question wording were implemented during the 2017-2018 cycle. The primary change
that was introduced to the sexual orientation question was to expand the universe of respondents who are
asked this question to all adults. Prior to the 2015-2016 cycle, this question was restricted to adults ages 70
and younger.
The first concern about the English version is that the original CHIS questions include the words
“heterosexual” and “homosexual” (Table 8) which are words that are technical, confusing, and less well-
known to respondents who have less education and/or limited English proficiency. Both words are
multisyllabic, begin with the letter H, and end in –sexual, which can make them more difficult to understand
or distinguish between over the phone, particularly for those for whom English is not their first language.
However, the alternative phrasing, which uses the terms “gay” and “straight”, may also lead to confusion for
many who are unfamiliar with these slang terms. The increasing prominence of the acronym LGBT in public
life has likely also increased familiarity with these terms, but they may remain unfamiliar to those for whom
English is not their first language.
Table 8. Wording of current CHIS sexual orientation question (AD46/AD46B)
English Version:
Do you think of yourself as straight or heterosexual, as gay {, lesbian} or homosexual, or bisexual?
Interviewer Instructions: IF NEEDED SAY:
Straight or heterosexual people have sex with, or are primarily attracted to people of the opposite
sex, Gay {and Lesbian} people have sex with or are primarily attracted to people of the same sex,
and Biseexuals have sex with or are attracted to people of both sexes.
Spanish Version:
¿Se considera usted heterosexual, gay, {lesbiana} u homosexual, o bisexual?
Interviewer Instructions: IF NEEDED SAY:
La gente heterosexual tiene relaciones sexuales o siente atracción principalmente por personas del
sexo opuesto. Los homosexuales y gay {y lesbianas} tienen relaciones sexuales o sienten atracción
principalmente por personas del mismo sexo. Los bisexuales, tienen relaciones sexuales o les
atraen personas de ambos sexos.
A second concern about the current question wording in English is that by placing the technical terms
(heterosexual and homosexual) and the slang terms (straight and gay) together in the response options
separated by “or” the question could read as if these are separate response options rather than different ways
of stating the same response option. When combined with the potential lack of familiarity with either set of
terminology, this could increase respondents’ confusion with the question, leading to higher item non-
response.
These concerns become more acute when considering their impact on the non-English translations of the
sexual orientation question. For example, Table 8 shows that the Spanish-language question is a direct
12translation of the English-language version of the sexual orientation question. The terms “heterosexual” and
“homosexual” are even less familiar terms in Spanish than they are in English; therefore, the same concerns
about the use of these terms that were present for the English version also apply to the Spanish-language
translation.
This problem is compounded by the fact that there is no direct Spanish translation for the English slang term
“straight” that indicates heterosexuality. The closest approximation to this term in Spanish roughly translates
in English to “normal”; however, asking respondents whether they are “normal”, particularly in this context,
is likely to (understandably) offend some respondents. For this reason, no additional clarifying slang term is
presented with “heterosexual” in the Spanish-language translation of the sexual orientation question. The
terms “gay” and “lesbian” do have Spanish-language translations; however, members of the Working Group
raised the concern that these translated terms are less likely to be familiar to older and less-educated
respondents, as well as those who recently immigrated to the United States.
Item non-response rates from the most recent cycles of CHIS bear out these concerns. Figure 1 shows the
percentage of respondents who either refused to answer or answered “Don’t Know” to the sexual orientation
question in the CHIS 2011-2017 waves for English- and Spanish-speaking respondents. Though historically
item non-response rates were low, beginning in 2015 it increased dramatically among Spanish-language
interviews. Most of this increase was in the percentage of Spanish-language respondents who answered
“Don’t Know”, indicating higher levels of confusion with the question.
Figure 1. Item Non-Response Rates for Sexual Orientation Question, CHIS 2011-2017
There were two important changes that occurred between the 2014 and 2015 waves of CHIS. First, CHIS
changed data vendors for the first time since the survey began in 2001. With this change in vendors came a
new set of interviewers who had less experience in administering the CHIS questionnaire. This could have led
to higher rates of item non-response throughout the survey, but particularly for questions that were more
difficult for respondents to understand or answer. The second change was a methodological change to shift
13the sample composition from 80% landline/20% cell phone numbers to 50% landline/50% cell phone
numbers. This increased the number of Spanish-speaking respondents with lower levels of education who
were more likely to have difficulty understanding the terminology used in the question. However, the increase
in item non-response occurred within both the landline and the cell phone sample, indicating that this
methodological change alone could not explain the increase.
In addition to Spanish, the CHIS survey is translated into several Asian languages, including Korean,
Vietnamese, Chinese (both Cantonese and Mandarin dialects), and Tagalog. Though the discussion here
focuses on the English- and Spanish-language versions of the questionnaire, the same concerns also apply to
the Asian-language translations. As with Spanish, the direct translations of the words “heterosexual” and
“homosexual” are unfamiliar in many of these languages and the slang term “straight” cannot be directly
translated. Due to the smaller number of Asian-language interviews, it was not possible for the CHIS team to
examine trends over time in item non-response for the Asian-language interviews.
SECTION 2B: RECOMMENDATIONS
Recommendation 4: Modify the English-language wording of AD46 to more closely mirror the
wording currently used by NHIS and several other national health surveys.
The June 2016 SOGI Working Group considered several options for modifying the sexual orientation
question to address the concerns that were raised about the current question wording, but did not come to a
decision regarding which one method they should recommend. To prepare for the 2018 SOGI Working
Group meeting, the CHIS team surveyed the set of questions used by different health surveys to collect
information on sexual orientation. The results of this investigation can be found in Table 9. CHIS is not
alone in using its current question wording; this method is also used by two other surveys: the National
Survey of Family Growth (NSFG) and the Health Center Patient Survey (HCPS).
Table 9. Sexual Orientation Question Wording from Several State and National Health Surveys
SOURCE QUESTION WORDING
CHIS (AD46), English Version:
NSFG†, Do you think of yourself as straight or heterosexual, as gay {, as lesbian} or
HCPS homosexual, or bisexual?
Spanish Version:
¿Se considera usted heterosexual, gay {, lesbiana} u homosexual, o bisexual?
NATS English Version:
Do you think of yourself as (lesbian or) gay, straight, that is, not gay, or bisexual?
Spanish Version:
N/A
BRFSS (pre- English Version:
2018), Do you consider yourself to be straight, lesbian or gay, or bisexual?
SMART Spanish Version:
recommendation ¿Usted se considera heterosexual, lesbiana o gay (homosexual), o bisexual?
NHIS (2018), English Version:
NHANES Which of the following best represents how you think of yourself? (Lesbian or) gay;
NCVS straight, that is, not (lesbian or) gay; bisexual; or something else?
BRFSS (2018) Spanish Version:
¿Cuál de las siguientes mejor representa su manera de pensar en sí mismo? (Lesbiana
o) gay; heterosexual, o sea, no (lesbiana o) gay; bisexual; o otra cosa?
14† NSFG lists the terms “heterosexual” and “homosexual” before the terms “straight” and “gay”/”lesbian”, respectively,
in the question, but is otherwise similarly worded to CHIS and HCPS.
One health survey that uses a method that addresses the first concern that was raised about the current CHIS
sexual orientation question is the National Adult Tobacco Survey (NATS). NATS uses similar question
wording to CHIS, but eliminates the use of the terms “heterosexual” and “homosexual” and reverses the
order of the straight and gay/lesbian response options. The slang term “straight” is used with the follow-up
clarifying text, “that is, not gay”. This method has the benefit of eliminating the problematic technical terms
“heterosexual” and “homosexual” while otherwise making only minimal changes to the current question
wording. The June 2016 SOGI Working Group noted that the minor change of eliminating these terms could
be easily tested using an experimental design. One concern with this method is that NATS is not translated
into Spanish or another language, where the word “straight” cannot be directly translated. This omission
could have a larger impact if the word “heterosexual” is also omitted.
In 2009, the Williams Institute’s Sexual Minority Assessment Research Team (SMART) released a
collaborative report that outlined a set of best practices for collecting sexual orientation information on
surveys. This method was used by the Behavioral Risk Factor Surveillance System (BRFSS) prior to 2018.
Like the NATS question, this version of the question omits language referring to “heterosexual” and
“homosexual” from the English version, but preserves the response option order currently used in CHIS (i.e.,
“straight” appears as the first response option). This base part of this question is also a larger departure from
the original CHIS question; while the original question in CHIS (and NATS) asks respondents what they
think of themselves as, the SMART recommendation asks respondents what they consider themselves to be.
One concern about this method of collecting sexual orientation data is that because the slang term “straight”
does not have a direct translation, the first response option in the Spanish version is “heterosexual”, with no
clarifying text to indicate what this word means.
The final option that the SOGI Working Group considered appears to be the current preferred wording for
most national health surveys. This method is used in the National Health Interview Survey (NHIS), the
National Health and Nutrition Examination Survey (NHANES), the National Crime Victimization Survey
(NCVS), and, beginning in 2018, BRFSS. Like CHIS and NATS, this question asks respondents to report
what they think of themselves, though the base question is more detailed and indicates that a list of response
options will be provided (i.e., “Which of the following…”). Like the version used by NATS, the English
version of the question omits the potentially confusing or unfamiliar terms “heterosexual” and “homosexual”
and reverses the order of straight and gay/lesbian response options. The term “straight” is clarified using text
indicating that the term refers to those who are not gay/lesbian.
One of the most important differences between this version and the preceding ones is that it includes “or
something else” among the response options that are explicitly read to the respondent. Historically, CHIS has
allowed respondents to provide an alternative open-text response to the sexual orientation question, but this
option has not been explicitly read to the respondent. Despite this, CHIS has experienced an increase in the
number of respondents providing an alternative response, such as “asexual”. Modifying the question to
explicitly include this as a response option may increase the number of respondents who provide this type of
response.
The Spanish-language translation cannot avoid the use of the term “heterosexual” due to the inability to
translate the word “straight” into Spanish. However, unlike the Spanish-language version based on the
SMART recommendation, the Spanish-language translation of the NHIS sexual orientation question includes
clarifying text that indicates what is meant by the term heterosexual (i.e., “heterosexual, o sea, no (lesbiana o)
gay”). This improvement is likely to reduce confusion and provide clarification for those who are unfamiliar
with the term “heterosexual”.
15Taken together, the SOGI Working Group recommended using the NHIS version of the sexual orientation
question. This version addresses the concerns about the use of the heterosexual/homosexual terminology in
the English version, while maintaining clarity when translated into Spanish. Moreover, the inclusion of an
explicit “something else” category would provide respondents with the greatest reporting flexibility. Previous
research has shown that this wording reduces confusion and item non-response rates. Though this change
could potentially show a break in historical trends across cycles of CHIS, this was outweighed by the benefit
that transitioning to this question wording would increase comparability between CHIS and other major
health surveys, such as BRFSS and NHIS.
One Working Group member noted that the construction that lists the response options as “gay/lesbian, not
gay/lesbian, or bisexual” reads somewhat strangely because the options “gay/lesbian” and “not gay/lesbian”
would seem to encompass the full universe and make the “bisexual” response option superfluous though it is
not. A potential change could be to change the order of the response options to accommodate this. Other
members suggested that the “bisexual” category was similar to an “all of the above” response that often
appears at the end of a list of response options. After some discussion, the group recommended leaving the
ordering as it appears in NHIS.
Recommendation 5: Re-translate the new sexual orientation question wording into Spanish and
adopt recommendations of Milesi et al (2017) for Spanish translation.
After the SOGI Working Group recommended transitioning to the NHIS question wording, the CHIS team
provided some additional information about recent research involving the Spanish-language version of this
question. In particular, a research team at NORC found that a minor change in the ordering of the wording
of the heterosexual response option (Table 10) further improved response rates to the sexual orientation
question among Spanish-speaking respondents. This change placed the word “heterosexual” at the end of the
response option so that it read, “No (lesbiana o) gay, o sea, heterosexual,” rather than “Heterosexual, o sea,
no (lesbiana o) gay”. The SOGI Working Group recommended that CHIS adopt this change when translating
the sexual orientation question into Spanish.
Table 10. Proposal for Sexual Orientation Question Spanish Version Response Option Order
Question:
¿Cuál de las siguientes mejor representa su manera de pensar en sí mismo?
NHIS Milesi et al (2017)
(Lesbiana o) gay (Lesbiana o) gay
Heterosexual, o sea, no (lesbiana o) gay No (lesbiana o) gay, o sea, heterosexual
Bisexual Bisexual
Otra cosa Otra cosa
SOGI WORKING GROUP MEMBERSHIP
The CHIS 2019-2020 SOGI Working Group was co-chaired by Tara Becker and Brian Wells. Members who
attended by phone for the two working group meetings and helped to review and approve the final
recommendations include Bryn Austin (Harvard), Philip Brenner (UMass Boston), Kerith Conron (Williams
Institute), Jody Heymann (Williams Institute), Patricia Lee (DHCS), Stuart Michaels (NORC), Ilana Newman
(CDPH), Maricel Miguelino (DHCS), Cary Sanders (CPEHN), Diem Tran (CDPH), Bianca Wilson (Williams
Institute), and Tracy Zhao (API Equality-LA). Additional membership who were unable to participate include
Michelle Adams (CDPH), Cullen Fowler-Riggs (CDPH), Angelique Lastinger (DHCS), Kevin Limrick
(CDPH), Marisa Ramos (CDPH), and Deanna Sykes (CDPH).
16You can also read
