A look at recommended practices for demographic questions from ESOMAR and the IDEA Council
Editor’s note: This article is based on a report from ESOMAR titled, “Gender: Best Practice Recommendations for multi-country work” and an article from the Insights Association IDEA Council entitled, “The Evolution of Demographic Questions.”
All around the world there is controversy surrounding how to ask gender questions in surveys. With differing views, translation issues and confusion on terms, it can be difficult to conduct multinational studies.
ESOMAR released a report with tips on how to conduct multicountry demographic studies. The Insights Association IDEA Council also has an article on the issue of how to ask gender and race/ethnicity demographic questions.
While the ESOMAR report is focused on a global scale, the IDEA Council article focuses on these topics in the United States. In this blog, I will share some of the key highlights from both.
Gender vs. sex
Before getting into the recommendations of ESOMAR and the IDEA Council, let's explore the difference between gender and sex. These terms are often intertwined but are truly different.
According to the Merriam-Webster Dictionary, gender is defined as “the behavioral, cultural or psychological traits typically associated with one sex.”
Whereas the Merriam Webster definition for sex is “either of the two major forms of individuals that occur in many species and that are distinguished respectively as female or male especially on the basis of their reproductive organs and structures.”
ESOMAR and the IDEA Council share additional descriptors that dive into the differences:
“Gender is personal, how someone sees themselves, while sexual orientation is interpersonal, concerned with to whom someone is emotionally, romantically and/or sexually attracted.” - ESOMAR
“We have seen a lot of new questions that are mixing these elements. Many of the terms used are not well understood by research participants leading to many using the wrong category or not responding at all. The result is often biased responses as respondents are confused with the questions and how best to answer them.” - The IDEA Council
According to the ESOMAR report, the majority of the world population is “cisgender” meaning “their gender is the same as the sex they were assigned and birth”
For more definitions of terminology that is important in this context, see page 10 of the ESOMAR report and pages three and 11 in the Insights Association article.
General recommendations - gender and marketing research studies
ESOMAR gave three overarching recommendations for approaching the measurement of gender in studies. They are as follows:
- Acknowledge that gender is not a binary concept for some people.Consider adding a statement about inclusivity and acknowledging that there are more than two genders.
- Be mindful that not everyone’s sex and gender will be the same. Your study could convey this by giving the reason why you are asking about gender.
- Do not ask participants their gender if the information required is already held by the sample provider and can be appended to your survey, noting it is helpful that panelists can update their own status information on a dynamic basis for this specific topic. According to ESOMAR, this is a good practice to avoid participant friction based on asking the same question over and over again.
The Insights Association IDEA Council article has the following general recommendations for researchers conducting studies in the United States.
- To decide if you need to ask these demographic questions, look at the objectives, topics and the target demographic.
- In terms of placement, if a question is sensitive and not needed for screening, put it at the end of the questionnaire.
- Make sure that the experience you are creating for your respondents is inclusive and appropriate by choosing questions that reflect those values.
- Use language such as “Identify on my own terms (Please specify)” or “I prefer to self-identify (Please specify)” with open-ended write-in options, rather than “Other (Please specify).”
- Include a “prefer not to answer” response option to allow for participants to skip or opt out of answering.
- If possible, utilize open-ended questions.
- Be sure your coding system is inclusive and equitable.
Gender questions
When wording the question(s) about gender in your study, keep in mind that the original language might not transfer to others. For example, the English description of “nonbinary” might not translate into Spanish, Russian, Arabic and so on.
Due to the sensitive nature of gender in some populations, make sure your core question can be adapted as needed.
The example ESOMAR uses is to use a “core nonbinary question” in any country that it would be safe for participants to answer the question. However, you should change it slightly in countries where it is not safe.
Example wording from the ESOMAR report is:
(For safe countries) Are you:
- Female
- Male
- Another gender
- Prefer not to answer
(For not safe countries) Are you:
- Female
- Male
- Prefer not to answer
ESOMAR recommends keeping the list of genders short to avoid translation issues and to avoid being too invasive. They mention that a study among the LGBTQ+ population on a related topic could have a self-describe option to ensure you are not leaving anyone out.
It is also recommended by ESOMAR to alphabetize or randomize the list of genders to treat them equally. The IDEA Council agrees by recommending rotating male and female in the list to show equality.
Another way to ensure wording is following best practices as defined by the local population is to look at the country census and see if they have a gender question on there. By using the same terminology, research can more easily ensure weighting to the population is accurate.
On the other hand, the IDEA Council recommends using a three-question approach that looks like this:
- What sex were you assigned at birth, on your birth certificate? (Please select one.)
- Male
- Female
- How do you describe yourself? (Please select one.)
- Male
- Female
- Transgener
- None of these
- What is your current gender identity? [open end]
The council argues that this formatting allows researchers to:
- Capture sex to help match your quota to the census results.
- Capture broad categories of gender identity.
- Offers respondents the chance to self-identify, ensuring there is not a misunderstanding in language.
Safety and legal considerations for gender questions in surveys
As mentioned above, it is important to consider where the survey or study is taking place. There are many places around the world where it is extremely dangerous to not be cisgender.
ESOMAR specially mentioned these restrictions when asking gender questions:
- In some countries, it is unsafe, illegal or unacceptable for a participant to be something other than male or female (cisgender).
- Be especially considerate when interviewing children and/or vulnerable/at-risk populations or asking parents/guardians about their children.
- When asking open-ended questions or if there are specific answers that may indicate a medical procedure, keep in mind they could require legal or privacy opt-ins.
The Human Rights Watch is a great resource to find information on laws against the LGBTQ+ community around the world. They have an interactive map and country profiles page linked here.