Five months ago, Chalkbeat and the Reynolds Journalism Institute launched a project that will make it easier for newsrooms to track the diversity of their sources. 

This project is designed to expand Chalkbeat’s existing source audit work, which centers on two of our recurring features: How I Teach and First Person. And we also have the lofty ambition of building a better system of source auditing that newsrooms around the nation can adopt. More on that in the coming months — and if you want to stay connected, let us know here.  

We recently published our review of the audits — everything we learned about best practices in source auditing from leading U.S. newsrooms. We wanted to take that work a step further by analyzing the actual surveys and making recommendations for how newsrooms can best structure their survey language with equity in mind. WPR, MPR, KUER, KCBX, KQED, Inewsource, Reframe, and WHYY graciously shared their auditing processes with us, the majority of which are public radio stations. 

Our hope is this explainer, while not exhaustive, will give journalists food for thought on how to design questions that are inclusive and clear. We created this example survey on Google Forms incorporating all of our recommendations for a visual guide on what a potential source auditing survey might look like. 

Here’s what we learned, and what we recommend.

Race/ethnicity 

The question

We saw that when asking about race, newsrooms typically did so in one of three ways: 

1. Race or Race/Ethnicity 
2. What is your race?
3. How would you best describe your racial identity?

What we recommend: Option No. 3: How would you best describe your racial identity? The use of “best” gives acknowledgment to the fact that there probably won’t be a comprehensive list of every single race/ethnicity on a survey, but it will still encourage responses from sources. 

Native American vs. American Indian vs. …   

There isn’t any one term used to describe the Indigenous people who live in the United States. Indian Country Today interviewed six prominent Native Americans, and all had different preferences in terminology. While the term used most broadly in recent decades is Native American, those who grew up with the term American Indian continue to prefer it over all others. 

American Indian is the most widely used terminology among the newsrooms we interviewed. The U.S. Census groups the term American Indian with Alaskan Natives, as did three surveys. 

Native Hawaiian or Pacific Islander should be its own choice because Hawaii is geographically grouped with the Pacific Islands rather than the North American continent.  

What we recommend: Create two choices for Native people who live in the United States: American Indian or Alaskan Native and Native Hawaiian or Pacific Islander. 

Asian   

While nearly all surveys we looked at used some variation of Asia/Asian to cover the entire region, some broke it down further into different parts of Asia, with options such as South Asian, East Asian, Filipino, etc. Newsrooms could do the latter, being careful not to exclude any part of Asia. 

However, we don’t typically see a breakdown of Asia in survey designs, which creates a question of how to analyze this later on when compared to U.S. population data. And it could also create confusion: Central Asia overlaps with the Middle East— should people who are from Afghanistan click both options? Should one be omitted in favor of the other?

What we recommend: The data will be simplest to analyze if newsrooms keep the choice as Asian. 

Middle Eastern

The Middle East was listed as a separate option in half of the surveys we analyzed, and all referred to the ethnic identity differently: Middle Eastern/Arab-American, Middle Eastern/Arab, Middle Eastern, and Middle Eastern or North African. While the Middle East includes many Arab countries, it’s also home to non-Arab countries such as Israel and Egypt. The most equitable phrasing narrows down to either Middle Eastern or Arab or Middle Eastern or North African.

What we recommend:  Because both are inclusive, we looked at what the U.S. Census does. It uses the term Middle Eastern or North African. We recommend newsrooms follow suit with this phrasing.  

Multiracial 

For the surveys that did provide multiple-choice options for race and ethnicity, all had some way of incorporating an option for people who described themselves as multiracial: making the choices multi-selectable, having a separate choice to denote multiracial, or a mix of both. 

What we recommend: Multiracial can be covered by making the race/ethnicity question multi-selectable. We recommend against solely having an option on the survey that says Multiracial to prevent respondents from feeling like their race doesn’t actually matter. Having an option for multiracial on top of being multi-selectable will help in later analyzing the data, For instance, newsrooms will be able to make an assessment such as, 5% of all who responded as being white also chose multiracial.

Black vs. African American

Only one survey we looked at had Black as the only option. All the others also included the phrase African American. African American isn’t the sole option on any survey, though, which is all the better considering that using only African American omits Black Hispanics and Black people who are not from Africa. 

What we recommend: While the term Black seems all-encompassing, newsrooms should be as inclusive as possible and write the option in their surveys as Black or African American.

Hispanic/Latino vs. Latinx

Some newsrooms incorporated Hispanic, Spanish, or Latino Origin as a part of white — White (of Hispanic, Spanish, or Latino Origin) and White (not of Hispanic, Spanish, or Latino Origin). This, however, centers white as the main identity and excludes groups that aren’t white but do identify as being Hispanic, Spanish, or Latino. We decided to solve this by creating a separate choice for Hispanic, Spanish, or Latino, and making the question itself multi-selectable.

This way, those who identify as, say, Black Hispanics will be able to indicate as such by clicking Hispanic, Spanish, or Latino Origin and Black or African American. 

Newsrooms also have the option of separating this into a separate question. As for the term Latinx, most Latino adults haven’t heard of it and few use it.

What we recommend: Keeping a separate choice written as “Hispanic, Latino, or Spanish Origin” is standard and inclusive to use in surveys under the question of race/ethnicity.

Gender

The question

The best practice for asking about gender is taking the same approach when asking about race, “How would you best describe your gender identity?” One newsroom we looked at took it a step further by asking for current gender identity, recognizing that it might be different from the gender a person was assigned at birth. This is an important distinction to be aware of, and we will acknowledge it in our surveys going forward. 

Some newsrooms also asked for pronouns, either to supplement the gender question or to replace it altogether. While it is an option for newsrooms, we believe asking solely for gender identity is sufficient when the purpose of the survey is purely for source auditing.

What we recommend: We believe the most inclusive way of asking this question is the following: “How would you best describe your current gender identity?”

Transgender

Overall, newsrooms tend to separate male and female identifiers to Female, Male, Transgender female, and Transgender male when they ask about gender identity. 

We found two problems with this: People who identify as transgender want to be identified as the gender they transitioned to. For instance, someone whose current gender identity is female will want to indicate that they are female, rather than transgender or transgender female. Second, putting these options into a question about gender has the additional effect of treating cisgender as the norm. 

What we recommend: Newsrooms should omit choices about identifying as a transgender person on the gender identity question. Instead, break it up into two: Ask first about gender identity, and then whether the respondents identify as transgender. (Note: Breaking up the gender identity question has also been supported by multiple universities that post best practices on their site — Vanderbilt and UCLA, just to name a few — as well as the CDC.) 

Non-binary, gender-nonconforming, etc.

The key takeaway here is that having male and female as the only options just isn’t enough.

But similar to race/ethnicity, it’s difficult to be inclusive without incorporating a whole list of the various gender identities people can have. Including the most common genders in the U.S. while allowing for a write-in option is likely the most inclusive newsrooms can get in a limited multiple-choice question.

What we recommend: Non-binary should also be included as a choice. Those who identify as a gender outside of these three can use the Other type-in option if using Google Forms or a “prefer to self describe” section to write in their preferred term.  

Unsure

While it wasn’t popular to include some variation of Unsure or Questioning for gender identity across the newsrooms interviewed, we felt that it was important to acknowledge that there are some people who still don’t know or are exploring what gender identity they are most comfortable with. 

What we recommend: Newsrooms should include an option on the survey for people who are questioning. The choice itself can be written as Unsure, Questioning, or a mixture of both. 

Other survey questions

Region, expertise and age

We realized that asking for race and gender alone won’t be able to catch all the blind spots in our sourcing. Asking additional questions will provide more information on the diversity of our source. 

At Chalkbeat, we want to use our source auditing survey to make sure that the individuals we used as a source are representative of both the specific bureau and the national population. This would be impossible without knowing which bureau the source is from. Another example: A newsroom that reports on a single state would want to make sure all its sources who identify as Black or African American don’t come from the same county. Using the same reasoning, expertise and age are also important questions to better understand the variety of sources being used across your newsroom. 

What we recommend: Asking where your source is from, what their profession is, and what age group they’re in provides a fuller image of a newsroom’s source diversity. 

Feedback

Even in our quest to recommend the most equitable language we can, we know there are areas respondents may disagree with. Regardless of the final language newsrooms choose to use, allowing a space for sources to critique and ask about language choices will be beneficial in the long run. 

What we recommend: For the last question of your surveys, ask for feedback on the language used, and use any responses to inform future changes.

Conclusion

In reality, the most inclusive way to write a source auditing survey is by leaving all of the questions open-ended, allowing for complete self-identification by the respondent. For those analyzing the survey, one drawback to this is the time it will take to sort out responses at the end⁠ — the opposite of what we wanted our source auditing survey to do. Standard survey platforms like Google Forms or Typeform also don’t allow for write-in options within each choice, for example: 

Which of the following best describes your race/ethnicity?

Asian: (Please specify here)
Middle Eastern: (Please specify here)
American Indian or Alaskan Native: (Please specify here)

There are, however, several ways to get around these obstacles:

1. To allow people the chance to specify their identity further, you might consider adding this question at the end of the survey: Is there any other demographic information you would like to share with us?
2. If you plan to use a platform like Typeform where there’s only one question on the screen at a time, programming in follow-up identifying questions based on what choice respondents pick is worthwhile.
3. By making choices multi-selectable, respondents don’t feel boxed into any one category and it gives them the liberty to choose the identities that best fit them. 
4. The use of the Other write-in option will allow people to self-describe if the given options for race and gender truly do not describe any part of their identity. If newsrooms want to avoid listing an Other option, which can feel alienating, an alternative would be to create another question asking respondents to self-describe.