Trusting the Numbers?

Is Gallup Spinning Survey Data to Prop Up an Abortion Narrative?

Several news stories recently have trumpeted the finding that college students care about access to “reproductive health services”—code for abortion. Currently enrolled and potential students care so much about such services, we are told, that access to, or lack of access to, “reproductive health” would even sway their decisions about what school to attend.

“Most adults value reproductive rights when deciding where to go to college, poll shows,” hails CNN. “Abortion Bans Are Causing Students To Radically Rethink Their College Plans,” says Women’s Health. “Abortion Access Is Affecting Where College Students Decide to Study,” according to Time. All of these stories link to a specific study—the State of Higher Education 2023, by the Lumina Foundation and Gallup, or this follow up report.

The problem is, the State of Higher Education 2023 itself nowhere says anything at all about reproductive healthcare. The questions instead pertain to such practical things as finances, student loans, reasons for attending higher education (to get a better job, make more money, etc.). I challenge you to download the actual document and take a look for yourself—do a keyword search, browse the table of contents, or just carefully look over each page. “Reproductive care access” is nowhere in the report.

Digging Deeper

Instead, the news items seem to be launching primarily off of  the follow-up Gallup report. According to this report, written by staff at Gallup, data from the State of Higher Education 2023 show that 72 percent of enrolled students say reproductive health laws in the state where they are enrolled are “somewhat to very” important to staying at the school. Similarly, 60 percent of unenrolled, non-degreed U.S. adults ages 18-59 claim that reproductive health laws are important when evaluating which school they might want to attend. The report also claims to find that party lines are not terribly important, as “majorities of enrolled independents (71%) and Republicans (62%) also agree [that reproductive health access] is an important consideration. Similarly, 76% of currently enrolled women and 62% of unenrolled women report these laws either do or would influence their college enrollment . . . ”

The family or friend of someone considering college might doubt such claims. For most, things like cost, availability of financial aid, proximity to a home or work environment, availability of a chosen course of study, etc., are primary factors when considering when, where, and how to go to college. The State of Higher Education 2023 itself even states that financial barriers loom as the greatest threat to those considering enrolling in higher education. For those currently enrolled, emotional stress and mental health rank as the top reasons they have considered dropping out—many cite the difficulty of balancing work, family, and school. But no matter which group you look at, cost remains a significant issue, and again, reproductive healthcare access is nowhere mentioned.

Selective Data Reporting

So where does one find the raw data and questions about reproductive healthcare? The best way to examine surveys and their responses is to get the whole context. Which questions, precisely, were asked? How were they asked, and in what order? All these things weigh heavily on how someone might answer a question about reproductive healthcare.

Take a look at the example graphic below.

The questions asked were “How important is each of the following characteristics of your college in your decision to stay enrolled until you have graduated?” and “How important would each of the following characteristics of a college be in your decision to enroll?” Clearly, there was a list of factors survey respondents were asked to evaluate in terms of which was most important. But we’re not allowed to see the whole list. If Gallup put “reproductive healthcare access” on a list with things like “how cool the student center is,” presumably more people are going to rate something “serious” like the healthcare question as more important. Presumably, the list isn’t that ridiculous, but we simply can’t know, because Gallup won’t let us see it.

Unknowns

The other thing that we don’t know is how this year’s answers may have differed from those of previous years. This is a fairly new survey, begun only during the pandemic, in 2020. Furthermore, CNN reports that “Questions were added with the recent developments on access to abortion”—i.e., last year’s overturning of Roe by Dobbs. The stories seem to imply some great effect, that college students are actually changing their minds on where to go to school, depending on state laws surrounding abortion. But we have no way of knowing if this is in fact true. It’s possible that if these questions were asked ten years ago, the result would have been very similar. The news coverage implies that more restrictive abortion access is causing students to make changes to their behavior, but these surveys themselves don’t give that evidence.

The other unknown is what went through students’ (and potential students’) minds when they read “reproductive healthcare.” Technically, this term could cover such things as availability of contraception, free condoms in dorms, or even prenatal or postnatal care. Reproductive healthcare covers a wide swath, and unless the surveyors clearly define the terms, they can’t claim that students’ answers are in response to abortion legislation.

Cause for Healthy Skepticism

In short, the version of the State of Higher Education 2023 that is available for public download doesn’t give us the whole story. Instead, the researchers at Gallup parse out pieces of data from the larger report that they’d like to present to the public in a very specific (ideologically driven?) way, without allowing the public to have access to the entirety of the survey.

We should be wary of any kind of reporting that demands we pay attention to one part of a picture, without allowing us to see and examine the whole picture for ourselves.