Efforts to reform higher ed will founder on bad data

By:

Sep 29, 2016

Despite the current unproductive political climate, a sensible and pragmatic bipartisan proposal to reform higher ed has been put forward in Congress. The bill, co-authored by Senators Chris Coons (D-DE) and Johnny Isakson (R-GA), would require elite schools to admit more low-income students; it would also require underperforming colleges to improve their graduation rates. But, as always, the devil is in the details, and it’s unclear if the tools to implement such a proposal even exist.

College graduation rates are often described as stubbornly low, with six-year completion rates hovering just below 60 percent. Private schools fare somewhat better than the average, with six-year completions slightly above 65 percent; for-profit schools are much worse, with six-year completion rates averaging only 27.8 percent for the cohort that started college in 2008. This measure, however, also varies widely by selectivity: non-selective schools are almost as bad as for-profit ones in terms of completion rates. Selective schools, on the other hand, have completion rates approaching 90 percent.

Six-year completion rates by institutional selectivity

figure-ctr-2

From the National Center for Education Statistics.

Although these numbers may at first blush seem to be a sweeping indictment of U.S. higher ed, the picture is complicated by what the numbers measure—and what they don’t.

What they measure

  • First-time, full-time students who complete bachelor’s degree programs within six years at the institution at which they started.

What they don’t measure

Given these huge data gaps, it’s tough to know which colleges really aren’t performing well—and which ones are. Incorporating transfer students into the data would likely make completion rates look a lot higher; adding in part-time students could make them a lot worse. For example, if College A were to accept a large number of transfer students but never graduate any of them, it’s completion rate would be unaffected. But if College B were to enroll a lot of students who earned credits cheaply before transferring to College C to complete their degrees, College B would get dinged. Other schools that could get hurt? Those that try to enroll students that need additional remediation—graduation grates fall precipitously for students who need remedial courses. It’s possible that highly selective schools have high graduation rates because they are good at what they do—but it’s more likely that having the most academically prepared and well-resourced students has a lot to do with it.

In the Gates Foundation’s recent report, “Answering the Call: Institutions and States Lead the Way Toward Better Measures of Postsecondary Performance,” which examines data issues and solutions in higher ed, the authors note, “Currently, higher education’s data ‘infrastructure’ is a set of disconnected systems, all of which were created for their own purposes at distinct points in time, but none of which are presently able to fully provide the answers we need to pressing questions about key student outcomes.” Beyond information about students’ progress through institutions, including credit accumulation, transfers, and graduation, we also need information about learning, earnings, and employment. We also need to recognize that the diversity of our higher ed system is a good thing; different institutions are playing very different roles, and different metrics apply. A graduation rate below 90 percent at an Ivy League School would be a sign that something was wrong. But for most schools, 90 percent is so far from reality, it might as well be a dream.

Senators Coons and Isakson are going after a real problem in higher ed. But it’s hard to see how punishing “underperforming” schools will fix the problem when there still isn’t decent data on college performance. The first step in holding higher ed accountable is to build better data. Ironically, the bill is called “self-financing” because it assumes that the fees high-performing schools would pay for their lack of diversity would provide low-performing, high-access schools with $8 million over five years to improve their graduation rates. It’s hard to imagine how a school could substantially move its graduation rate with only $1.6 million—but it’s even harder to imagine how we could get data to measure those graduation rates without spending any money. Congress has identified the problem, but solutions will take a little more effort.

For more, see:

Alana leads the Institute’s higher education research and works to find solutions for a more affordable system that better serves both students and employers. In this role, Alana analyzes disruptive forces changing the higher education landscape.