The Penn State Problem

Ben posted last week on what I think of as the “Penn State Problem”. It’s a problem that only data nerds would spend much time thinking about, but it actually matters for parents and students. And it is important for the one, small piece of accountability (cohort default rates) that we have in our higher education system.

Ben does a good job of describing the problem in his post. I wanted to illustrate the problem with a couple charts, using Penn State as an example.

Penn State likes to think of all of its campuses as part of one, large, happy family and reports data to the Office of Postsecondary Education this way. As a result, it reports one student loan default rate–3.5%–for all of its 23 campuses. But what are the odds that this default rate is accurate for the more competitive State College campus, the medical college campus in Hershey, PA, and the Penn State–Dubois campus, which granted mostly associate’s degrees last year? Not very.

ES published a report in February examining causes of student loan defaults and found that two variables were consistently important in predicting a college’s default rate: the percent of Pell grant recipients and the institution’s graduation rate. So let’s take a look at these two variables across Penn State’s campuses. (We can do this because the Department of Education collects data by individual campus in its IPEDS survey – see Ben’s post for why this difference in data collection is incredibly annoying).

The chart below shows the percent of Pell grant recipients in each of Penn State’s 23 campuses–notice the wide variation:

If Pell Grant percentages are highly predictive of an institution’s default rate, then one would expect that the State College campus (far left) would have a very different default rate than the Shenango campus (far right). But, according to the Department of Education, they both have a 3.5% default rate.

Below is a chart of the graduation rates at each Penn State campus:

Again, it is highly unlikely that the 3.5% default rate is accurate for both State College (far left – 85% graduation rate) and Penn State-Brandywine (far right – 36% graduation rate).

To take this illustration one step further, let’s look at what the graduation rate would look like if Penn State reported graduation rates the same way it reports default rates. Because all of the data is rolled up, the much-larger State College campus likely dominates the default rate calculation, making it even less relevant to the many smaller Penn State campuses. If we calculated the average graduation rate this way, weighted according to the size of each campus, the average would be 67% (the red line). As the chart below shows, a 67% graduation rate isn’t representative of the vast majority of Penn State campuses.

The U.S. Department of Education recently added default rates to its ‘College Navigator’ website, which is intended to be a tool for students and parents interested in comparing colleges. By adding default rates, the Department of Education clearly thinks this is important information for parents and students to know. But for the Penn State campuses, and the many institutions like it, College Navigator adds the following caveat to the default rate number, “These default rates apply to all locations of this institution.”

They should also add “…and may be meaningless for the institution you’re looking at.”