How to improve outcomes data for online programs (opinion)

Even before the corona pandemic, a growing proportion of students were completing all courses online. In the fall of 2019, almost 15 percent of undergraduates and 33 percent of graduate students were enrolled in online courses only. As students and colleges alike have embraced online education, more students will likely enroll in fully online programs once the pandemic is finally over than when they started.

Online programs provide access to higher education for students who are unable to attend face-to-face courses due to work, family, or geographic challenges. However, previous research has found that students who attend face-to-face classes tend to do better in these courses than students enrolled in online courses. A study also concluded that students attending colleges that are primarily or entirely online see at most a minimal return on investment.

And little is known about the outcomes of a growing group of students: those enrolling in online programs at public and private nonprofits that have traditionally conducted in-person courses. Spurred by student demand for online offerings, competition from the for-profit sector, and the need to generate additional revenue, many traditional colleges have rapidly expanded their online offerings. Some of this has been accomplished with the help of online third-party providers (Online Program Management Companies, or OPMs) who offer a range of services to support new online programs, but some of this has also been accomplished by colleges using internal resources.

The long-standing debate about the value of online education is associated with recent scrutiny of OPMs by some prominent Senate Democrats Encouraging interest in learning about student debt burdens and postcollege earnings from students enrolled in online programs at traditionally in-person institutions. I have long been interested in program-level outcomes data and have written extensively on using federal data college scorecard and the Ministry of Education’s Integrated Post-Secondary Education Data System for research and accountability purposes. When a group of leading online third-party providers hired me to see if I could say anything about the value of their partners’ programs, I was keen to dive into the data to see what was possible. I received a list of their programs at partner institutions and set off.

The key to take away my research is that while the US Department of Education College Scorecard includes institution-level data on debt and income by field of study, it is not possible to separate the scores data from students enrolled in online and face-to-face programs. This is a problem with traditional institutions as it seems that colleges often launch online programs when they already have an in-person option. And these in-person programs are often substantial: Among first-time adopters of OPMs, the in-person programs are likely to draw as many or more students than the online versions.

To answer the questions everyone wants to know about the outcomes of students attending online programs at traditional institutions, and to better compare student outcomes from online and on-site programs, the U.S. Department of Education needs to make several improvements to the data they collect from colleges. The three main recommendations are as follows:

  • Make it clear when a college only offers a specific program online, rather than having both online and in-person options. As part of IPEDS data reporting are colleges currently in demand whether no, some or all of the programs within each code of the classification of teaching programs can be completed through distance learning (which in 2022 usually means online). This metric has value, but cannot distinguish programs that have both in-person and online options from programs that can only be completed online. A small change in data collection would allow only online programs to be identified.
  • Report the IPEDS data on the number of graduates by program separately for online-only programs and all other programs. There is no way to determine the percentage of graduates from online programs compared to in-person or hybrid programs. This makes it difficult to see how widespread online delivery models are and how traditional institutions have adapted their strategies. Universities have to do that report deals B. by CIP code, race and gender, so adding a measure of whether the program is fully online or not should be easy.
  • Report college scorecard debt and income data separately for full online programs and all other programs. Similar to the point above, the combination of data for online and in-person programs makes it impossible to tell how the two delivery models are performing. The disadvantage of reporting results by modality is a smaller sample size, meaning that results from some programs would not be reported due to privacy restrictions. However, this could be mitigated by combining additional cohorts of students to get a picture of student outcomes.

Improving the quality of college outcomes data should be an area on which both parties agree, as evidenced by the Obama administration’s introduction of the modern college scorecard and the Trump administration’s addition of program-level outcomes data. Providing information on graduate debt and income based on whether they enrolled in online or face-to-face programs gives students better information about their options and also gives policymakers a sense of how to approach new placement models and partnerships . But for now, despite significant public investment in data infrastructure and financial support for students, it is impossible to answer some key questions about higher education performance.

Leave a Comment