Why We Need to Change the Way We Evaluate MOOCs

Illustration saying "EdTech ABC – brought to you by the term 'MOOC'"

It’s easy to find articles criticizing the effectiveness of MOOCs, and we’ve sufficiently covered the topic, pointing to the low retention rates and the failure of Udacity’s partnership with San Jose State University.

Rather than harping on the negatives, it’s important to consider what MOOCs (which stands for Massive Open Online Courses) have accomplished – even if that means redefining the measures of success – in order to improve the field of online learning.

A series of working papers released by MIT and Harvard University offers detailed analysis of 17 online courses using the edX platform, drawing from student metrics and interviews with faculty. Here’s a summary of the key takeaways:

1. If small percentage of students finish a course out of a class size of several thousand, that is still a significant number.

Although up to 90% of students may drop out of a given MOOC, that leaves 10% who see it through to completion. When enrollment is in the tens of thousands and includes students from all around the world, there could potentially be hundreds of people from developing nations that would otherwise lack access to a quality university education.

Although Bill Gates scoffed – and rightly so – at Mark Zuckerberg’s statement that providing Internet connectivity to 5 billion people is “one of the greatest challenges of our generation,” those efforts would certainly improve the online education experience internationally. This article from FastCompany describes the impact of educational technology throughout Africa, where the most successful efforts use a blended model that brings together small groups of people to discuss online content. (That’s what we at Worldview Stanford have always believed too, which is why we feature an on-campus experience for participants to synthesize the material they’ve learned online at their own pace.)

2. Online registration and completion dates are not constrained by an academic calendar, so they can’t be compared to university retention rates.

Most MOOC attrition happens in the first week or two, but it decreases substantially after that. Not to mention that some students sign up months in advance or a few days before the final assignment submission date, and some return to the course even after it has ended.In a typical college course, the deadline to enroll or to drop a course is clearly established. Nonetheless, wait-listed or merely interested students may audit the course if the professor allows it – and in the case of MOOCs, a significant portion of course registrants could be considered “auditors." Which leads to the final takeaway...

3. Even if most students don’t earn a certificate of completion, a large percentage still visit the course page and view material.

Following a syllabus and submitting weekly assignments may be valuable for some people, but others may just want access to high-quality content on topics that interest them. After all, the majority of MOOC users already have a higher degree, so it’s likely that many enroll in courses out of curiosity or a lifelong love for learning, rather than a need to add certifications to their resumes.

“Course” may not be the right term to use – if viewed as an online library of content to browse as opposed to a sequence of lessons and assignments to complete, then millions of registered users can hardly be considered a failure.

What do you think about these findings? How else should we be evaluating the success of MOOCs, aside from enrollment and course completion rates? Are there other ways in which MOOCs can be beneficial for certain populations? Let us know in the comments below!

Crisis Text