MOOCs: Try, Try Again

What Is a MOOC? (Photo credit: giulia.forsythe)

by Carl Straumsheim, Inside Higher Ed: http://www.insidehighered.com/news/2014/02/18/stanford-u-mooc-instructors-trial-and-error-breeds-success

Third time isn’t necessarily the charm for massive open online course instructors, but through a process of trial and error, some faculty members at Stanford University say their MOOCs are living up to (some of) the potential promised two years ago.

As many instructors are finding out, teaching a MOOC is not that different from teaching a face-to-face course - at least the kind where you stand in front of a large auditorium, said Scott Klemmer, associate professor of cognitive science and computer science and engineering at the University of California at San Diego.

And since MOOCs have for more than a year commanded such a large part of the conversation about online education in general, instructors can more easily see what other instructors are experimenting with, he said.

“As we run these classes multiple times, a major question is: What is the role of the instructor?” said Klemmer, who is also a visiting associate professor of computer science at Stanford. 

“When you run it for the 10th time, how do you have it so you can be involved in the class at the level you want without you feeling like ‘Groundhog Day,’ where you’re doing the same thing over and over again?”

Keith Devlin, a Stanford researcher who is spending a semester as a visiting professor of mathematics at Princeton University, recently launched the fourth iteration of his Introduction to Mathematical Thinking MOOC. 

He said the number of students who persist through the first and second weeks has grown with every new version of the course. He declined to provide any numbers, since enrollment varies between the fall and spring semesters, but said the results mean he has finally reached a point where he has moved from “crude-tuning to fine-tuning.”

Yet through those four versions, Devlin said, the course’s content has remained more or less constant. Some of the lectures have been tweaked or edited, but the structure of the MOOC and the experience of taking it have become “radically different.” 

Devlin said he was inspired by massively multiplayer online role-playing games such as “World of Warcraft,” which have enthralled millions of players - himself included - to the point where they grind through repetitious tasks on the road to a reward.

“I’ve played hours of ‘World of Warcraft’ - in part because I saw [massively multiplayer online role-playing games] as a vehicle for education,” Devlin said. 

“They’ve made it virtually impossible to get the cool stuff unless you enter into at least temporary collaborations. MOOCs offer the opportunity to do the same thing - not in the fictional world of Azeroth - but in the real world of Stanford courses.”

The success of MMORPGs, Devlin said, is that they have managed to simultaneously be massive while encouraging players to form smaller social groups, or guilds. “It’s about communities, and so all of the changes I’ve been making have been about putting students in  a position where they feel comfortable in forming communities,” he said.

Right now, those groups form manually through the discussion forum, and, Devlin acknowledged, the most successful groups are often those where members meet in real life on a regular basis. 

He said he is exploring web-based tools that, in a future iteration of the MOOC, could pair students with like-minded study partners.

The most significant change, however, “is that the course is now explicitly about participation, not about getting a good grade at the end,” Devlin said. 

He has stopped short of giving students points for a task such as posting on the forum, saying he wants students to appreciate the rush of solving problems that at first glance seemed impossible. “What I’m looking for is intrinsic rewards,” he said.

Devlin said he has also changed what it means to complete his MOOC. To earn a certificate of completion in one of the previous versions, students had to score 60 percent or higher on a final exam. 

Now, students who are consuming the lecture materials and interacting with their peers can still experience “valuable learning, even if they haven’t solved a single math problem,” he said. 

After the standard eight weeks, students are also invited to join a two-week “Test Flight” program, which challenges them to apply the “mathematically-based thinking skills” they have learned in the course to math itself.

Completing the “Test Flight” program awards an additional certificate, and Devlin also provides students who want to show how they performed in the course with a certificate showing which percentile they tested into. “If they get that piece of paper, that’s legitimate, that means something, and it’s up to them if they want to show it,” he said.

Despite his progress, Devlin still said he will continue to fine-tune his MOOC through several more iterations. “It’s just barely getting off the ground,” he said. “Where it’s going, none of us that are involved know.”

Klemmer recently finished the fourth session of his Human-Computer Interaction MOOC, and he compared his revision process to that of an automaker’s, with each new version bringing a round of tweaks and improvements. “Once or twice a decade, you will jump to a different point in the design phase,” he said.

Many of the major additional features in Klemmer's MOOC - including a community teaching assistant, a LinkedIn group for former students and the curation of a forum post of the day - were added in time for the second round. 

If one of the students in the original MOOC were to re-enroll, they would likely experience a more polished course, see some new assignments along the way and feel less burdened by deadlines, but essentially, "The class uses the same basic format and strategy as when I first offered it in the spring of 2012," he said.

They would also sit through the same video lectures, which Klemmer highlighted as one of the failures of how MOOCs are delivered.

"If you want to change the text for an assignment, it’s really easy," Klemmer said. "But if you want to change a video - even by one sentence - it’s a huge amount of work." 

Since MOOC providers were able to build the automatic evaluation and peer assessment systems required to teach a MOOC, Klemmer said he expects developers will focus on video editing tools next.

"I think the reason MOOCs came out of computer science is that we had the ability to build the tools to do the things that we need," Klemmer said. "I think there are big opportunities for rethinking from first principles what the interactive experience is like."