Professors take lessons from online teaching

Researchers mine a trove of data from courses offered free on Web

Pedro Brito discussed a physics problem with MIT professor David Pritchard. At right is Kira Street. Pritchard is taking lessons from his MOOC class to the classroom.David L Ryan/Globe Staff

CAMBRIDGE — David E. Pritchard has dedicated his life to physics, conducting pioneering work in atom optics and mentoring Nobel Prize winners at the Massachusetts Institute of Technology.

But now Pritchard, whose aviator glasses and flyaway gray hair give him the look of the quintessential MIT professor, has dropped his physics research “cold turkey,” as he puts it.

A new frontier of human knowledge has captivated him and others in academia: studying how people learn and finding ways to teach more effectively.

Fueling their enthusiasm is the explosion of massive open online courses, or MOOCs, the new species of free classes prestigious universities are offering to students around the world. As educators debate what the classes mean for the future of traditional universities, one thing is clear — they provide a vast laboratory to study learning, using a trail of electronic data to examine what resources or study habits best help students, whether they take courses online or in traditional classrooms.

The data from just one online class, Circuits and Electronics, the first offering from edX — the nonprofit founded by MIT and Harvard University to develop MOOCs — have already spawned multiple research papers by Pritchard and others.

The class enrolled more than 154,000 people in spring 2012, although only a small fraction completed the course. Everything they did was recorded: all 230 million times someone clicked on course materials. Students posted more than 100,000 comments on class discussion boards.

If you printed all that data out, Pritchard said, you would have 110,000 books.

“When we do research on campus, I have no idea whether the student bought the book, read it, when they read it . . . did they stay awake in lecture?” Pritchard said. “Now, I have all of these things.”

For generations, students have lamented that teaching is a neglected art on elite campuses — they are called research universities for a reason. Traditionally, tenure at most colleges has been based on professors’ scholarship, with in-class performance an afterthought at best.

MOOCs are helping to change that. They are turning charismatic lecturers like Harvard’s Michael Sandel into international stars, though also sparking fears that a few famous faces on a screen might endanger the jobs of lecturers at lesser-known schools. But even beyond the superstars, the online classes have made teaching seem newly edgy and created what edX president Anant Agarwal calls “learning’s Big Data.”

Agarwal, a computer scientist at MIT who taught the Circuits and Electronics class, confessed that he read his first research paper about learning only a year ago, a quarter century into his teaching career.

“For 25 years, I came in and walked into a class and saw 100-odd students sitting in front of me, and I was like a deer in front of headlights,” he said. “What MOOCs have done, if anything, they have really brought education and learning to the forefront.”

On college campuses, the main tool to study how much effort students put into a class, and what they got out of it, is the end-of-the-semester survey. But given the often-poor response rate and the vagaries of trying to estimate one’s own time spent studying, the results are of questionable value.

Now, the data clearly show that students rarely used the online textbook in Agarwal’s circuits class, except just before or during tests, which were open book. That raises fundamental questions about whether textbooks are serving students well, Pritchard said.

Agarwal said he was fascinated to discover that half of his edX students started working on their homework before watching the video lectures. If students get more excited to learn when they are trying to puzzle out a problem, he said, maybe on-campus classes should assign homework before the lecture, instead of after.

Researchers are also designing controlled experiments within MOOCs in which different students see different material, allowing their performance or satisfaction to be compared.

One experiment asked some students enrolled in Harvard’s class on ancient Greek heroes to pick someone outside the course to hold them accountable. If they start lagging in their work, the friend gets a message. If those students do better, it would point to an easy way to keep people on track.

Coursera, the for-profit provider of MOOCs that offers classes from Stanford, Yale, and other elite institutions, has conducted dozens of experiments with its university partners.

One study indicated that Coursera students were turned off when they got an e-mail reminding them about homework deadlines. But if they got an e-mail congratulating them for turning in their assignment — and only then reminded them they had a new assignment — students became more engaged, according to Andrew Ng, a Stanford computer scientist and cofounder of Coursera.

“I think it’s nice to be given a report reminding yourself that you did some good things this week, before bugging you to take the next step,” Ng said. “The reason that Google, Facebook, and Amazon can serve their users so well is in large part because of the data they have accumulated. With the data that we are collecting in online education, I would like to make education much more data-driven.”

Some researchers caution, however, that more data do not automatically mean better results. In an online class with diverse students from around the world, there is a danger of misunderstanding who is learning what.

Perhaps students who pick up the material easily already have a strong background in the field. Maybe someone who appears to spend a long time looking at a certain resource has the screen open while doing something else, said Isaac
Chuang, another MIT professor studying MOOCs

“As in any well-designed experiment, there must be the possibility of failure,” Chuang said. Otherwise, “you’re not pushing the envelope enough.”

But some professors say they are already applying some of what they’ve learned from online classes to what they’re doing in traditional classrooms.

Pritchard and his postdoctoral fellows have been refining a physics class on mechanics that was developed on campus and then offered on an older online platform. This month, it debuted on edX.

Students in the classroom, as well as those taking the class online, use an online “e-text” with questions embedded in the lessons. Frequent questioning prods students to retain more information, according to recent research.

The online materials freed up Pritchard from the need to lecture in his spring MIT class for students who had done poorly in introductory physics. Instead, students spent almost all their time at the blackboard, working in twos or threes on problems with Pritchard and his postdoctoral fellow helping.

Even in the last week of classes before the final, some of Pritchard’s students had a tentative, puzzled air. But Pedro Brito, a freshman from Puerto Rico, said the class cured him of his dread of physics. The online material, together with the abundance of time at the blackboard, helped him learn to grapple with the underlying concepts instead of blindly guessing at the right formula to plug in.

“Sometimes I thank God that I failed” the intro class, he said. “Being in this class . . . I actually learned how to love physics.’’