When Phillip Phillips won “American Idol” last Wednesday night, a team of researchers from Northeastern University was not surprised. The scientists had risen early that morning to analyze about 100,000 messages posted overnight on the social network Twitter and made the prediction that despite his competitor’s greater worldwide support, the eligible population — residents of the United States — would vote Phillips to victory.
Forecasting who will win a TV contest may seem trivial, but the researchers saw it as a lighthearted way to test approaches being developed for more serious uses: how to forecast complicated social events from large datasets.
As social media have grown in popularity and smartphones and wireless access have become ubiquitous, major events like the Arab Spring or the outcome of political elections appear, often in retrospect, to have been foreshadowed by activity online. Interest has grown in figuring out how to use data to augur the future, whether it is the ups and downs of the stock market or the spread of disease. But as the datastream has grown, so has debate over whether and in what circumstances accurate Twitter predictions are feasible.
“Can we, at the end, predict the outcome of elections? Can we understand if we’re approaching social unrest? Of course in all those cases, everything is very complex,” said Alessandro Vespignani, a professor of physics at Northeastern.
Not only are the social behaviors complicated, but the online data are messy, for reasons that can range from the sometimes narrow subset of the population generating the tweets to the substance of the postings, which can vary from sarcasm to genuine support.
For the group at Northeastern, analyzing the outcome of “American Idol” was a side project in a larger quest to understand phenomena such as how ideas catch on, how a virus will spread, and how consensus forms. Analyzing the controlled case of people choosing their favorite contestant allowed the researchers to understand which factors are informative and which are not.
In the case of “American Idol,” the researchers’ original technique would have predicted Jessica Sanchez as the winner. But there was an important caveat: When they looked at the origin of the tweets, they saw she had strong international support, especially from the Philippines. Because only US residents can vote, they predicted Phillips would win — unless people abroad found a way around the voting restrictions. That experience suggests how such data can mislead, unless it is carefully examined.
The question of how much insight the deluge of data from online activity can provide about real-world behaviors remains open.
“Predicting X from Twitter is a popular fad within the Twitter research subculture. It seems both appealing and relatively easy,” Daniel Gayo-Avello, a computer scientist at the University of Oviedo in Spain, wrote in a recent paper that was skeptical of claims Twitter could predict election winners. “This is not only an interesting research problem, but, above all, it is extremely difficult.”
Gayo-Avello points out in his paper, published this month, that when people talk about the power of Twitter as a predictive tool in elections, they are selecting examples with the benefit of hindsight and therefore often excluding elections in which predictions based on tweets would have been wrong.
Other researchers have reported successes in using such analysis as a predictive tool.
In 2010, researchers from HP Labs, the central research organization for Hewlett-Packard, analyzed nearly 3 million tweets that referred to two dozen movies from November 2009 to February 2010 and found they could successfully predict which movies would do well at the box office, outperforming more conventional techniques, including the Hollywood Stock Exchange, a Web-based virtual market which pins predictions on the prices of movie stocks.
Another team, led by Johan Bollen at Indiana University, assessed the mood of tweets to see if they could be used to predict fluctuations in the Dow Jones industrial average. The work, published in the Journal of Computational Science last year, found that shifts in the calmness of Twitter users were often reflected later in market ups and downs.
Ultimately, Vespignani hopes to build models that explain how consensus forms and even how scientific knowledge emerges and catches on.
“What the Web is doing,” Vespignani said, “is creating a kind of human collider in which you have all the interactions of people and traces of interaction, the crumbs we leave on the Web, on Twitter.”