Unwise Crowds and the Oscars
Almost everyone thought that Brokeback Mountain would win the Oscar for Best Picture. This belief was so widespread that relevant prediction markets -- including Tradesports and the Hollywood Stock Exchange, the latter of which went 8 for 8 last year -- accepted it too. But almost everyone, including the prediction markets, was wrong; Crash won instead. What happened? And what broader lessons might be drawn for group behavior and for prediction markets?
Here's the first possibility. Groups tend to be right when all or most members are more than 50% likely to be right; if so, the likelihood that the group will be right, if it decides by majority rule, expands to 100% as the size of the group increases. This is the Condorcet Jury Theorem, which helps to explain the general success of prediction markets. (Most investors/bettors, or most dollars, are at least 50% likely to be right.) But the Jury Theorem supposes that people are making their judgments independently, and not simply following the crowd. Suppose, as seems likely, that many people believed that Brokeback Mountain would win simply because many people believed that Brokeback Mountain would win. If predictors, including predictors in markets, were merely following others, a key assumption of the Jury Theorem does not hold. And in fact, many groups blunder because their members follow one another, rather than revealing their private knowledge.
The second possibility is that for best picture, many or most people, using their own private information, were less than 50% likely to be right. When this is so, the likelihood that the group will be right, if it uses majority rule, falls to 0% as the size of the group expands! Condorcet himself emphasized that "prejudice" may lead to this unhappy result. Large groups of people, in many countries, are wrong on obvious questions of fact. The reason is that most individual citizens are less than 50% likely to be right. Something of this kind undoubtedly contributed to the widespread belief that Brokeback Mountain would win the Oscar.
Prediction markets do offer protection against both of these problems for group behavior. When there is an economic incentive (as on Tradesports; the Hollywood Stock Exchange does not use real money), people are unlikely to act unless they (think they have) reason for confidence in their views, which is at least a safeguard against errors. And if people have private information, they should be eager and able to exploit that information at the expense of those who are either following the crowd or simply wrong.
In this light we should notice, but not overemphasize, the mistaken prediction that Brokeback Mountain would win the Oscar. After all, the Hollywood Stock Exchange's favorites won in the other seven categories on the site. And a careful observer would have noticed that the Exchange's second choice was Crash -- and that it attracted significant betting attention. It should be most interesting to see how prediction markets do on issues of relevance to public policy, including the avian flu.