Unwise Crowds and the Oscars

Almost everyone thought that Brokeback Mountain would win the Oscar for Best Picture. This belief was so widespread that relevant prediction markets -- including Tradesports and the Hollywood Stock Exchange, the latter of which went 8 for 8 last year -- accepted it too. But almost everyone, including the prediction markets, was wrong; Crash won instead. What happened? And what broader lessons might be drawn for group behavior and for prediction markets?

Here's the first possibility. Groups tend to be right when all or most members are more than 50% likely to be right; if so, the likelihood that the group will be right, if it decides by majority rule, expands to 100% as the size of the group increases. This is the Condorcet Jury Theorem, which helps to explain the general success of prediction markets. (Most investors/bettors, or most dollars, are at least 50% likely to be right.) But the Jury Theorem supposes that people are making their judgments independently, and not simply following the crowd. Suppose, as seems likely, that many people believed that Brokeback Mountain would win simply because many people believed that Brokeback Mountain would win. If predictors, including predictors in markets, were merely following others, a key assumption of the Jury Theorem does not hold. And in fact, many groups blunder because their members follow one another, rather than revealing their private knowledge.

The second possibility is that for best picture, many or most people, using their own private information, were less than 50% likely to be right. When this is so, the likelihood that the group will be right, if it uses majority rule, falls to 0% as the size of the group expands! Condorcet himself emphasized that "prejudice" may lead to this unhappy result. Large groups of people, in many countries, are wrong on obvious questions of fact. The reason is that most individual citizens are less than 50% likely to be right. Something of this kind undoubtedly contributed to the widespread belief that Brokeback Mountain would win the Oscar.

Prediction markets do offer protection against both of these problems for group behavior. When there is an economic incentive (as on Tradesports; the Hollywood Stock Exchange does not use real money), people are unlikely to act unless they (think they have) reason for confidence in their views, which is at least a safeguard against errors. And if people have private information, they should be eager and able to exploit that information at the expense of those who are either following the crowd or simply wrong.

In this light we should notice, but not overemphasize, the mistaken prediction that Brokeback Mountain would win the Oscar. After all, the Hollywood Stock Exchange's favorites won in the other seven categories on the site. And a careful observer would have noticed that the Exchange's second choice was Crash -- and that it attracted significant betting attention. It should be most interesting to see how prediction markets do on issues of relevance to public policy, including the avian flu.

Comments

[Eh Nonymous]

A fun piece, but you totally lost me when you wrote that as the size of a group goes to Large, the probability of the group predicting the right Best Picture winner goes to zero if most people, using their own private information, were not 50% or more likely to be correct.

Falls to zero? In a field where one of five will win? The possibilities are discrete, not continuous, and you weren't talking about the nominations, were you?

[Rob]

I think the key to that statement is if the group uses majority rule.

[odderie]

Although HSX doesn't have an economic incentive for accuracy, there is definitely a prestige incentive that comes along with having a high bankroll with which to buy, trade, and sell your stars and movies and, ultimately, reach that leader board.

I do agree, however, that they (Hollywood and the voting members of the Academy) probably told the media what they believed others believed while thinking they were the only ones who voted for the non-Brokeback nominees.

Couldn't it also be a function of who is the voting body, though? They're mostly located in LA, and maybe they could more easily relate to the "Crash" complexities than they could to Brokeback. Perhaps it was a much closer race than the non-LA-based predictors anticipated.

[Chenyun Zhu]

"how prediction markets do on issues of relevance to public policy?"
Prevailing optimistic mental state and crow-dominated world are ongoing. However, they are oriented by phase-final statistics.

[v]

majority rule is irrelevant, it's about local maxima. and the local maxima shift further away from the right answer if a large percentage of the group is wrong.

[jvarisco]

What if the Academy picked Crash only because everyone thought Brokeback would win?

[JackD]

One thing that might be learned is that when one seeks to predict the results of a subjective judgment by a specific group of individuals, one might be well advised to forget about it.

[Phil]

Yeah. As lessons go, doesn't "Don't Believe The Hype" just about cover it? [cf. C.Ridenhour, It Takes A Nation Of Millions... (DefJam 1988)]

[Raw Data]

Just for the record, I called it correctly.

[Kimball Corson]

Raw Data,

That should not have been too hard. I made it through half of Brokeback Mountain before leaving. It wasn't the homosexuality that bothered so much as the cardboard characters, the endless pamoramic shots of sheep and the sense I got someone was using my purse and time to make a political statement. I haven't seen Crash, but with effort, I am sure I could have sat through all of it.

[Win on Tradesports]

One of the most exciting aspects of having a Tradesports
account is the in-running trading available on live sports
events, right through to the end of the game.

A market maker provides contract liquidity and the players
battle it out over every possesion, every field goal attempt,
every time a player goes to the foul line.

Here are two tips that I have found to be extremely valuable
when trading in-running events:

1. Enter positions in-running and die with the result. This is
really just another way of finding overlays, but it's much
easier to do after the game starts!

2. Don't sell your contract early if you bought it pre-game and
are winning! The math that explains why this is detrimental to
your overall results is fully explained in How to Milk the
Tradesports Cash Cow. But for now let me just say that you need
to get the full value on your winners in order to come out ahead
over the course of the season.

The exception would be if you are using a trading system with a
profit objective and stop-loss, as explained in the manual.

Happy trading,

www.winontradesports.com