« Conditional Prediction Markets, Presidential and Otherwise | Main | Video: Mark Heyrman on Involuntary Commitment to Mental Hospitals »

January 10, 2008

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d8341c031153ef00e54fdb01e58834

Listed below are links to weblogs that reference Today's Availability Cascade:

» When Do Prediction Markets Beat the Common Wisdom?: from The Volokh Conspiracy
Over at the Chicago Faculty Blog, Cass Sunstein has an interesting post on prediction markets. Commenting on the broad coverage of the fact that prediction markets didn'... [Read More]

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

bcowan

I agree that the occurrence of statistically improbable events should not be taken as discrediting prediction markets.

Their predictive power arises from concentrating enormous numbers of inductive judgments. The resulting probabilities (on which the "prices" are based) are statistical in character, because they express what most judging subjects take to be salient factors for the particular case in the general outcome-producing operations of the contemporary world.

It does not seem wise to ignore this information, since it takes account of what is, to the ordinary observer, otherwise unattainable depth of resources and breadth of perspectives.

But the occasional failure to judge rightly from all these concentrated perspectives raises the question of how it is that they go wrong when they do. What is the information about the contemporary world and its operations that is sometimes hidden from these clouds of witness? What is it that persuades many to vote contrary to the signals emanating from the same broad environment that prompt the statistically predominant judgment?

This is a different question from whether an unusual failure of accurate prediction casts doubt on the efficacy of the practice.

I think there is a kind of probability judgment that is not essentially statistical but is rather based on the intrinsic appeal of a new potential predicate to the judging subjects. Something happens that reveals a new factor beyond the usual or recent patterns of potential value to be realized. Often it is something vivid that changes the picture in a way that obscures, perhaps temporarily, the massive regularities that have pointed in the usual directions.

The market may simply not have been able to process the meaning of some genuinely new or anomalous pattern. In this case it might have been the wet-eyed Hillary episode or, as one professional pollster pointed out in an op-ed in the NYTimes this morning, a resurgence of race-based aversiveness to Obama once he was leading.

What this shows about prediction markets is not that nobody knows anything, nor that the regularities of patterned processes on which they depend are not really there somehow.

But it does show that markets emphasize data that are effectively in the past and depend for their predictive accuracy on some realizations of patterned values in objective events recurring in roughly the same way as they have played out before.

To me this is a more satisfying account than that "8% events occur 8% of the time," when there is no way whatsoever to show that what actually occurred was actually in that 8%. What occurred actually occurred and so there can be no statistical question relevant to it at all.

minderbender

bcowan expresses an idea I see a lot: "Their predictive power arises from concentrating enormous numbers of inductive judgments." This may be true, but it seems much likelier to be false.

Literally, the price is set by two people, a buyer and a seller - that's not an enormous number of people. More importantly, the price is not set by a process of supply and demand for that particular contract. If I decide to sell Microsoft stock, and nothing else changes about the world (expected profits at Microsoft, interest rates, etc.), then the price won't change even though the supply of Microsoft stock has increased (and this is true even if I sell massive amounts of Microsoft stock - temporarily the prices may fluctuate, but unless my sale of stock transmits new information about the fundamental value of Microsoft stock, it won't affect the prices for long). The price is actually set by the discounted expected returns on the asset in question.

So in all likelihood these markets don't aggregate information from random people. Rather, they reward people who bring information to the market. These are the people who move the prices and whose judgment is supposed to be so good.

At least, this is what I've been taught. The prediction markets might still be pretty good, but let's not pretend that they do something they don't.

Nigel Eccles

I think this ruckus over prediction markets getting it wrong is hilarious. Have these commentators never been to a horse race? I could see them all standing around muttering that the market got it wrong every time a 25/1 shot romps home!

This episode reminds us that most people find it really hard to think using probabilities. My concern is that the US's restriction on online gambling further dulls people's understanding. Gambling is the best teacher you can have in the concepts of probabilities, risk and return.

Joey Fishkin

Sure, the New Hampshire data point is no reason to throw out polls or prediction markets. That would be ridiculous.

Nonetheless, this incident does illustrate important limits of both.

Polls report sampling error, but not other sources of error (such as non-response bias and problems with the "likely voter" screen). When many polls say the same thing, as they did in New Hampshire, the aggregate sampling error is very small. That is to say, if sampling error were the only source of error, then 10-12 polls (which predicted a significant Obama victory) would be correct nearly all the time. As far as polls are concerned, this incident is illustrative because it is almost certainly _not_ a case of statistical variation, a 1%-likelihood-happens-1%-of-the-time sort of phenomenon. Some more systematic bias, or late change, generated error other than statistical error here (my bet would be something in the "likely voter" models, which are full of hand-waving voodoo). We pretend that polls are as accurate as the sampling errors indicate at our peril.

As for prediction markets, this incident illustrates once more that they are only as good as the data available to the market participants. If you ask a crowd to estimate the number of jelly beans in a jar, you can expect a very accurate average prediction precisely because each market partipant makes her own independent observations. If you ask a crowd to predict the outcome of an election going on in a state where very few of them live, they can make very few independent observations. Instead of being independent observers, the market participants are, by and large, independent interpreters of a common core of data consisting of the observations and opinions of other people (pollsters, journalists) that are posted on the internet. And this core may not be enriched by the large number of journalists and pollsters scrutinizing New Hampshire: the journalists talk to one another, reinforcing storylines that are always partial at best.

In other words, election prediction markets are fine if you want a quick numerical distillation of the conventional wisdom currently available on the Internet. Maybe the prediction markets even do a slightly better job than a single expert would do with the same data -- although I would bet that as you shrink the pile of data, this ought to become less and less true. In any event, when all are basically working with a common pile of observations -- observations that are limited and subject to known and unknown biases -- it really is just false to imagine that the magic of the prediction market trumps the flaws in the data, especially polls, that we're all working with.

Kimball Corson

Scuttlebutt is the poll results were in before Hillary did her tear up scene which brought out a larger than expected women vote in New Hampshire. Obama largely got what the polls predicted because he did not have a last minute tearful break down. Statistically, no one got to factor in the tears.

Sean O'Hara

I don't think the market was as wrong as people are saying. Yes, Clinton did win the popular vote, but it's the delegates that really matter, and in that she only tied with Obama.

JohnLopresti

Statistics are a viable tool. Consider the intolerable 1% doctrine Cheney has espoused and about which the author has written. But there are political reasons for Cheney's opting for the percent doctrine. In the NH matter, my personal guidance supports what one commenter said about the gender factor adding to HC's margin but insufficiently to accout for the shift. The horse race paradigm while delightful nevertheless ignores the cerebral considerations voters incorporate which are beyond the ken of mere 2-year-old equestrians chomping at the bit to excel in competitive exercise; so, there should be a dissociation in our willingness to accept this as a completely homologous fit to political races. Speaking of which latter, regrettably, having lived some time during my young years in New England or thereabouts, there is an ethnic bias well known to be carefully hidden in poll responders' answers, resulting in another minor shift imbalancing the prognisticators' adumbrations preelection in this instance. There is another factor, ongoing; NH evidently has a new statute allowing it to sequester and not recount ballots first rejected as uncountable; plus, the most prominent tally electronic apparatus in use in NH is the same brand and production model known for erratic results in several tests. Here I will ascribe a countershift toward Obama; in fact, in one partly recounted urban area the recount showed a 10% shift away from the first announced result for Clinton.

In sum, politics at its most intense in the granite state; a drama ongoing.

There are other dynamics between the leading candidates probably best left for a less probability and statistics oriented thread. But I think we have yet to hear the entire story from the NH primary, and that the present moment is an attuned time during which vote strategists are seeking public fora to characterize NH's results as configurations quite disparate from what actually occurred. I will omit mention of which horse I am rooting for in that race.

Joan A. Conway

It appears to be a race between a thorough bred by the name of Mitt Romney, and a disused pommel horse by the name of Hillary Clinton, as far as individual political parties are concerned.

However, the contest is really about the political parties between the Republican Camp and the Democratic Camp, both experiencing faction strife, like the private war between Alexander Hamilton and John Adams, and Aaron Burr and Thomas Jefferson in the election cycle of 1800. My bet is still on the Republican Party to garner the electorial vote to win the Presidency. The Democratic Party's strong political machine has an extension problem in mid-america.

The comments to this entry are closed.