Wikis and More

(The following is excerpted and condensed from an op-ed in yesterday's Washington Post.)

In the past year, Wikipedia, the online encyclopedia that "anyone can edit," has been cited four times as often as the Encyclopedia Britannica in judicial opinions, and the number is rapidly growing. In just two years, YouTube has become a household word and one of the world's most successful Web sites. Such astounding growth and success demonstrate society's unstoppable movement toward shared production of information, as diverse groups of people in multiple fields pool their knowledge and draw from each other's resources.

Developing one of the most important ideas of the 20th century, Nobel Prize-winning economist Friedrich Hayek attacked socialist planning on the grounds that no planner could possibly obtain the "dispersed bits" of information held by individual members of society. Hayek insisted that the knowledge of individuals, taken as a whole, is far greater than that of any commission or board, however diligent and expert. The magic of the system of prices and of economic markets is that they incorporate a great deal of diffuse knowledge.

Wikipedia's entries are not exactly prices, but they do aggregate the widely dispersed information of countless volunteer writers and editors. In this respect, Wikipedia is merely one of many experiments in aggregating knowledge and creativity, that have been made possible by new technologies.

The Central Intelligence Agency disclosed the existence of its top-secret Intellipedia project, based on Wikipedia software (and now containing more than 28,000 pages), in late October. The agency hopes to use dispersed information to reduce the risk of intelligence failures. NASA officials have adopted a wiki site to program NASA software, allowing many participants to make improvements. . . .

But wikis are merely one way to assemble dispersed knowledge. The number of prediction markets has also climbed over the past decade. These markets aggregate information by inviting people to "bet" on future events -- the outcome of elections, changes in gross domestic product, the likelihood of a natural disaster or an outbreak of avian flu. . . . (If you want to learn who is likely to win the Oscars, check out the Hollywood Stock Exchange at http://www.hsx.com.) . . .

Of course, collaborative projects can go badly wrong. Pranksters have altered Wikipedia entries to say that Tony Blair's middle name is "Whoop-de Do"; that David Beckham was a Chinese goalkeeper in the 18th century; that the golfer Fuzzy Zoeller had abused alcohol and drugs; and that John Seigenthaler, a respected journalist, was thought to be involved in the assassinations of both Kennedys (before absconding to the Soviet Union).

The falsehoods about Zoeller and Seigenthaler were no laughing matter, and more serious mistakes, endangering reputations or causing financial losses, are possible. Anyone can vandalize an encyclopedia that "anyone can edit." No less than stock prices, prediction markets may be subject to manipulation. And in medicine and biotechnology, as elsewhere, intellectual property law may be needed to provide adequate incentives for innovation.

But the track record of the new collaborations suggests that they have immense potential. In just a few years, Wikipedia has become the most influential encyclopedia in the world, consulted by judges as well as those who cannot afford to buy books. If the past is prologue, we're seeing the tip of a very large iceberg.

Comments

[Eve Sheridan]

If only humans could work collaboratively on environmental issues then we would have a good chance of reversing some of the environmental damage that has been done.

[Michael Martin]

The connection to Hayek is useful -- more Web 2.0 folks should know about Hayek.

I've been thinking lately about how -- at a very high level -- one could argue that the printing press facilitated the development of representative democracy. An educated public is one contingency that needs to be met for such governments to take root.

I would love to see Prof. Sunstein do a followup post sharing his thoughts on how aggregated information technologies might be used in turn to facilitate new forms of government. Could we be constituted more freely if political decisions were made by aggregated information rather than by representatives?

Unfortunately, I don't know enough about Wikipedia to know just how much control is centralized into the hands of its organizers.

[Joan A. Conway]

I was once informed that nothing in print nowadays is a fresh idea! Having an aggregated information makes for a fast survey of what has occurred before and throws into conflict any current coinage used to mean a certain thing, like a Presidental Impeachment, as it pertained to our government over its history, in other words the reference lack historical context.

Having aggregated information denies manipulators of emphasizing its current coinage, because all will know what it meant when this country was formed.

[Martin Compare]

It really is interesting how quickly wikipedia and youtube have grown over the last 1-2 years. Something like this wouldn't have been possible without the internet. Truly amazing.

[Richard Bennett]

Wikipedia is all about process, and because its process is so different from Britannica's, it's not really accurate to describe it as an "encyclopedia". Wikipedia is actually the world's largest collection of trivia, gossip, received wisdom, rumor, and innuendo. It's valuable because any large collection of information is valuable, but not in the same way that the verified, expert summaries in an encyclopedia are valuable.

If it's true that the "knowledge of individuals, taken as a whole, is far greater than that of any commission or board," it's also true that the sum of their prejudice, mistaken beliefs, wishful thinking, and conformance to tradition is greater.

All of this is to say that group endeavors like Wikipedia produce breadth but not depth. For some endeavors depth is important, but for all others it's fine to consult the rabble.

Marketing, for example, can gain much by mining the dark corners of Wikipedia; engineering and medicine, not so much, as knowledge is not dispersed at the depths as it is at the surface.

Which brings us back to Hayek. Markets do a great job of bringing information about the wishes of buyers to bear on the consciousness of sellers. Everybody who participates in a market is an expert on the subject of his own wishes or his own product. But when you leave the realm of buying and selling, expertise is not as widely dispersed as participation, and then the decentralized model falls down.

[Michael Martin]

Richard,

Interesting comment. I can see how aggregated information is vulnerable to systematic errors in popular belief (this must be part of why gossip and rumor have a bad moral rep?). But please say more about why social networks are "valuable because any large collection of information is valuable, but not in the same way that the verified, expert summaries in an encyclopedia are valuable." If an Encyclopedia is valuable because subject to "verification" by a non-expert editor, who presumably has to rely on external indications such as the expert author's reputation and paper credentials, along with whatever clues about credibility can be gleaned by considering internal consistency of the expert's summary, then why should social networks be less valuable, social networks being also subject to verification in this sense, but perhaps by a far larger number of editors, and so probably (or at least potentially) a larger number of experts.

That being said, I think that anonymity leads to serious problems with information aggregation through social networks. Accuracy and privacy are in tension within this domain. But maybe they are in general. And like many under 30 today, I tend to think privacy is overrated, and protection of "privacy rights" (not the Griswold kind; the personal information kind) are sometimes a fig leaf for efforts to manipulate others' views.

That being said, I have to confess confusion over the last part of what you write: one of the big leaps of modern economics was the realization that markets can exist and function every bit as effectively without prices being explicit (see, e.g., most of Gary Becker's career).

[Richard Bennett]

Wikipedia is a valuable collection of information because of what it tells us about its editors, not because the content is reliable in relation to its alleged subject matter.

If you're looking for a record of the biases and obsessions of early twenty-first century computer geeks, it's the only place to go. If you're trying to learn about Britney Spears, it's a good place to go because it centralizes information that exists on a large number of fan sites and has links to all of them.

If you're looking for information on the dynamics of the Democratic and Republican Parties, it's not so great because it's going to say "Dems rule, Reeps drool" or words to that effect. The bias is very evident.

Some of the more obscure technical articles are good, depending on when you catch them in their life-cycle. Typically, the first edit by an expert is good, and then entropy sets in and an army of morons eats away at the quality until nothing much is left. Many Wikipedia editors can't write a grammatical sentence with a dependent clause.

So the lesson is that peer production networks can be too large; there is no need to extend the net beyond the circle of people who have expertise in the subject matter. The Wikipedia prejudice that any average citizen and read a text and determine whether it's supported by expert commentary is generally wrong. Experts are often overturned by other experts, so there's a time dimension to knowledge that Wikipedia fails to capture.

Markets are different, because every buyer knows what he's looking for and generally what he's willing to pay, and every seller knows that he needs to get for any item. I don't know what Becker has to say about pricing, that's my simple-minded summary of market behavior.

[Michael Martin]

Thanks for the response, Richard. It seems you may have spent more time reading posts on Wikipedia than have I. I have found it to be a helpful resource on many occasions. I must admit, however, that I have not looked up any hot-button political issues, which I know would be the most relevant to the potential for using Wikipedia as a tool for decisionmaking within gov't.

I belive that much of the "entropy" effect you describe is attributable to the anonymity of the editors. If the editors were identifiable, rather than entropy, we'd have "superposition," or aggregation. Bad information would tend to be weeded out and good information sustained. The larger the network, the better the information.

The issue with there being a time dimension to knowledge is a good one, and one in which Wikipedia is better than its predecessors as well -- older versions of an article are readily available for review, so "rollback" is easier. There will always be expert debates at the margin, but at least we don't have to wait until the next edition gets published to find out where the battle lines are drawn.

Also, I noticed today that Wikipedia is considering at least partially abrogating the anonymity that led to the whole Essjay debacle. For me this confirms both the tension between privacy and accuracy and the trend away from privacy toward accuracy within social networks -- when the contributors are forced to internalize the effects of their posts, they'll be less likely to make mistakes in the first place.

Gary Becker and others are responsible for the idea of implicit markets, i.e., markets that work without there being an explicit price. For example, the "marriage market" for a potential spouse has the characteristics of markets that you describe, but without explicit prices being negotiated. I think Wikipedia has some of the characteristics of markets that you describe as well. Every reader knows what he's looking for and generally what he's willing to pay (i.e., what risk he's willing to take that the information read is incorrect, out-of-date, incomplete, etc.). Every editor knows the value of the time he is giving up in order to gather and writeup information. The fact that the buyers and sellers on Wikipedia never meet in person doesn't matter. Buyers and sellers on the stock market never meet either. The big problem with Wikipedia is that unlike the stock market, there is nothing to deter fraud because the buyers and sellers can maintain (almost complete) anonymity.

[Richard Bennett]

I think you need to edit Wikipedia to fully appreciate its dynamics. You'll find that it's very difficult to correct errors that are evident to you in articles within your area of expertise. Each high-traffic article has a self-appointed owner with some sort of weird emotional attachment to the content, the phrasing, the authorities, and even the spelling. Butting heads with these people is inevitable, and that ultimately leads to tedious recitations of Wikipedia rules and an arcane dispute resolution process that never ends. The typical Wikipedia editor suffers from OCD, and the typical reader from ADD, so the two worlds barely intersect.

It's the proverbial sausage factory of ideas, losing its appeal as you learn how it works.

[AD]

Here's a recent entry on the Wikipedia entry on current Republican Party ideology:

"The Republican Party is composed of Fiscal Conservatives, Social Conservatives, Neoconservatives, Libertarians, Moderates, Evangelicals, Liberals (sometimes derided as Republican In Name Only, or RINOs, by more conservative Republicans), and Log Cabin Republicans.
"The Republican Party is the more socially conservative and economically libertarian of the two major parties. The party generally supports lower taxes and limited government in some economic areas, while preferring government intervention in others. In the 1980s, the Republican Party was more strongly libertarian. In his 1981 inaugural address, Republican President Ronald Reagan summed up his belief in limited government when he said, "In the present crisis, government is not the solution to our problem; government is the problem."[1] Since 1980, the GOP has contained what George Will calls "unresolved tensions between, two flavors of conservatism -- Western and Southern." The Western brand, wrote Will, "is largely libertarian, holding that pruning big government will allow civil society -- and virtues nourished by it and by the responsibilities of freedom -- to flourish." The Southern variety, however, reflects a religiosity based in evangelical and fundamentalist churches that is less concerned with economics and more with moralistic issues, such as opposition to abortion and same-sex marriage. Noting the waning influence of libertarian philosophy on contemporary Republican ideology, Will describes the current Republican Party as "increasingly defined by the ascendancy of the religious right."[2]"

It may not be perfect--for one, it quotes George Will excessively--but it's not "rabble" either.

[Fab L]

Something like this wouldn't have been possible without the internet.