hayek's central insight, which he developed over a career and summarized most sharply in a 1945 paper, was that the fundamental economic problem is not "how do we optimize production given our knowledge" but "how do we make use of knowledge that is dispersed across millions of people and not available in its totality to anyone." the soviet planning problem was not, at root, that the planners were corrupt or indifferent, though some were. it was that no committee could gather and process enough information to allocate resources well across an entire economy. the farmer knows this season's crop conditions; the trucker knows road conditions in his region; the factory manager knows what his machines can do and what his workers are good at. none of this can be aggregated into a spreadsheet. it's too detailed, too contextual, too fast-changing.
prices solve this. when a drought hits wheat-producing regions, wheat prices rise. that price rise tells every baker and pasta manufacturer and livestock farmer in the world something important — wheat is scarce — without anyone needing to know why. they adjust. they find substitutes. they reduce waste. they make billions of decentralized decisions that roughly approximate the right response to the drought, mediated entirely by one number they can read off a commodity exchange. no central coordinator required. the information is encoded into the price and transmitted through the price signal to everyone who needs it.
one way to think about prediction markets is as an attempt to extend this mechanism to the domain of beliefs about the future. a poll tells you what people say they think will happen. a prediction market tells you what people are willing to stake money on. these are different quantities. a pundit can confidently assert that X will happen at zero personal cost; a trader who is wrong about X loses money. prediction markets create incentives for the most informed people — the ones with edge — to show up and reveal what they know through their bets. the price reflects not just the average of opinions but the opinions of the people who have been right before, weighted by how much they're willing to bet.
this is why the prediction market prices on the 2016 trump/clinton race (giving trump ~30% when polls showed 15-20%) and the 2024 trump/harris race (giving trump 60-65% when polls showed a dead heat) were more informative than the polling averages. some people had edge — whether from better models, better demographic intuitions, or just better calibration — and the market price incorporated that edge. the polls couldn't, because polls don't weight by accuracy of previous predictions.
the mechanism generalizes. imagine you're running an R&D division and you want to know which of your projects is most likely to succeed. you can ask the researchers, but researchers are optimistic about their own work (it's a human thing). you can ask management, but management is optimistic about the projects they've been championing (it's a different human thing). or you could run an internal prediction market: create contracts that pay out if a project hits certain milestones, let employees trade them with notional dollars, and watch the prices. the prices will reveal what people in the building who are close to the work actually believe, not what they're willing to say in a meeting. google ran experiments like this. the results were informative.
what makes this hard is that markets require two things: resolution criteria and liquidity. resolution criteria means the outcome has to be clearly definable and eventually verifiable — "trump wins the 2024 election" works; "AI becomes transformative to the economy" doesn't, at least not in a form you can trade. liquidity means there need to be enough participants, with enough skin in the game, that the price reflects more than one or two people's opinions. a prediction market with twenty participants on an obscure question is not producing a useful signal; it's producing noise. both requirements constrain where you can deploy the mechanism usefully.
but wherever you can satisfy them, you probably should. the information embedded in prices is hard to replicate by other means. committees can sit in a room and agree with each other at zero cost. traders can't — the cost of being wrong is real. that difference in incentive structure is what makes prices, in hayek's phrase, a "marvel." not because markets are magical or infallible. they're neither. but because they aggregate dispersed knowledge through a mechanism that is structurally biased toward incorporating people who know things, and that tends to produce better decisions than the alternatives.