Can an LLM trade a prediction market?

May 2026

Prediction markets like Polymarket and Kalshi are, at bottom, crowdsourced probabilities. A price of 63 cents is just the crowd saying it is about 63 percent sure. A wave of recent papers asks the obvious next question: can a language model read the same news, form its own probability, and trade the gap whenever it disagrees with the crowd? For whatever reason this tipped from a curiosity into a small flood of work in early 2026.

The loop these papers automate.

A quick tour of what is actually out there. PolyBench (arXiv 2604.14199) is the serious attempt at whether they can really make money: a benchmark of roughly 38,000 binary Polymarket markets, each snapshot locked to its order book and a news stream, then scored on directional accuracy, a return that accounts for confidence, and Sharpe. PolySwarm (arXiv 2604.03888) runs fifty model personas that vote on a probability and bolts on a latency module that gets ahead of stale prices; it reads more as an engineering flex than a forecasting result. And the paper the others lean on, Halawi et al., Approaching Human-Level Forecasting with Language Models (arXiv 2402.18563, NeurIPS 2024), builds a retrieval system that comes close to the human crowd. Worth flagging that this last one is judgmental forecasting, not trading.

Here is what I actually think. The framing everyone reaches for is whether the model can beat the market and turn a profit. To me that is the least interesting question, and also the one most likely to disappoint. A liquid market is a genuinely hard thing to beat, and I suspect most of the reported edge lives in latency tricks and thin, barely traded markets rather than in anyone reading the world better than the crowd does. Beat a market that nobody is trading and you have shown almost nothing, and the headline will always be the backtest that happened to work.

The part that holds my attention is buried in PolyBench’s setup, almost as plumbing. The order book is a time series. The news is text. Trading well means fusing the two under a strict clock, where being right slowly is the same as being wrong. That is a multimodal forecasting problem wearing a betting problem’s clothes, which happens to be the shape of thing I spend my thesis on, so take my enthusiasm with a pinch of salt.

It also exposes a distinction the literature keeps blurring. Judgmental forecasting means reading the world and guessing whether some discrete event happens. Statistical time series forecasting means extrapolating a signal. They are different problems, and a prediction market sits awkwardly between them: the order book drags it toward time series, while the resolution criteria, did the candidate win, did the rate cut land, drag it back toward judgment.

Where prediction markets sit, uncomfortably in the middle.

So I am interested, and a little sceptical. I would want to watch one of these systems survive a few months of real money and real fee drag before I trust a single Sharpe figure off a backtest. My honest bet is that the benchmark outlives the trading: PolyBench will still be useful long after the returns it reported are forgotten. A clean benchmark is a quietly useful thing; a profitable bot is a claim that has to keep being true.