Eight AI systems lose money betting on Premier League matches and fail testing.
General Reasoning tested eight artificial intelligence systems in betting during the 2023-24 English Premier League season . The experiment was released in April 2026. All evaluated models recorded financial losses. The best-performing system ended the test with an average balance of £89.035, representing a loss of £10.965 on the initial value of £100.000.
The models tested included Claude, Grok, Gemini, and GPT-5.4. Each system received virtual capital to develop betting strategies based on machine learning throughout the entire season. According to information released by Portal do Bitcoin , the study revealed significant limitations in the ability of AI systems to apply theoretical knowledge to practical financial decision-making scenarios.
Results of the evaluated models
The test was called KellyBench. The name refers to the Kelly criterion, a formula created in 1956 that determines the ideal bet value when there is an advantage over the market. The systems demonstrated the ability to recite the formula. They were unable to apply it effectively in practice.
Grok 4.20, developed by xAI, had the worst results. The system failed in all three rounds. It went completely bankrupt in one of them. It gave up mid-season in the other two attempts.
Google's Gemini Flash abandoned the test in two out of three rounds. The model placed a single bet of approximately £273.000. The bet was based on a three percentage point advantage in the historical win rate. The system lost the amount wagered.
Anthropic's Claude Opus 4.6 model recorded an average loss of 11% throughout the experiment. OpenAI's GPT-5.4 model lost an average of 13,6%. Running just one round with this model cost approximately US$2.012.
Statistical model from the 1990s surpasses modern systems.
A statistical model from the late 1990s, called Dixon-Coles, outperformed six of the eight artificial intelligence models evaluated. The older model operated with limited data.
The researchers observed: “Dixon-Coles is an outdated baseline from the 2000s that doesn’t utilize all available data nor does it consistently consider non-stationarity.” They added: “It is therefore all the more surprising that many high-end models, such as the Gemini 3.1 Pro, fail to outperform or match it on KellyBench.”
Assessment of the sophistication of the systems
The research team constructed a 44-point sophistication rubric with experts from quantitative betting funds. The rubric covered feature development, bet sizing, non-stationarity handling, and execution.
The Claude Opus 4.6 achieved the highest score among the models tested, with 32,6%. This result represents less than a third of the available points.
Higher sophistication scores predicted statistically significant lower failure rates (p = 0,008). The scores correlated with better overall returns. The systems did not fail because the market is unbeatable. They failed because they did not adequately utilize their available resources.
Examples of operational failures
GLM-5 produced three self-criticism documents during its participation in the test. The documents correctly identified that its coded draw rate of 25% and overestimation of home-field advantage were detrimental to its returns.
When his balance was at £44.200, the model observed that his prediction of 40% home wins was only achieving 30% in practice. The system never changed the code. It continued betting in the same way until it exhausted its resources.
Kimi K2.5 developed a mathematically correct fractional Kelly allocation function. The function had the proper formula and structure. The model never executed the function.
A formatting error caused the system to send a faulty bash command approximately 50 times consecutively. The model's reasoning detected the problem. It sent the broken command again. An unintentional bet of £114.000 — 98% of the remaining balance — on a match between Burnley and Luton ended his participation.
GPT-5.4 adopted a more cautious approach. The model used 160 tool calls to build models before placing a single bet. It calculated that its logarithmic loss (0,974) was slightly worse than the market loss (0,971). It concluded that it had no advantage. The system spent the remainder of the season placing minimum bets to preserve capital.
Diagnosis of central failure
The researchers identified the central problem as a "gap between knowledge and action." The models demonstrated the ability to articulate correct strategies. They diagnosed failures. They identified the causes of losses.
The systems failed to verify whether the code implemented what was planned. They didn't notice when the execution deviated from the intended course. They didn't act on their own findings.
Business decisions are primarily based on fixed conditions. Sports betting, however, constitutes a fluid and changeable market. Researchers argue: "KellyBench requires agents to maintain a consistent intention across thousands of sequential decisions, monitor the consequences of those decisions, and close the loop between observation and action."
Characteristics of the test environment
KellyBench used data from 120 days of matches from the 2023-24 English Premier League season . The test has specific characteristics: constantly changing data, a market that gets smarter every week, and promoted teams without a history of matches.
In early 2026, artificial intelligence benchmarks showed that Claude could dominate business simulations through price fixing, cartel agreements, and strategic deception. This decision-making process involved static competition, limited opponents, and clear scoring. KellyBench exhibits characteristics that are the opposite of business simulations.
Ross Taylor, CEO of General Reasoning and former artificial intelligence researcher at Meta, told the Financial Times that most artificial intelligence benchmarks operate in "very static environments" that bear little resemblance to the real world. "There's a lot of excitement around AI automation, but there haven't been many attempts to evaluate AI in long-term, real-world environments," he stated.
Previous studies on AI behavior
Research published the previous year found that artificial intelligence models develop something similar to gambling addiction when instructed to maximize rewards. The systems failed up to 48% of the time in simulated slot machine tests.
A separate cryptocurrency trading competition involving real money encountered the same reliability issues over extended periods.
General Reasoning did not immediately respond to a request for comment on the experiment.
