Coherence training for forecasts · Kelvin, MATS

Frontier market game

Astra and Luna trade a combinatorial LMSR market (bayes-market, factored engine) over Turtel et al.’s 1,265 Polymarket test questions.

Brier by pass

Market Brier score by pass, Astra vs Luna Astra: 0.1746, 0.1696, 0.1699. Luna: 0.1874, 0.1887, 0.1950. 0.2000.190 0.1800.170 0.160 Pass 1Pass 2Pass 3 Brier, 1,265 questions (lower is better) Astra pass 1: 0.1746 Astra pass 2: 0.1696 Astra pass 3: 0.1699 Luna pass 1: 0.1874 Luna pass 2: 0.1887 Luna pass 3: 0.1950 0.1750.170Astra 0.170 0.1870.189Luna 0.195
Brier of the market after each pass. Same market, prompt and budget. Astra N=16 reached 0.166. For reference on the same questions (Turtel et al. Table 1): Polymarket price 0.151, their RL-trained 14B model 0.191, o1 0.202, constant 0.5 0.250.

The Bayes nets it built

NHL 4 Nations Face-Off (Feb 2025)

|P(a|b=yes) − P(a|b=no)| = 0.05 |P(a|b=yes) − P(a|b=no)| = 0.57 |P(a|b=yes) − P(a|b=no)| = 0.29 |P(a|b=yes) − P(a|b=no)| = 0.98 |P(a|b=yes) − P(a|b=no)| = 0.98 Will Finland win the NHL 4 Nations Face-Off? · market 0.03 · resolved NO Will the following market resolve to "USA"? NHL 4 Nations: USA vs. Sweden · market 0.57 · resolved NO Will Canada win the NHL 4 Nations Face-Off? · market 0.45 · resolved YES Will Sweden win the NHL 4 Nations Face-Off? · market 0.03 · resolved NO Will USA win the NHL 4 Nations Face-Off? · market 0.54 · resolved NO Will the following market resolve to "Canada"? Canada vs. USA · market 0.46 · resolved YES Finland wins 3% USA beats SWE 57% Canada wins 45% Sweden wins 3% USA wins 54% CAN beats USA 46%

Oscars 2025: Best Cinematography

|P(a|b=yes) − P(a|b=no)| = 0.29 |P(a|b=yes) − P(a|b=no)| = 0.36 |P(a|b=yes) − P(a|b=no)| = 0.27 |P(a|b=yes) − P(a|b=no)| = 0.04 Will "Dune: Part Two" win Best Cinematography at the 2025 Oscars? · market 0.13 · resolved NO Will "Maria" win Best Cinematography at the 2025 Oscars? · market 0.16 · resolved NO Will "Nosferatu" win Best Cinematography at the 2025 Oscars? · market 0.12 · resolved NO Will "Emilia Pérez" win Best Cinematography at the 2025 Oscars? · market 0.03 · resolved NO Will "The Brutalist" win Best Cinematography at the 2025 Oscars? · market 0.60 · resolved YES Dune: Part Two 13% Maria 16% Nosferatu 12% Emilia Pérez 3% The Brutalist 60%

Bitcoin on 28 Feb 2025

|P(a|b=yes) − P(a|b=no)| = 0.31 |P(a|b=yes) − P(a|b=no)| = 0.19 |P(a|b=yes) − P(a|b=no)| = 0.45 |P(a|b=yes) − P(a|b=no)| = 0.32 |P(a|b=yes) − P(a|b=no)| = 0.14 |P(a|b=yes) − P(a|b=no)| = 0.39 |P(a|b=yes) − P(a|b=no)| = 0.19 Will the price of Bitcoin be greater than $103000 on Feb 28? · market 0.10 · resolved NO Will the price of Bitcoin be between $97000 and $99000 on Feb 28? · market 0.16 · resolved NO Will the price of Bitcoin be between $99000 and $101000 on Feb 28? · market 0.15 · resolved NO Will the price of Bitcoin be between $101000 and $103000 on Feb 28? · market 0.11 · resolved NO Will the price of Bitcoin be between $93000 and $95000 on Feb 28? · market 0.12 · resolved NO Bitcoin above $99,000 on February 28? · market 0.32 · resolved NO Will the price of Bitcoin be between $95000 and $97000 on Feb 28? · market 0.16 · resolved NO Will the price of Bitcoin be less than $93000 on Feb 28? · market 0.23 · resolved YES over $103k 10% $97–99k 16% $99–101k 15% $101–103k 11% $93–95k 12% above $99k 32% $95–97k 16% under $93k 23%

Real Betis vs Real Madrid

|P(a|b=yes) − P(a|b=no)| = 0.33 |P(a|b=yes) − P(a|b=no)| = 0.34 |P(a|b=yes) − P(a|b=no)| = 0.64 Will Real Betis Beat Real Madrid? · market 0.21 · resolved YES Will Real Madrid Beat Real Betis? · market 0.39 · resolved NO Will Real Betis vs. Real Madrid End in a Draw? · market 0.41 · resolved NO Betis win 21% Madrid win 39% Draw 41%
Four real components from Astra N=16 after pass 3. Fill = market probability; ring = resolved YES; edge width = |P(a|b=yes) − P(a|b=no)|.
RunLinked questionsDependenciesComponentsLargestLog loss, marginaljoint
Astra, N=850641315380.5180.491
Luna, N=82651689980.6090.599
Astra, N=16456404111120.5080.477

Joint log loss beats marginal in every run: the dependencies carry information.

Caveat: the questions closed Feb–Mar 2025, inside both models’ training data; pass-to-pass change is the clean result.