Coherence training for forecasts · Kelvin, MATS

One week, every conditional

Manifold test events open on 2025-06-23 00:00 UTC. Qwen3-32B (no thinking) was asked each event alone, "A and B" and "B given A" both ways for every link, and "A and B and C" for every triangle. One joint model (event, link and triangle terms) was fitted to all 4,361 answers, exactly.

449events open

1030links

822triangles

15treewidth (exact)

0.501log loss, raw

0.474log loss, fitted

Top: the largest linked group (114 events). Hover a dot for the question, an edge for P(B|A), P(A|B) and the Luna relation. Why this week: The week with the most open events whose full question set fits the $2 cap (at the conservative 0.7/2.8 $/M price): 449 events, 1,030 Luna links of all five relation types, 822 triangles, largest component 114 events, treewidth 15 so the junction tree is exact. Later weeks have 500-1,000 events but cost $2.2-4.8 each; earlier weeks have <=319 events and a largest component of 48.

Stated vs fitted conditionals, largest group

The 16 strongest links. Stated: Qwen3-32B asked directly. Fitted: read off the joint model.

A	B	P(B\|A) stated	fitted	P(A\|B) stated	fitted
Iran-Israel conflict Hormuz closure 2025	Iran blocks Hormuz Strait in 2025	0.75	0.69	0.75	0.70
Iran-Israel conflict Hormuz closure 2025	Israel-Iran 2025: Hormuz Strait Closure	0.85	0.75	0.75	0.65
Israel-Iran 2025: Hormuz Strait Closure	Iran blocks Hormuz Strait in 2025	0.75	0.64	0.85	0.75
Iran-Israel conflict Hormuz closure 2025	Iran closes Hormuz Strait 2025	0.75	0.69	0.75	0.66
Iran closes Hormuz Strait 2025	Iran blocks Hormuz Strait in 2025	0.75	0.66	0.95	0.70
Israel-Iran 2025: Hormuz Strait Closure	Iran closes Hormuz Strait 2025	0.75	0.65	0.75	0.71
Strait of Hormuz Closure by 31 Dec 2025	Iran-Israel conflict Hormuz closure 2025	0.85	0.60	0.85	0.73
Iran-Israel conflict Hormuz closure 2025	Israel-Iran 2025 Hormuz Closure <72h	0.75	0.69	0.75	0.62
Strait of Hormuz Closure by 31 Dec 2025	Iran blocks Hormuz Strait in 2025	0.65	0.59	0.85	0.73
Israel-Iran 2025 Hormuz Closure <72h	Iran blocks Hormuz Strait in 2025	0.75	0.61	0.35	0.68
Strait of Hormuz Closure by 31 Dec 2025	Iran closes Hormuz Strait 2025	0.75	0.61	0.85	0.70
Israel-Iran 2025 Hormuz Closure <72h	Israel-Iran 2025: Hormuz Strait Closure	0.75	0.67	0.85	0.64
Strait of Hormuz Closure by 31 Dec 2025	Israel-Iran 2025: Hormuz Strait Closure	0.75	0.65	0.85	0.68
Israel-Iran 2025 Hormuz Closure <72h	Iran closes Hormuz Strait 2025	0.45	0.62	0.65	0.66
Strait of Hormuz Closure by 31 Dec 2025	Israel-Iran 2025 Hormuz Closure <72h	0.75	0.61	0.92	0.67
Iran-Israel conflict Hormuz closure 2025	Iran oil export halt to China 2025	0.75	0.67	0.45	0.48

Log loss against outcomes

Question	n	Raw 32B	Fitted	Independent
Event alone	449	0.634	0.693	0.634
A and B	1030	0.485	0.422	0.384
A and B and C	822	0.330	0.291	0.261
B given A	394	0.745	0.746	0.738
All questions	2695	0.501	0.474	0.440

Independent: the stated single-event answers multiplied. Conditionals scored where the condition came true. This week cost $1.53 (4,361 calls). All 35 test weeks: 231,293 calls, about $81.