Scoring Guide — Brier Score and Prediction Accuracy
What is the Brier Score?
The Brier Score is a numerical measure of probabilistic forecast accuracy. It ranges from 0 to 1, where lower is better.
Formula
BS = (probability/100 - outcome)²
Example: 70% YES forecast → outcome is YES (1)
BS = (0.70 - 1.0)² = 0.09 → GOOD
Example: 70% YES forecast → outcome is NO (0)
BS = (0.70 - 0.0)² = 0.49 → POOR
Score Tiers
| Brier Range | Tier | Meaning |
|---|---|---|
| 0.00 – 0.05 | EXCELLENT | Near-perfect calibration |
| 0.05 – 0.15 | GOOD | Better than most forecasters |
| 0.15 – 0.25 | FAIR | Competitive with market consensus |
| 0.25 – 0.35 | WEAK | Systematic bias or poor calibration |
| 0.35 – 1.00 | POOR | Worse than random (50/50) for some predictions |
Current Nowpattern Score
1116TOTAL35HITS18MISSES66.0%ACCURACYAccuracy = Hits ÷ (Hits+Misses) = 35÷53 | Sample size n=53 | 95% CI: [53.3%, 78.8%]Avg Brier Score: 0.47 — POOR ⚠️ (provisional)⚠️ Score Provenance Note (Honest Disclosure)
The Brier Score of 0.47 (POOR) shown above is provisional because all forecast probabilities were backfilled from current values rather than recorded at prediction time (FROM_CURRENT_VALUE_BACKFILL). This makes the Brier Score an unreliable calibration estimate.
Additionally, the 66.0% accuracy is based on a small sample of n=53 (35 hits + 18 misses), giving a wide 95% confidence interval of [53.3%, 78.8%].
Future predictions will have probabilities locked via OTS timestamp at the time of publication. Details: Integrity & Audit
Resolution Process
- Oracle deadline reached
prediction_auto_verifier.pyautomatically gathers evidence via Grok search- Claude Opus issues verdict (HIT / MISS / NOT_SCORED)
- Brier Score is calculated and recorded
- Result reflected on /en/predictions/
Unresolved Predictions
Predictions that cannot be resolved after their oracle deadline become EXPIRED_UNRESOLVED. No Brier Score is calculated, and they are not included in the accuracy rate.
Note: 4 predictions judged NOT_SCORED are excluded from hits, misses, and the accuracy calculation.
View the scoreboard: Prediction Tracker