FORECAST RESEARCH / FREE CALCULATOR
Was the probability useful? Check the record.
Evaluate binary forecasts with the 0–1 Brier convention. Inspect the arithmetic, compare a fixed 50% reference and keep uncertainty visible.
FORECAST QUALITY / BINARY 0–1 SCALE
Score the probability, not just the pick.
Compare forecasts on the same four fictional resolved events. One unresolved row stays outside the score. Replace the CSV with your own observations when ready.
Use the header shown. Enter 70 for 70%; 0.7 means 0.7%. Mark pending events unresolved. Up to 1,000 rows; blank lines are ignored.
Scoring stays in this browser tab. This tool does not upload or save your CSV. It does not verify forecast timestamps, event selection or outcomes.
A negative difference beats the fixed 50% reference on these entered rows; a positive difference is worse. This is not a profit or significance test.
mean((p − outcome)²)p = percent ÷ 100 · Yes = 1 · No = 0
What happened in each probability band?
Five fixed bands: lower edge included, upper edge excluded; the last band includes 100%. Dots show mean forecast versus observed Yes frequency. The diagonal is equality, not a fitted model. Counts matter; small or correlated samples cannot establish reliable calibration.
| Band | N | Mean p | Yes rate |
|---|---|---|---|
| 0–20% | 0 | — | — |
| 20–40% | 2 | 20.0% | 50.0% |
| 40–60% | 0 | — | — |
| 60–80% | 0 | — | — |
| 80–100% | 2 | 80.0% | 50.0% |
Inspect every entered forecast and squared error
| CSV line | Forecast | Outcome | Squared error |
|---|---|---|---|
| 2 | 80.0% | Yes (1) | 0.0400 |
| 3 | 80.0% | No (0) | 0.6400 |
| 4 | 20.0% | Yes (1) | 0.6400 |
| 5 | 20.0% | No (0) | 0.0400 |
| 6 | 70.0% | Unresolved | — |
The Brier score measures overall probabilistic error. Its value also depends on resolution and the outcome mix; it is not calibration alone. Use the forecast evaluation guide for sampling, timing and interpretation.
What this calculator measures
For each resolved Yes/No event, the error is (p − outcome)². The Brier score is the equal-weight mean of those errors. This binary convention ranges from 0 to 1. The mixed example has errors 0.04, 0.64, 0.64 and 0.04: a mean of 0.3400. The unresolved fifth row is excluded.
Use the same comparison set
A forecast of 50% always has a squared error of 0.25 for a resolved binary outcome. It is a transparent reference, not a strong baseline for every domain. Compare forecasters on the same events and information cutoff; do not replace missing forecasts with 50%. The tool cannot establish that entries were recorded before their outcomes.
Calibration needs more than a score
The five-band chart is descriptive. Empty bands stay empty, and small counts or correlated events limit what a dot can tell you. This calculator does not fit a recalibration model, estimate confidence intervals or establish a trading advantage. Fees, price, available size and cash flows belong to a separate performance analysis.
Method: scikit-learn's binary Brier definition and calibration guidance. Read the full prediction-market walkthrough or evaluate a trader's PnL separately.