Comparing AI and conventional physics-based models
The models scored on this page. Every number below is the same score the leaderboard keeps: a model's daily high against what a station recorded, at one, three and seven days out, over 2026-09-22 to 2026-10-21.
Two organizations run an AI model beside their physics one, from the same starting conditions, so within a panel only the model differs. That is the comparison this page exists to make; the rows below put both in the full field.
Single-run rows are scored against the leaderboard's station highs; ensemble rows are means scored beside means and never beside the single runs. Typical miss is the mean absolute error of the daily high, pooled across every scored city and weighted by days.
The roster
What each column is, before any number. An ensemble is scored as a mean against the other means, never as a model line.
| Model | Produced by | Type | Ensemble |
|---|---|---|---|
| AIFS Single | ECMWF (Europe) | AI | no |
| AIGFS | NOAA (US) | AI | no |
| GEM | ECCC (Canada) | physics | no |
| GFS | NOAA (US) | physics | no |
| ICON | DWD (Germany) | physics | no |
| IFS | ECMWF (Europe) | physics | no |
| Unified Model | Met Office (UK) | physics | no |
| GEFS | NOAA (US) | physics | yes — 31 members |
| AIGEFS | NOAA (US) | AI | yes — 31 members |
| ECMWF ENS | ECMWF (Europe) | physics | yes — 51 members |
| AIFS ENS | ECMWF (Europe) | AI | yes — 51 members |
| WeatherNext 2 | Google DeepMind | AI | yes — 64 members |
How we approach model scoring
Verifying a weather model is typically done by comparing the forecasts produced by that model with one of several things: another weather model, a historical re-analysis, or actual surface observations. All these methods have limitations. However, we believe true model skill at the surface can best be determined by using the actual observations that occurred, instead of a computed matrix, because computed data is usually less accurate than corresponding observations, when those are available. So this page scores against stations that produce observations: 30 of them, spread across the US and Europe.
Typical miss, by lead
Mean absolute error of the daily high, pooled across every scored city and weighted by days. One scale for all three rows, so the pack visibly widens with lead. Lower is better.
AI model physics model best at that lead. Dashed rules are the AI (dark) and physics (grey) medians. Hover a dot for the model, its miss and how many model-days it rests on.
At 1 day: GFS 2.14°F, ICON 2.19°F, UKMO 2.36°F, GEM 2.4°F, AIFS 2.63°F (AI), AIGFS 2.77°F (AI), ECMWF 2.78°F. Ai median 2.7°f, physics median 2.36°f over 844–844 model-days.
At 3 days: ICON 2.76°F, GFS 2.79°F, AIFS 2.9°F (AI), UKMO 3.06°F, GEM 3.11°F, ECMWF 3.12°F, AIGFS 3.23°F (AI). Ai median 3.06°f, physics median 3.06°f over 844–844 model-days.
At 7 days: ICON 4.5°F, AIFS 4.59°F (AI), GEM 4.73°F, GFS 5.02°F, ECMWF 5.1°F, AIGFS 5.24°F (AI). Ai median 4.92°f, physics median 4.88°f over 86–843 model-days.
Who gets crowned, against chance
A city crowns a model only when it beats the runner-up by more than 0.4°F — the same floor the leaderboard uses. "Chance" is the share of scored models that are AI, so an AI win rate above it is evidence and one at it is not.
| Lead | AI crowned | Physics crowned | Too close | Cities | Chance (AI) |
|---|---|---|---|---|---|
| 1 day out | 4 | 7 | 19 | 30 | 29% |
| 3 days out | 4 | 6 | 20 | 30 | 29% |
| 7 days out | 8 | 6 | 16 | 30 | 39% |
How far out, and what each model publishes
Reach is the furthest day any city's newest archived forecast carries a high for the model. Rain chance is yes when any city's forecast archived in the 24 hours up to the newest one carries one. The AI models are thinner than their headlines: neither publishes a gust or an instability figure, so on those the physics models are not being beaten, they are alone. CAPE measured 2026-08-15 (probe_model_cape.py, 6 cities, 16 leads); gusts 2026-09-28 (probe_hourly_absent.py, 4 cities).
| Model | Kind | Reach | Rain chance | Gusts | CAPE |
|---|---|---|---|---|---|
| AIGFS | AI | 16 d | no | no | no |
| GFS | physics | 16 d | yes | yes | yes |
| AIFS | AI | 15 d | no | no | no |
| ECMWF | physics | 15 d | yes | yes | yes |
| GEM | physics | 10 d | yes | yes | yes |
| ICON | physics | 7 d | yes | yes | yes |
| UKMO | physics | 7 d | yes | yes | yes |
How often each model changes its mind
Between one sweep and the next, how often a model's forecast high for the same day moved by more than 1.0°F. Pooled over 959 consecutive six-hour steps across the archived cities since 2026-10-09. Steady is not the same as right — read this beside the misses above, never alone.
AI model physics model. Dashed rules are the AI (dark) and physics (grey) medians. Hover a dot for the model's rate and its typical revision.
The full table: typical revision, how far it moved when it did, and the unchanged share
| Model | Kind | Moved > 1.0°F | Typical revision | When it moved | Unchanged | Steps |
|---|---|---|---|---|---|---|
| AIFS | AI | 12% | 0.4°F | 1.35°F | 10% | 835 |
| AIGFS | AI | 12% | 0.4°F | 1.4°F | 8% | 929 |
| GFS | physics | 24% | 0.5°F | 1.7°F | 11% | 929 |
| ICON | physics | 26% | 0.5°F | 1.6°F | 7% | 929 |
| UKMO | physics | 29% | 0.6°F | 1.7°F | 10% | 929 |
| ECMWF | physics | 30% | 0.6°F | 1.7°F | 7% | 929 |
| GEM | physics | 36% | 0.7°F | 1.85°F | 5% | 929 |
| Model | Kind | Moved > 1.0°F | Typical revision | When it moved | Unchanged | Steps |
|---|---|---|---|---|---|---|
| AIFS | AI | 19% | 0.4°F | 1.5°F | 8% | 835 |
| AIGFS | AI | 31% | 0.6°F | 1.8°F | 7% | 929 |
| ECMWF | physics | 41% | 0.8°F | 2.0°F | 6% | 929 |
| UKMO | physics | 41% | 0.8°F | 2.0°F | 14% | 929 |
| ICON | physics | 43% | 0.9°F | 1.9°F | 4% | 929 |
| GFS | physics | 43% | 0.8°F | 1.9°F | 7% | 929 |
| GEM | physics | 48% | 1.0°F | 2.1°F | 10% | 929 |
| Model | Kind | Moved > 1.0°F | Typical revision | When it moved | Unchanged | Steps |
|---|---|---|---|---|---|---|
| AIFS | AI | 59% | 1.4°F | 2.6°F | 4% | 835 |
| GEM | physics | 63% | 1.7°F | 2.8°F | 16% | 929 |
| ECMWF | physics | 66% | 1.8°F | 2.9°F | 10% | 929 |
| AIGFS | AI | 67% | 1.9°F | 3.1°F | 3% | 929 |
| GFS | physics | 75% | 2.3°F | 3.3°F | 2% | 929 |
"Unchanged" is the share of steps where the forecast did not move at all — high for a model whose newest run had not yet arrived between two sweeps, so a low move rate with a low unchanged share is a model revising in small steps rather than one standing still. Steps where a column changed which model it meant are left out.
The ensembles, mean against mean
NOAA and ECMWF each run an AI ensemble beside their physics one — NOAA's AIGEFS beside GEFS, ECMWF's AIFS ENS beside its ENS — and Google's WeatherNext 2 is an ensemble with no single run at all. What we archive for each is the members' mean and spread. A mean of many runs flattens the afternoon peak, so none of them is scored beside the single-run models above; they are scored beside each other, where the flattening cancels and, within one organization, only the model differs. None enters "the models" anywhere else on this site.
Bias of each ensemble mean's daily high against the station, by lead: warm is a mean that runs too warm, cool too cool, on one scale of ±5.0°F. The typical miss is in the table beneath.
| Ensemble | Organization | Type | 1 d miss | bias | 3 d miss | bias | 7 d miss | bias | Days |
|---|---|---|---|---|---|---|---|---|---|
| GEFS | NOAA | physics | 2.73°F | -0.21°F | 3.09°F | -0.31°F | 4.24°F | -0.47°F | 1789–1969 |
| AIGEFS | NOAA | AI | 4.22°F | -3.85°F | 4.74°F | -4.3°F | 5.85°F | -4.75°F | 1789–1969 |
| ECMWF ENS | ECMWF | physics | 2.83°F | -1.96°F | 3.15°F | -1.97°F | 4.49°F | -2.66°F | 1789–1969 |
| AIFS ENS | ECMWF | AI | 2.78°F | -2.26°F | 3.02°F | -2.25°F | 4.21°F | -2.59°F | 1789–1969 |
| WeatherNext 2 | AI | 3.95°F | -3.6°F | 4.18°F | -3.61°F | 5.06°F | -3.84°F | 1813–1989 |
GEFS since 2026-07-30; AIGEFS since 2026-07-30; ECMWF ENS since 2026-07-30; AIFS ENS since 2026-07-30; WeatherNext 2 since 2026-07-29. A forecast is scored once its day has passed and the station has reported.
Two things to hold in mind before reading a bias here. First, these are highs of an averaged series, and averaging members that peak at different hours lowers the peak a little: for GEFS the mean's high sits about half a degree below the median of its members' highs, more at longer leads. That is the size of the effect, so a lean of several degrees is the ensemble's own. Second, this is one month of one season. A lean this consistent will not average out with more days, but its size may move with the weather. Read the rows against each other, and in particular compare each AI ensemble with the physics ensemble from the same organization, since those share their starting conditions and differ only in the model. Never compare these rows with the single-run models above.
How wide each ensemble is right now
Each ensemble's member spread (one standard deviation) at the valid date's daily high, averaged across cities, at the same lead from today's newest run.
| Ensemble | Type | 1 d spread | 3 d spread | 7 d spread | Cities |
|---|---|---|---|---|---|
| GEFS | physics | 1.69°F | 2.2°F | 4.59°F | 30 |
| AIGEFS | AI | 1.85°F | 2.33°F | 3.94°F | 30 |
| ECMWF ENS | physics | 1.13°F | 1.88°F | 4.52°F | 30 |
| AIFS ENS | AI | 1.39°F | 2.07°F | 4.14°F | 30 |
| WeatherNext 2 | AI | 1.43°F | 1.91°F | 3.6°F | 30 |
Rain: the share of ensemble members that are wet
How close each ensemble's share of wet members came to the rain gauge. 0 is perfect, lower is better.
| Ensemble | Type | 1 d score | 3 d score | 7 d score |
|---|---|---|---|---|
| AIGEFS | AI | 0.129 | 0.138 | 0.189 |
| GEFS | physics | 0.128 | 0.125 | 0.185 |
| Lower score | too close to call | GEFS | too close to call | |
| Dates scored | 40 (1142 city-days) | 38 (1083 city-days) | 34 (966 city-days) | |
Dates 2026-09-12 to 2026-10-21, at the rain gauges of the 30 stations scored.
Method
Scores are the leaderboard's: each model's daily high, the last forecast The Spread archived for it N days before, against the station's observed high, every model on the same days, pooled across cities and weighted by scored days. "AI median" and "physics median" are the medians of the pooled per-model misses in each group. A model's bias has no sign until you name what it was scored against; every sign on this page is against a thermometer. Where a station is the truth, a model's high is the warmest of its hourly values and the station's is the warmest it got at any moment, so a model's high reads a little cool, and its low a little warm, even on a day it gets every hour right. ECMWF IFS gives a value only every 3 hours (every 6 past day 6), and ECMWF AIFS (AI) and NOAA AIGFS (AI) only every 6, with the hours between filled in, so their highs read cooler still and their lows warmer. In the forecasts scored here GFS also steps to every 3 hours from day 5, ICON and GEM from day 4, and UKMO from day 3, every 6 by day 6. Sample sizes are printed because station-scored days began in late August 2026 and a calendar span is not evidence. The National Blend of Models and our own blend are consensus products and are excluded from every count.