Compare models
21 registered · 21 with usable out-of-sample days · scope: val + test
By cell type
| cell | horizon | models | rank IC (mean ± sd) | spread across models | LS decile | hit (alpha) |
|---|---|---|---|---|---|---|
| gru | +1d | 11 | +0.0689 ± 0.0055 | +0.0585 … +0.0772 | +1.06% | 55.1% |
| gru | +7d | 11 | +0.0833 ± 0.0079 | +0.0724 … +0.0956 | +3.73% | 59.2% |
| gru | +30d | 11 | +0.1040 ± 0.0058 | +0.0973 … +0.1124 | +10.80% | 60.8% |
| lstm | +1d | 10 | +0.0605 ± 0.0059 | +0.0489 … +0.0693 | +0.93% | 55.0% |
| lstm | +7d | 10 | +0.0746 ± 0.0082 | +0.0592 … +0.0854 | +3.16% | 59.6% |
| lstm | +30d | 10 | +0.0935 ± 0.0069 | +0.0807 … +0.1040 | +9.10% | 62.2% |
Two-stage aggregation: each model's daily rank-IC series is averaged to one score, then those scores are averaged and spread across models. So ±sd is the model-to-model (seed) spread, not day-to-day noise — that distinction is the whole point of a seeded sweep.
Note this group holds every registered model of that cell type, including any that are not part of the seeded sweep (
Note this group holds every registered model of that cell type, including any that are not part of the seeded sweep (
beta_5y, for instance, which is a gru model but was trained separately). The Paired by seed section below is the one that corresponds to compare_cells.py.Paired by seed
Same seed = same initialisation stream and the same data, so per-seed differences cancel seed-level noise. Values are lstm − gru in rank IC.
| horizon | seeds paired | mean Δ | ± SE | t | win rate | relative | verdict |
|---|---|---|---|---|---|---|---|
| +1d | 10 | -0.0080 | 0.0022 | -3.56 | 20% lstm | -11.6% | gru better |
| +7d | 10 | -0.0079 | 0.0026 | -3.04 | 10% lstm | -9.6% | no measurable difference |
| +30d | 10 | -0.0102 | 0.0023 | -4.34 | 0% lstm | -9.8% | gru better |
These figures are val + test.
compare_cells.py selects on the validation split only — choose the val only pill above and this table reproduces its output exactly. Verified to 4 decimals against the 10-seed run: gru +0.0745/+0.0920/+0.1190 vs lstm +0.0646/+0.0810/+0.1036, paired Δ −0.0099/−0.0110/−0.0154 with t = −3.94/−4.06/−5.28. Under any other scope the numbers legitimately differ, because they cover more dates. The val-only figure is the one the GRU-over-LSTM decision was made on.Every model
| model | cell | params | data end | IC +1d | LS +1d | hit +1d | IC +7d | LS +7d | hit +7d | IC +30d | LS +30d | hit +30d | days |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| beta_5y | gru | 250,771 | 2026-10-21 | +0.0738 | +1.05% | 55.6% | +0.0915 | +3.74% | 60.1% | +0.1074 | +11.32% | 61.6% | 530 |
| gru_s0 | gru | 250,771 | 2026-10-21 | +0.0772 | +1.12% | 55.3% | +0.0956 | +4.30% | 58.4% | +0.1101 | +10.96% | 60.5% | 530 |
| gru_s1 | gru | 250,771 | 2026-10-21 | +0.0720 | +1.15% | 55.5% | +0.0868 | +3.86% | 59.8% | +0.1051 | +10.97% | 61.1% | 530 |
| gru_s2 | gru | 250,771 | 2026-10-21 | +0.0706 | +1.20% | 54.7% | +0.0765 | +3.72% | 56.4% | +0.0984 | +10.79% | 58.7% | 530 |
| gru_s3 | gru | 250,771 | 2026-10-21 | +0.0645 | +0.99% | 55.5% | +0.0730 | +3.05% | 60.2% | +0.0988 | +9.51% | 60.9% | 530 |
| gru_s4 | gru | 250,771 | 2026-10-21 | +0.0585 | +0.96% | 53.0% | +0.0724 | +3.38% | 58.0% | +0.0979 | +11.75% | 57.6% | 530 |
| gru_s5 | gru | 250,771 | 2026-10-21 | +0.0665 | +0.97% | 55.6% | +0.0773 | +3.16% | 60.8% | +0.0973 | +9.12% | 62.8% | 530 |
| gru_s6 | gru | 250,771 | 2026-10-21 | +0.0650 | +1.00% | 54.6% | +0.0833 | +3.64% | 57.4% | +0.1121 | +10.86% | 61.8% | 530 |
| gru_s7 | gru | 250,771 | 2026-10-21 | +0.0649 | +1.05% | 55.6% | +0.0840 | +3.92% | 60.6% | +0.1013 | +10.85% | 62.1% | 530 |
| gru_s8 | gru | 250,771 | 2026-10-21 | +0.0710 | +1.04% | 55.6% | +0.0836 | +3.83% | 60.3% | +0.1035 | +10.92% | 62.4% | 530 |
| gru_s9 | gru | 250,771 | 2026-10-21 | +0.0741 | +1.09% | 55.2% | +0.0923 | +4.39% | 59.1% | +0.1124 | +11.81% | 59.9% | 530 |
| lstm_s0 | lstm | 316,819 | 2026-10-21 | +0.0571 | +0.99% | 55.4% | +0.0758 | +3.36% | 60.3% | +0.0929 | +8.98% | 63.2% | 530 |
| lstm_s1 | lstm | 316,819 | 2026-10-21 | +0.0630 | +0.99% | 55.7% | +0.0831 | +3.89% | 60.8% | +0.1040 | +10.63% | 60.2% | 530 |
| lstm_s2 | lstm | 316,819 | 2026-10-21 | +0.0649 | +1.06% | 53.5% | +0.0744 | +3.32% | 56.6% | +0.0918 | +9.01% | 59.2% | 530 |
| lstm_s3 | lstm | 316,819 | 2026-10-21 | +0.0538 | +0.85% | 55.4% | +0.0645 | +2.54% | 60.3% | +0.0870 | +8.25% | 62.7% | 530 |
| lstm_s4 | lstm | 316,819 | 2026-10-21 | +0.0606 | +0.93% | 52.7% | +0.0808 | +3.27% | 56.0% | +0.0960 | +9.57% | 61.3% | 530 |
| lstm_s5 | lstm | 316,819 | 2026-10-21 | +0.0489 | +0.82% | 55.4% | +0.0592 | +2.11% | 60.7% | +0.0807 | +7.37% | 63.3% | 530 |
| lstm_s6 | lstm | 316,819 | 2026-10-21 | +0.0604 | +0.90% | 55.5% | +0.0694 | +2.62% | 60.4% | +0.0891 | +8.51% | 63.0% | 530 |
| lstm_s7 | lstm | 316,819 | 2026-10-21 | +0.0654 | +0.95% | 55.7% | +0.0787 | +3.76% | 60.9% | +0.0942 | +10.23% | 63.4% | 530 |
| lstm_s8 | lstm | 316,819 | 2026-10-21 | +0.0610 | +0.90% | 55.6% | +0.0745 | +3.15% | 60.5% | +0.1006 | +9.22% | 63.2% | 530 |
| lstm_s9 | lstm | 316,819 | 2026-10-21 | +0.0693 | +0.93% | 55.3% | +0.0854 | +3.56% | 59.9% | +0.0990 | +9.24% | 62.7% | 530 |