Compare models
21 registered · 21 with usable out-of-sample days · scope: validation only
By cell type
| cell | horizon | models | rank IC (mean ± sd) | spread across models | LS decile | hit (alpha) |
|---|---|---|---|---|---|---|
| gru | +1d | 11 | +0.0753 ± 0.0052 | +0.0668 … +0.0830 | +0.88% | 55.5% |
| gru | +7d | 11 | +0.0932 ± 0.0073 | +0.0853 … +0.1056 | +2.63% | 60.0% |
| gru | +30d | 11 | +0.1191 ± 0.0048 | +0.1082 … +0.1271 | +8.68% | 60.1% |
| lstm | +1d | 10 | +0.0646 ± 0.0067 | +0.0510 … +0.0721 | +0.76% | 55.3% |
| lstm | +7d | 10 | +0.0810 ± 0.0089 | +0.0639 … +0.0911 | +2.27% | 60.5% |
| lstm | +30d | 10 | +0.1036 ± 0.0099 | +0.0854 … +0.1155 | +7.53% | 60.9% |
Two-stage aggregation: each model's daily rank-IC series is averaged to one score, then those scores are averaged and spread across models. So ±sd is the model-to-model (seed) spread, not day-to-day noise — that distinction is the whole point of a seeded sweep.
Note this group holds every registered model of that cell type, including any that are not part of the seeded sweep (
Note this group holds every registered model of that cell type, including any that are not part of the seeded sweep (
beta_5y, for instance, which is a gru model but was trained separately). The Paired by seed section below is the one that corresponds to compare_cells.py.Paired by seed
Same seed = same initialisation stream and the same data, so per-seed differences cancel seed-level noise. Values are lstm − gru in rank IC.
| horizon | seeds paired | mean Δ | ± SE | t | win rate | relative | verdict |
|---|---|---|---|---|---|---|---|
| +1d | 10 | -0.0099 | 0.0025 | -3.94 | 10% lstm | -13.3% | gru better |
| +7d | 10 | -0.0110 | 0.0027 | -4.06 | 20% lstm | -11.9% | gru better |
| +30d | 10 | -0.0154 | 0.0029 | -5.28 | 0% lstm | -12.9% | gru better |
These figures are validation only.
compare_cells.py selects on the validation split only — choose the val only pill above and this table reproduces its output exactly. Verified to 4 decimals against the 10-seed run: gru +0.0745/+0.0920/+0.1190 vs lstm +0.0646/+0.0810/+0.1036, paired Δ −0.0099/−0.0110/−0.0154 with t = −3.94/−4.06/−5.28. Under any other scope the numbers legitimately differ, because they cover more dates. The val-only figure is the one the GRU-over-LSTM decision was made on.Every model
| model | cell | params | data end | IC +1d | LS +1d | hit +1d | IC +7d | LS +7d | hit +7d | IC +30d | LS +30d | hit +30d | days |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| beta_5y | gru | 250,771 | 2026-10-21 | +0.0830 | +0.92% | 56.0% | +0.1056 | +2.92% | 61.3% | +0.1209 | +9.38% | 60.6% | 265 |
| gru_s0 | gru | 250,771 | 2026-10-21 | +0.0806 | +0.86% | 55.6% | +0.1011 | +2.94% | 58.4% | +0.1212 | +9.37% | 58.2% | 265 |
| gru_s1 | gru | 250,771 | 2026-10-21 | +0.0768 | +0.96% | 55.7% | +0.0954 | +2.94% | 60.6% | +0.1186 | +9.39% | 60.3% | 265 |
| gru_s2 | gru | 250,771 | 2026-10-21 | +0.0796 | +1.08% | 55.0% | +0.0951 | +2.93% | 57.7% | +0.1239 | +8.50% | 59.2% | 265 |
| gru_s3 | gru | 250,771 | 2026-10-21 | +0.0728 | +0.84% | 55.8% | +0.0862 | +1.95% | 60.7% | +0.1199 | +7.61% | 59.9% | 265 |
| gru_s4 | gru | 250,771 | 2026-10-21 | +0.0679 | +0.88% | 53.8% | +0.0865 | +2.67% | 59.4% | +0.1184 | +9.57% | 58.3% | 265 |
| gru_s5 | gru | 250,771 | 2026-10-21 | +0.0727 | +0.82% | 55.9% | +0.0855 | +2.43% | 61.8% | +0.1082 | +8.01% | 62.0% | 265 |
| gru_s6 | gru | 250,771 | 2026-10-21 | +0.0722 | +0.83% | 55.3% | +0.0897 | +2.22% | 58.2% | +0.1191 | +7.66% | 60.7% | 265 |
| gru_s7 | gru | 250,771 | 2026-10-21 | +0.0668 | +0.80% | 55.9% | +0.0853 | +2.42% | 61.2% | +0.1147 | +8.14% | 61.1% | 265 |
| gru_s8 | gru | 250,771 | 2026-10-21 | +0.0769 | +0.83% | 55.9% | +0.0928 | +2.46% | 60.9% | +0.1185 | +8.46% | 61.1% | 265 |
| gru_s9 | gru | 250,771 | 2026-10-21 | +0.0788 | +0.87% | 55.5% | +0.1022 | +3.04% | 60.0% | +0.1271 | +9.36% | 59.8% | 265 |
| lstm_s0 | lstm | 316,819 | 2026-10-21 | +0.0584 | +0.80% | 55.6% | +0.0779 | +2.57% | 61.0% | +0.1032 | +8.24% | 61.9% | 265 |
| lstm_s1 | lstm | 316,819 | 2026-10-21 | +0.0684 | +0.81% | 56.0% | +0.0894 | +2.97% | 61.8% | +0.1155 | +9.49% | 60.3% | 265 |
| lstm_s2 | lstm | 316,819 | 2026-10-21 | +0.0713 | +0.84% | 54.1% | +0.0828 | +2.19% | 57.3% | +0.1100 | +7.89% | 57.4% | 265 |
| lstm_s3 | lstm | 316,819 | 2026-10-21 | +0.0588 | +0.73% | 55.7% | +0.0725 | +1.83% | 61.0% | +0.1001 | +6.04% | 61.4% | 265 |
| lstm_s4 | lstm | 316,819 | 2026-10-21 | +0.0662 | +0.80% | 53.0% | +0.0875 | +2.34% | 56.5% | +0.1043 | +7.79% | 59.2% | 265 |
| lstm_s5 | lstm | 316,819 | 2026-10-21 | +0.0510 | +0.64% | 55.8% | +0.0639 | +1.53% | 62.1% | +0.0891 | +5.99% | 62.3% | 265 |
| lstm_s6 | lstm | 316,819 | 2026-10-21 | +0.0644 | +0.82% | 55.9% | +0.0728 | +1.68% | 61.3% | +0.0854 | +5.86% | 61.6% | 265 |
| lstm_s7 | lstm | 316,819 | 2026-10-21 | +0.0701 | +0.71% | 56.0% | +0.0876 | +2.58% | 62.1% | +0.1071 | +8.75% | 62.2% | 265 |
| lstm_s8 | lstm | 316,819 | 2026-10-21 | +0.0653 | +0.78% | 56.0% | +0.0845 | +2.38% | 61.6% | +0.1151 | +8.10% | 61.9% | 265 |
| lstm_s9 | lstm | 316,819 | 2026-10-21 | +0.0721 | +0.72% | 55.3% | +0.0911 | +2.63% | 60.4% | +0.1062 | +7.17% | 60.8% | 265 |