ArticleOct 21, 20266 min read

Choose Your Train Set Carefully

The same linear model on ETH, trained on two train sets that only differ by their start date, gives two opposite strategies. We look at what each one learned, and why a bull market hides in the intercept.

Lately, I was watching a MemLabs video about cross-asset returns, and I tried to reproduce their results on my own data, without success. After a few tries, I noticed something: my train set wasn't the same as theirs, and the way I trained my model wasn't good at all.

Today we'll see why understanding your train set is essential: it tells you what your model actually learns, and how to make your strategies better.

The model

We'll work with the following model, which predicts the next-week log return of ETH from the past-week log returns of BNB, CRV and UNI:

\begin{gathered} r^{\text{ETH}}_{t+1} = w_1 \, r^{\text{BNB}}_t + w_2 \, r^{\text{CRV}}_t + w_3 \, r^{\text{UNI}}_t + b \\[4pt] {\scriptsize \text{with } r^{X}_t = \ln\left(P^{X}_t / P^{X}_{t-1}\right) \text{, the weekly log return of } X} \end{gathered}

The weights are found with a classic MSE loss (ordinary least squares). The strategy is long when the prediction is positive and short otherwise, and we train the model on two train sets:

one starting in September 2020;
one starting in January 2022.

Both end on 2025-07-27. The test set runs from 2025-08-04 to 2026-10-11. For the record, MemLabs uses the train set starting in 2022 in their video.

Cumulative log return of the Cross-Asset model trained from Sep 2020 and from Jan 2022, in sample and on the test set, next to ETH buy and hold. The 2022 model climbs steadily to about +3 and keeps rising on the test set, while the 2020 model peaks around +1.5 and falls back on the test set — Each model is fitted once on its train set, then traded on its own train rows (in sample) and on the test set.

Cumulative log return on the test set only, Aug 2025 to Oct 2026. The model trained from Jan 2022 ends around +0.5, the model trained from Sep 2020 ends around −0.5, tracking ETH buy and hold closely — The test set alone. The model trained since 2020 follows ETH buy & hold almost step for step.

Train set	In-sample Sharpe	In-sample win rate	In-sample long share	Test Sharpe	Test win rate	Test long share
Sep 2020 → Jul 2025	+0.35	51.3%	86%	−0.72	48.4%	89%
Jan 2022 → Jul 2025	+1.00	55.8%	49%	+0.67	56.0%	50%
ETH buy & hold	+0.61	52.9%	100%	−0.60	47.7%	100%

The strategy trained on the longer set underperforms on the test set. More interesting: it is long 89% of the time, while the one trained since 2022 is long only half of the time. The train set completely changes the strategy: the two models don't learn the same thing.

Let's see what happens to the weights of the model when we retrain it through time.

Intercept and weights of BNB, CRV and UNI, refitted every month on an expanding window, for both train sets. The 2020 intercept starts near +0.04 and stays clearly positive, the 2022 one stays near zero. The BNB weight is negative for both, while the CRV and UNI weights differ in level and drift over time — Intercept and weights refitted on the first day of each month, on all past rows since the train start. The fit at the start of the shaded area is the model used on the test set.

First, the intercept (the $b$ in the model) is not the same at all for the two strategies. With the 2020 train set, it is large, about +1% per week against +0.14% for the 2022 one, because the model is trained on the 2020–2021 bull market: it carries a bullish bias. The weight of BNB stays negative through time, while the weights of CRV and UNI are not stable.

So the model is biased by the bull run, and that bias shows up as a positive intercept. Let's try to fix it by removing the intercept.

Removing the intercept

Cumulative log return of the model without intercept, trained from Sep 2020 and from Jan 2022, in sample and on the test set, next to ETH buy and hold. The 2022 model still rises steadily, the 2020 model is much flatter than before — Same backtest, model without intercept.

Cumulative log return on the test set for the model without intercept. The model trained from Jan 2022 rises to about +0.6, the model trained from Sep 2020 hovers around zero, and ETH buy and hold ends around −0.5 — The test set alone, model without intercept.

Train set	In-sample Sharpe	In-sample win rate	In-sample long share	Test Sharpe	Test win rate	Test long share
Sep 2020 → Jul 2025	+0.24	50.3%	44%	+0.05	49.5%	41%
Jan 2022 → Jul 2025	+1.03	56.1%	43%	+0.83	57.1%	44%
ETH buy & hold	+0.61	52.9%	100%	−0.60	47.7%	100%

Weights of BNB, CRV and UNI for the model without intercept, refitted every month on an expanding window, for both train sets. BNB is negative for both but around −0.06 for 2020 against −0.2 for 2022, CRV is negative then near zero for 2020 but around +0.1 for 2022, UNI decreases from +0.2 for 2020 and stays near zero for 2022 — Weights of the model without intercept, refitted every month on an expanding window.

Without the intercept, the two strategies have much closer long shares, and their equity curves are more correlated: the correlation of their daily trade returns goes from −0.02 to 0.66, and they take the same position 74% of the days instead of 59%. But their performance is still not the same at all.

Their weights are not the same either. The train set has a big influence on the result of the strategy, and in the end it is very easy to find a strategy that looks good but has no edge, just by picking the right train set.

Conclusion

The data you train a strategy on matters a lot, because it can bias your model. After training it, you need to understand what it learned and what it will do. In our example, the model learned a strong uptrend from the bull market; to get rid of that bias, we can remove the intercept.

You also need to check that the model is stable through time, or there is a good chance that your good strategy is just noise. Here, the model wasn't stable at all, and changing the start date of the train set was enough to change the result.

The test set matters too

Moving the test set changes the result as well: with a test set starting in January 2025 (and train sets ending in December 2024), the Sharpe of the model trained since 2022 drops from +0.67 to +0.27.

Want to try it on real data?

Angolin serves the full BTC history at 1d, 1h and 1m granularity, and the historical universe of each exchange, delisted pairs included, to avoid survivorship bias.

Choose Your Train Set Carefully

The model

Removing the intercept

Conclusion

More articles

Why Is It So Hard to Short?

Fractional Differentiation

The Impact of Survivorship Bias