I backtested my own stock rankings. They lost to the index

The event reveals a common pitfall in backtesting financial models: the use of future information to select assets. The original chart showed a $1,000 investment growing to $40,254 over 20 years, but this was achieved by ranking companies based on fundamentals as they exist today, not at the start of the period. This look-ahead bias is invisible in the final numbers but fundamentally alters the narrative. For engineers, this means any model that ranks or filters assets must ensure its selection criteria are based solely on data available at the time of decision-making. The cost of ignoring this is a model that appears predictive but is merely descriptive of past outcomes.

Survivorship bias compounds the problem. The original backtest only included companies that survived the entire 20-year period, excluding those that went bankrupt, were delisted, or were acquired. This artificially inflates returns because the dataset is pre-filtered for success. The corrected version, which ranked companies using only data available in 2016, showed the selected cohort underperforming the index in the subsequent decade. For engineers, this highlights the need to validate datasets for completeness, especially in domains where entities can disappear from the record. The cost of not doing so is a model that overestimates performance by ignoring failure cases.

The experiment also demonstrates how easily misleading visuals can be created. The original chart used a linear scale, which exaggerated the compounding effect in the final years, making the performance gap appear more dramatic. While not inherently deceptive, the choice of scale can influence perception, particularly when the audience is not familiar with the underlying data generation process. For engineers, this underscores the importance of transparency in how results are presented. The corrected version used indexed values to show relative performance, which is a more honest representation of the cohort's behavior. The takeaway is that even accurate data can mislead if the presentation is not carefully designed to avoid implicit assumptions.