Complexity Thinking

In my previous post , I examined various weighting strategies within the S&P 500’s top 20 stocks and illustrated the phenomenon of regression toward the mean in financial markets. In this post, I take a different approach: applying a genetic algorithm (GA) to the same problem. The question is whether an evolutionary process — driven by selection, crossover, and mutation — will reveal similar patterns of convergence and mean reversion.

In my last post , I introduced the principle of regression toward the mean — the tendency for extreme outcomes to move closer to average over time. In this article, I’ll illustrate how this principle plays out in investing, using a backtested comparison of portfolio strategies across the S&P 500 since 1990.

Regression toward the mean is one of the most common — and most misunderstood — phenomena in everyday life. People often mix it up with the gambler’s fallacy or with causal stories (e.g., “they must have gotten lucky” or “they must be slacking now”). In plain terms, regression toward the mean is a statement about probabilities: after an extreme observation, a less extreme observation is usually much more likely than an even more extreme one.

Streaks feel meaningful. After a run of good luck, we grow bolder; after a streak of losses, we tell ourselves a win must be “due.” Two competing intuitions emerge from that feeling: the hot‑hand belief (success breeds success) and the gambler’s fallacy (long losses make a win more likely). They are logically opposed — if past wins truly make future wins more likely, then the gambler’s fallacy claim that losses make future wins more likely must be false, and vice versa — yet people ironically often endorse both statements. In sports, the existence of a genuine hot hand remains debated; in pure chance settings like many casino games, any momentum is an illusion. In this post, I use two simple agent‑based models to make that illusion concrete. The first model tests whether streaks change next‑trial probabilities, and the second shows how streak‑driven betting rules affect wealth and survival when the game has a small house edge. At the end, you can try both models yourself and reproduce ...

Retrospective studies are tempting: they use existing data, are cheaper than experiments, and can point to interesting patterns. But patterns are not causes. Two very common biases — survivorship bias and self-selection bias — quietly distort many retrospective findings so that results look stronger, clearer, and more actionable than they really are. Below I explain each bias with vivid examples, show how they sneak into everyday research, and offer practical ways to judge and improve retrospective claims.