Luck, skill and a convincing story

Ask what was selected, what disappeared and what was knowable before the winning result.

Where did the winner come from?

32 fictional strategies with no skill advantage. Each has a 50% chance of beating its benchmark each year, independently across strategies and years.

Choose before results

Fix one strategy to follow before any annual wins are known.

Expected number still winning every year since the start, not observed results. All bars use a scale of 32 strategies.

Before results32
After year 116
After year 28
After year 34
After year 42
After year 51

Choose after results: advertise a strategy with five consecutive annual wins, if any exists.
Showing only that record leaves the other original candidates out of the story.

The expected count with five consecutive annual wins is one. An actual experiment could produce none, one or several; this experiment does not estimate the chance that a real manager’s success is luck.

Choosing after results gives less evidence than identifying a winner beforehand. A five-year record looks different when you can see how many strategies had a chance to produce it.

Expected count and real-world limits

Five wins have probability (1/2)⁵ = 1/32; across 32 strategies, the expected count is 32 × 1/32 = 1.

Real strategies share holdings, risks and periods; their probabilities are not known to be 50%. A winning fund, a fortunate share purchase and a twenty-year model chart all mix choices with events outside the decision-maker’s control.

Include the choices that disappeared.

Ten invented funds, A–J. Complete records through closure establish three benchmark winners, five surviving non-winners and two closed non-winners.

Original opportunity set: all ten records

Still on today's list

AWinBWinCWinDOtherEOtherFOtherGOtherHOther

Archived after closure

IClosed · non-winnerJClosed · non-winner

Include the archive: 3 / 10 = 30% winners
Exclude the archive: 3 / 8 = 37.5% winners

Same winners A, B and C; only the denominator changes. I and J are excluded only from the second calculation. Closure alone would not tell us whether a fund won or lost.

The denominator changes the story. Today’s list can omit funds closed or merged since the earlier decision: survivorship bias.

Missing records and persistence

Specify treatment of missing records, closures and mergers. SPIVA includes liquidated or merged funds. Its persistence research asks separately whether earlier leaders stay ahead later; neither study establishes a particular fund’s future result.

A smooth chart can be the winner of a historical search.

Lupa paints an unfinished target ring around a dart already embedded in a wall.

A backtest applies rules to past data without necessarily investing real money. Trying many markets, dates or switching rules and showing the best can fit accidents that future markets will not repeat.

What was knowable and achievable?

Ask when rules were fixed and what information existed at each simulated date. Today’s successful companies, revised economic data or later fund histories can introduce hindsight. A pre-specified assessment differs from rules chosen after inspecting results.

Unavailable transaction prices and omitted costs undermine reproducibility. The SEC distinguishes hypothetical from actual performance and warns against selecting favourable history. Asking these questions does not require building a testing system.

A profitable gamble did not make the rent safe.

Before investing: Next year’s rent must be paid. The money is put into one company; its future price is unknown.
After the result: The investment made a gain. That establishes profit; the rent obligation was exposed to a loss when the decision was made.

A convincing success story about product, management and market may also have fitted disappointing companies. Reconstruct the evidence and alternatives available before the outcome.

Keep the earlier rationale

Fischhoff’s outcome-knowledge research, first published in 1975 and reprinted in 2003, describes hindsight bias: known results feel more predictable than they were.

An earlier rationale records expectations, recognised risks and contrary evidence. A story rewritten after every result can explain almost anything.

Review the record and the decisions.

Return to the same rent decision. Ask two separate questions:

Was the decision suitable when made?: Check the rent deadline and the risk of relying on one company’s future price to meet it.
What happened to the investment?: It made a gain. Review that result against a suitable after-cost comparison, without rewriting what was known beforehand.

A sound process can disappoint; a poor decision can profit. That neither makes performance irrelevant nor excuses poor results indefinitely.

Check adherence to rules, understood risks and a suitable after-cost comparison. Hagstrom’s process/outcomes discussion explains why short-term returns alone cannot settle decision quality. An explanation should be assessable before the next result; acknowledging a strong record need not lead to buying it.

A historical sample can help examine experiences and assumptions. Selecting only the winners cannot justify assigning their rate to a future goal.

Establishing repeatability may take more work than a low-maintenance portfolio needs. Next, distinguish relevant news from investment sales claims.

Check your understanding

Try each question, then check your answer.

Who was included in this study?

A study of ten-year fund performance includes only funds still operating today. Which statement identifies the selection problem?

Ten-year records for every surviving fund make earlier fund closures irrelevant to the study.
Closed or merged funds and their known records may have been omitted.
A surviving fund's full ten-year record is a valid replacement for the actual histories of funds merged into it.

Show the explanation

Correct answer: Closed or merged funds and their known records may have been omitted.

Funds that disappeared were part of the opportunity set at the earlier date. Omitting them can make past choices look more successful than they were. Reconstruct the original set and how missing records were handled. A surviving fund's full record cannot simply stand in for each predecessor's actual history.

Return to the explanation

What does the profit establish?

A highly concentrated purchase makes money after its buyer ignored the goal and possible losses. Which assessment separates the outcome from the decision process?

The purchase met its goal because a profit was recorded, even though the spending requirement was never checked.
The decision was sound if its return beat a diversified comparison over the same dates.
The outcome was profitable; assessing the process still requires the facts, constraints and reasoning available at purchase.