The Law of Small Numbers

The incorrect belief that the result of small samples can represent their parent population.

People tend to overestimate the results of a small study. For example, if a coin has a 50% chance of landing on heads or tails, the law of small numbers will mistakenly lead someone who believes that tossing a coin 10 times will represent the same probability as tossing it 1000 times.

Image_Caption

An Example of Trading

We use the generated data 1 of a Sharpe ratio, which is the reward to risk ratio. The Sharpe ratio is often used to compare the change in overall risk-return characteristics when a new trade is added to a portfolio.

Assume a backtested strategy, or a strategy to access its success by using historical data, has a true Sharpe ratio of 2, throughout 40 days, and a normal distribution with mean returns of 0.1%.

Image_Caption

Figure 1: Sharpes ratio of 15 trades placed

The above histogram shows that if 15 trades are placed, there will be a lesser Sharpe Ratio than the backtested value (2.0).

The Law of Large Numbers

To avoid this from happening, the law of large numbers can play an important role. We can collect more data, and impute the missing data carefully so that it can conserve correlation structures. Moreover, methods designed for small data are also available out there such as calculating conditional probability using the bayesian approach.

Would it yield the same result as shown in the above figure if we add more numbers of trades? We will do some more experiments with 500 and 3,000 trades.

Image_Caption

Figure 2: 500 trades placed

Image_Caption

Figure 3: 3,000 trades placed

As expected, if we keep increasing the numbers of trades, we will eventually see more wins than losses as the Sharpe ratio seems to approach a true Sharpe ratio eventually.

Conclusion

The results of small samples can lead to misinterpretation of the parent population. In other words, in any data analysis, we need to observe data that is large enough before concluding the result.


References:

1 Law of Large Numbers in Finance Using Python: More imformmation is available here