Law of Large Numbers
|
|
The Law of Large Numbers (LLN) is an important result in probability theory and statistics, which formalizes our intuition that the sample average should converge to the true average, as the number of trials increases. For example, the ratio of heads to tails for a fair coin (50/50) should converge to 0.5, as the number of tosses increases. LLN is a statement about the stable long-term behavior of the sample average of a random variable. Therefore, even though after 10 tosses of a coin we might end up with 7 heads and 3 tails, after 10K tosses we should end up with approximately 5K heads and 5K tails.
|
Single Coin
The first plot is a simulation of repeated tosses of a single fair coin and the running sample proportion of heads to tails. We observe that initially the bars are imbalanced, but they quickly stabilize to around 0.5. Though there remain small fluctuations, any deviations from the true theoretical proportion of 0.5 are quick to resolve, as the number of tosses increases.
|
Sampling Distribution
The second plot is a simulation of the sampling distribution. We toss a coin a certain number of times, and take the ratio of heads to tails (sample proportion: $\mathrm{\hat{p}}$). We repeat this for many coins — here 10K of them. As the number of tosses for each coin increases, the distribution of the sample proportions collapses almost surely onto the true proportion: 0.5.
|
Multiple Coins
The third plot unifies the two previous plots. Here we plot the running sample proportion (like in the first plot) for multiple coins (like in the second plot). We can observe on the right hand side, as the number of tosses increases (x-axis), the sampling distribution for these 10 coins converges to the true proportion 0.5 (as it did in the second plot). We observe this phenomenon of the Law of Large Numbers is relatively robust, as even though we have drastically fewer coins here (10 vs 10K), the sampling distribution still collapses to the expectation (0.5), so long as the number of tosses is sufficiently large.
|
|
