Properties of Sample Mean

If we have sequence of random variables \(X_1, X_2, ... X_n\), we can define Sample Mean \(\bar X\) as their average:

$$\bar X = {1 \over n}(X_1 + X_2 + ... + X_n)$$

The mean and variance of \(\bar X\) are easily calculated by using basic theorems.

So, let \(X_1, X_2, ... X_n\) be a random sample from a distribution with mean μ and variance \(σ^2\) and \(\bar X\) - sample mean. Then \(E(\bar X) = μ\) and \(Var(\bar X) = σ^2 / n\).

Proof: If we know for Y = aX + b, we have E(Y) = aE(X) + b and \(E(X_1 + ... + X_n) = E(X_1) + ... + E(X_n)\), it follows that:

$$E(\bar X) = {1 \over n}\sum_{i=1}^n E(X_i) = {1 \over n} * nμ = μ.$$

Furthermore, since these random variables are independent and we know for Y = aX + b, we have \(Var(X) = a^2Var(X)\) and \(Var(X_1 + ... + X_n) = Var(X_1) + ... + Var(X_n)\), it follows that:

$$Var(\bar X) = {1 \over n^2}Var(\sum_{i=1}^n X_i)$$

$$= {1 \over n^2}\sum_{i=1}^n Var(X_i) = {1 \over n^2} * nσ^2 = {σ^2 \over n}.$$

In words, the mean of \(\bar X\) is equal to the mean of the distribution from which the random sample was drawn, but the variance of \(\bar X\) is only \({1 \over n}\) times the variance of that distribution. It follows that the probability distribution of \(\bar X\) will be more concentrated around the mean value than was the original one. In other words, the sample mean \(\bar X\) is more likely to be close to the mean than is the value of just a single observation \(\bar X\) from the given distribution.

By applying Chebyshev inequality to \(\bar X\), we get:

$$P(|\bar X - μ| \ge t) \le {σ^2 \over nt^2}.$$