# Law of Large Numbers

### Weak Law Of Large Numbers

Consider a sequence of independent random variables with a common mean $\mu$ . Suppose that each variable has a variance less or equal the quantity $\vartheta <\infty$ .

Consider the sample mean

$M_{n}={1 \over {n}}\sum _{i=1}^{n}{x_{i}}$ Then $M_{n}{\xrightarrow {P}}\mu$ as $n\rightarrow \infty$ .

PROOF

{\begin{aligned}E(M_{n})&=E\left({1 \over {n}}\sum _{i=1}^{n}{x_{i}}\right)={1 \over {n}}E\left(\sum _{i=1}^{n}{x_{i}}\right)=\\&={1 \over {n}}\sum _{i=1}^{n}E({x_{i}})={1 \over {n}}n\mu =\\&=\mu \end{aligned}} {\begin{aligned}{\text{var}}(M_{n})&={\text{var}}\left({1 \over {n}}\sum _{i=1}^{n}{x_{i}}\right)=\left({1 \over {n}}\right)^{2}{\text{var}}\left(\sum _{i=1}^{n}{x_{i}}\right)=\\&={1 \over {n^{2}}}\sum _{i=1}^{n}{\text{var}}({x_{i}})={1 \over {n^{2}}}n\vartheta =\\&=\vartheta /n\end{aligned}} Chebychev's inequality for each random variable $X$ :

$P(|X-E(X)|\geq \epsilon )\leq {{\text{var}}(x) \over \epsilon ^{2}}$ applying to sequence $M_{n}$ :
$P(|M_{n}-\mu |\geq \epsilon )\leq {\vartheta /n \over \epsilon ^{2}}$ $\Longrightarrow \lim _{n\to \infty }P(|M_{n}-\mu |\geq \epsilon )\leq 0$ so $M_{n}{\xrightarrow {P}}\mu$ as $n\rightarrow \infty$ .

### Strong Law Of Large Numbers

Consider a sequence of independent and identically distributed random variable $x_{1},x_{2},...,x_{n}$ with a common mean $\mu$ . Then $M_{n}{\xrightarrow {a.s.}}\mu$ .