# Law of Large Numbers

### Weak Law Of Large Numbers

Consider a sequence of independent random variables with a common mean ${\displaystyle \mu }$. Suppose that each variable has a variance less or equal the quantity ${\displaystyle \vartheta <\infty }$.

Consider the sample mean

${\displaystyle M_{n}={1 \over {n}}\sum _{i=1}^{n}{x_{i}}}$
Then ${\displaystyle M_{n}{\xrightarrow {P}}\mu }$ as ${\displaystyle n\rightarrow \infty }$.

PROOF

{\displaystyle {\begin{aligned}E(M_{n})&=E\left({1 \over {n}}\sum _{i=1}^{n}{x_{i}}\right)={1 \over {n}}E\left(\sum _{i=1}^{n}{x_{i}}\right)=\\&={1 \over {n}}\sum _{i=1}^{n}E({x_{i}})={1 \over {n}}n\mu =\\&=\mu \end{aligned}}}
{\displaystyle {\begin{aligned}{\text{var}}(M_{n})&={\text{var}}\left({1 \over {n}}\sum _{i=1}^{n}{x_{i}}\right)=\left({1 \over {n}}\right)^{2}{\text{var}}\left(\sum _{i=1}^{n}{x_{i}}\right)=\\&={1 \over {n^{2}}}\sum _{i=1}^{n}{\text{var}}({x_{i}})={1 \over {n^{2}}}n\vartheta =\\&=\vartheta /n\end{aligned}}}

Chebychev's inequality for each random variable ${\displaystyle X}$:

${\displaystyle P(|X-E(X)|\geq \epsilon )\leq {{\text{var}}(x) \over \epsilon ^{2}}}$
applying to sequence ${\displaystyle M_{n}}$:
${\displaystyle P(|M_{n}-\mu |\geq \epsilon )\leq {\vartheta /n \over \epsilon ^{2}}}$
${\displaystyle \Longrightarrow \lim _{n\to \infty }P(|M_{n}-\mu |\geq \epsilon )\leq 0}$
so ${\displaystyle M_{n}{\xrightarrow {P}}\mu }$ as ${\displaystyle n\rightarrow \infty }$.

### Strong Law Of Large Numbers

Consider a sequence of independent and identically distributed random variable ${\displaystyle x_{1},x_{2},...,x_{n}}$ with a common mean ${\displaystyle \mu }$. Then ${\displaystyle M_{n}{\xrightarrow {a.s.}}\mu }$.