Formulas to memorise

1Introduction and definitions¶

1.1Basic definitions¶

Union of two events

$P(A \cup B) = P(A) + P(B) - P(A \cap B)$

1.2Permutations and combinations¶

Permutations (i.e. when order does matter)

$\displaystyle P^n_r = \frac{n!}{(n-r)!}$

Combinations (i.e. when order does not matter)

$\displaystyle C^n_r = \frac{n!}{(n-r)!r!}$

1.3Conditional probability and independence¶

Conditional probability

$\displaystyle P(A|B) = \frac{P(A \cap B)}{P(B)}$

Definitions of independence

when $P(B|A) = P(B)$ and $P(A|B) = P(A)$ = $P(A \cap B) = P(A)P(B)$

1.4Bayes' rule and the law of total probability¶

No need to memorise this

2Discrete random variables¶

2.1Basic definitions¶

Properties of a probability function

Always non-negative, and the sum of all the values is 1

Expected value

$\displaystyle E(X) = \sum x P(X = x)$

Variance

$Var(X) = E(X^2) - E(X)^2 = E(X^2) - \mu^2$

2.2Special discrete distributions¶

2.2.1The binomial distribution¶

Probability density function

$P(X = x) = C^n_x p^n q^{n-x}$

Expected value

$E(X) = np$

Variance

$Var(X) = npq$

Moment generating function

$M_X(t) = (pe^t + q)^n$

2.2.2The geometric distribution¶

Probability density function

$P(X = x) = pq^{x-1}$

Expected value

$E(X) = \frac{1}{p}$

Variance

$Var(X) = \frac{q}{p^2}$

Moment generating function

$\displaystyle M_X(t) = \frac{pe^t}{1-qe^t}$

2.2.3The negative binomial distribution¶

Don't need to memorise this

2.2.4The hypergeometric distribution¶

Don't need to memorise this

2.2.5The Poisson distribution¶

Probability density function

$\displaystyle P(X = x) = \frac{\lambda^x e^{-\lambda}}{x!}$

Expected value

$E(X) = \lambda$

Variance

$Var(X) = \lambda$

Moment generating function

$e^{-\lambda(1-e^t)} = e^{\lambda(e^t-1)}$

2.3Moment generating functions¶

The significance of moment generating functions

$M_X^{(n)} = E(X^n)$

3Continuous random variables¶

3.1Distribution functions¶

Properties of a cumulative distribution function $F(y)$

Nondecreasing, from 0 to $\infty$, continuous

3.2Continuous random variables¶

Relationship between distribution and density functions

The distribution function $\displaystyle F(x) = \int_{-\infty}^x f(t) \,dt$

Properties of a probability density function

Non-negative, and the integral over the domain is equal to 1

Expected value for a continuous rv

$\displaystyle E(g(X)) = \int_{-\infty}^{\infty} g(x) f(x) \,dx$

Variance for a continuous rv

$Var(X) = E(X^2) - \mu^2$ and $Var(aX + b) = a^2 Var(X)$

Moment generating function for a continuous rv

$\displaystyle M_X(t) = E(e^{tx}) = \int_{-\infty}^{\infty} e^{tx} f(x) \,dx$

3.3Special continuous distributions¶

3.3.1The uniform distribution¶

Probability density function

$\displaystyle f(x) = \begin{cases} 0 & \text{ if } -\infty < x < a \\ \frac{1}{b-a} & \text{ if } a \leq x \leq b \\ 0 & \text{ if } b < x < \infty \end{cases}$

Expected value

$\displaystyle E(X) = \frac{a+b}{2}$

Variance

$\displaystyle Var(X) = \frac{(b-a)^2}{12}$

Distribution function

$\displaystyle F(x) = \begin{cases} 0 & \text{ if } x < a \\ \frac{x-a}{b-a} & \text{ if } a \leq x \leq b \\ 1 & \text{ if } x > b \end{cases}$

Moment generating function

$\displaystyle M_X(t) = \frac{e^{tb} - e^{ta}}{t(b-a)}$

3.3.2The exponential distribution¶

Probability density function

$\displaystyle f(x) = \begin{cases} 0 & \text{ if } y \leq 0 \\ \frac{1}{\beta} e^{-y/\beta} & \text{ if } y >0 \end{cases}$

Expected value

$E(X) = \beta$

Variance

$Var(X) = \beta^2$

Distribution function

$\displaystyle F(x) = \begin{cases} 0 & \text{ if } y < 0 \\ 1 - e^{-y / \beta} & \text{ if } y \geq 0 \end{cases}$

Moment generating function

$\displaystyle M_X(t) = \frac{1}{1-\beta t}$

Memoryless property (definitive feature)

$P(Y > s+t | Y > s) = P(Y > t)$

3.3.3The gamma distribution¶

Gamma function

$\displaystyle \Gamma(\alpha) = \int_0^{\infty} x^{\alpha - 1}e^{-\alpha} \,dx$

Properties of the gamma function

$\Gamma(1) = 1$, $\Gamma(n+ 1) = n\Gamma(n)$ = n!, $\Gamma(\frac{1}{2}) = \sqrt{\pi}$

Probability density function

$\displaystyle f(x) = \begin{cases} 0 & \text{ if } x \leq 0 \\ \frac{1}{\Gamma(\alpha)\beta^{\alpha}} x^{\alpha-1}e^{-x/\beta} & \text{ if } x > 0 \end{cases}$

Expected value

$E(X) = \alpha\beta$

Variance

$Var(X) = \alpha \beta^2$

Moment generating function

$\displaystyle M_X(t) = \frac{1}{(1 - \beta t)^{\alpha}}$ for $t < \frac{1}{\beta}$

3.3.4The normal distribution¶

Probability density function

$\displaystyle f(x) = \frac{1}{\sqrt{2\sigma^2 \pi}}e^{-\frac{1}{2} \left ( \frac{(x-\mu)^2}{\sigma^2} \right ) }, \quad -\infty < x < \infty$

Expected value

$E(X) = \mu$

Variance

$Var(X) = \sigma^2$

Moment generating function

$\displaystyle M_X(t) = E(e^{tX}) = e^{\mu t + \frac{\sigma^2 t^2}{2}} $

3.3.5The beta distribution¶

Don't need to memorise this

3.3.6The Cauchy distribution¶

Probability density function

$\displaystyle f(x) = \frac{1}{\pi} \frac{1}{1 + x^2} \quad -\infty < x < \infty$

Expected value

Does not exist

Variance

Does not exist

3.4Chebychev's inequality¶

Markov's inequality

$\displaystyle P(X > \epsilon) \leq \frac{E(X)}{\epsilon}$

Chebychev's inequality

$\displaystyle P(|X - \mu| > \epsilon) \leq \frac{\sigma^2}{\epsilon^2}$

4Multivariate distributions¶

4.1Definitions¶

Joint probability density function

$f(y_1, y_2) = P(Y_1 = y_1, Y_2 = y_2)$

Joint distribution function

$F(y_1, y_2) = P(Y_1 \leq y_1, Y_2 \leq y_2)$

Probability in a region

$\displaystyle P(a_1 < Y_1 < b_1, a_2 < Y_2 < b_2) = \int_{a_2}^{b^2} \int_{a_1}^{b_1} f(y_1, y_2) \,dy_1 \,dy_2$

4.2Marginal and conditional distributions¶

Marginal probability function

For the discrete case

Marginal density function

For the continuous case; $\displaystyle f_1(y_1) = \int f(y_1, y_2)\,dy_2$

Conditional probability density function

$\displaystyle f_{12}(y_1|y_2) = P(Y_1 = y_2 | Y_2 = y_2) = \frac{P(Y_1 = y, Y_2 = y_2)}{P(Y_2 = y_2)} = \frac{f(y_1, y_2)}{f_2(y_2)}$

Conditional expectation

$\displaystyle E(g(Y_1)|Y_2 = y_2) = \int g(y_1) f_{12}(y_1|y_2)\,dy_1$

4.3Independent random variables¶

Independence

If $f(y_1, y_2) = f_1(y_1) f_2(y_2)$ and the same for the cumulative version

4.4The expected value of a fuction of random variables¶

Expected value

$\displaystyle E(g(Y_1, Y_2)) = \int \int g(y_1, y_2) f(y_1, y_2) \,dy_1 \,dy_2$

4.5Special theorems¶

Can all be derived.

4.6Covariance¶

Covariance

$Cov(Y_1, Y_2) = E(Y_1 Y_2) - \mu_1 \mu_2$

Correlation coefficient

$\displaystyle \rho = \frac{Cov(Y_1, Y_2)}{\sigma_1 \sigma_2}$

4.7The expected value and variance of linear functions of random variables¶

Can all be derived

4.8The multinomial distribution¶

Joint probability density function:

$\displaystyle P(X_1 = x_1, \ldots, X_k = x_k) = \frac{n!}{x_1! \ldots x_k!}p_1^{x_1} \ldots p_k^{x_k}$

4.9More than two random variables¶

Whatever

5Functions of random variables¶

5.1Functions of continuous random variables¶

5.1.1The univariate case¶

5.1.2The multivariate case¶

5.2Sums of independent random variables¶

5.2.1The discrete case¶

Poisson

$X+Y$ where both are Poisson distributions is also one with a mean that is the sum of their means

5.2.2The jointly continuous case¶

Eh

5.3The moment generating function method¶

Gamma

$X + Y$ with Gamma$(\alpha_1, \beta)$ and Gamma$(\alpha_2, \beta)$ results in Gamma$(\alpha_1+\alpha_2, \beta)$

Exponential

If same $\beta$, $X+Y$ results in Gamma$(2, \beta)$

Normal

$X+Y$ for $N(\mu_x, \sigma^2_x)$ and $N(\mu_y, \sigma^2_y)$ results in $N(\mu_x + \mu_y, \sigma^2_x + \sigma^2_y)$

5.3.1A summary of moment generating functions¶

6Law of large numbers and the central limit theorem¶

6.1Law of large numbers¶

Not important

6.2The central limit theorem¶

Central limit theorem

$\displaystyle \frac{S_n - n \mu}{\sigma}{\sqrt{n}}$ maps to $N(0, 1)$

7Information theory¶

Don't need to know this