Ch3: Conditional probability and independence

Conditional probability

Conditional probability is the probability of one event occurring with some relationship to one or more other events.

Definition conditional probablity

When $A$ and $B$ are events and $P(B)$ > 0, then we define:

$$\mathit{P}(\mathit{A}\mid \mathit{B})=\frac{\mathit{P}(\mathit{A}\mathit{B})}{\mathit{P}(\mathit{B})}$$

as the (conditional) probability of $A$ under condition of $B$ (or: the (conditional) probability of $A$ given $B$).

The following Multiplication rule follows:

Muliplication Rule

Independent Events $P(X\cap Y)=P(X)\cdot P(Y)$

Dependent Events $P(X\cap Y)=P(Y)\cdot P(X\mid Y)$

Law of total probability and Bayes' rule

The law of total probability states that If $\left\{S_i\right\}$ is a partition of $S$ such that $P\left(S_i\right)>0$ for all $i$, then for each event $A$ we have:

$$\mathit{P}(\mathit{A})=P(A\mid S_1)\cdot P\left(S_1\right)+P(A\mid S_2)\cdot P\left(S_2\right)+\cdots =\sum _i\mathit{P}(\mathit{A}\mid {\mathit{S}}_i)\cdot \mathit{P}\left({\mathit{S}}_i\right)$$

Bayes' info

$P(X\mid Y)=\frac{P(X\cap Y)}{P(Y)}$

Note that the Bayes' info can be used in combination with the multiplication rules for example with dependent events:

$$P(X\cap Y)=P(Y)P(X\mid Y)$$$$\frac{P(X\cap Y)}{P(Y)}=\frac{P(Y)P(X\mid Y)}{P(Y)}$$

Independence of events and random variables

Independence

The events 𝐴 and 𝐵 are independent when: $\mathit{P}(\mathit{A}\mathit{B})=\mathit{P}(\mathit{A})\cdot \mathit{P}(\mathit{B})$

Bernoulli trials

A series of experiments is called Bernoulli experiments or trials if

1. each experiment has two possible outcomes, often denoted with 'Success' and 'Failure',
2. the experiments are independent and
3. the probability of success is the same for each experiment.

From this follows the Binomial formula If $X$ is the number of successes in $n$ Bernoulli experiments with success probability $p$, then: $\mathit{P}(\mathit{X}=\mathit{k})=(\genfrac{}{}{0ex}{}{\mathit{n}}{\mathit{k}}){\mathit{p}}^k(\mathbf{1}-\mathit{p}{)}^{n-k}$, where $k=0,1,2,\dots ,n$

If we conduct Bernoulli trials with success probability $p$ until a success occurs and $X$ is the number of required trials, then:

$$\mathit{P}(\mathit{X}=\mathit{k})=(\mathbf{1}-\mathit{p}{)}^{k-1}\mathit{p},\text{ where }k=1,2,3,\dots$$

Remember that $$p^{k}(1-p)^{n-k}$$ is the probability that the first $k$ trials are successful and the last $n$ − $k$ are failures and $(\genfrac{}{}{0ex}{}{n}{k})$ the number of possible orders of $k$ successes and $n$ − $k$ failures.