Probability theory

Random phenomena are observed by means of experiments. Each experiment results in an outcome $\omega$. The collection of all outcomes is a sample space $\Omega$. Any subset of $\Omega$ is an event.

The collection $\mathcal{F}$ of events to which a probability is assigned is not always identical to the collection of all subsets of $\Omega$. The requirement on $\mathcal{F}$ is that it should be a $\sigma$-field:

• F contains the sample space Ω.
• if event $A\in \mathcal{F}$, then $\Omega\backslash A\in \mathcal{F}$.
• if $A_i\in \mathcal{F}$, then $\bigcup_{i=1}^\infty A_i \in \mathcal{F}$.

Probability measure $P:\mathcal{F} \rightarrow [0,1]$ is a function on the $\sigma$-field $\mathcal{F}$:

• The measure of entire sample space is equal to one
• $P$ is countably additive. A random variable $x:\Omega \rightarrow \mathcal{F}$ maps from the sample space to events. We can now define the probability $P$ of a discrete random variable $x$ belonging to the event $x_i \in A$ where $A$ is an event.

If $\Omega$ is countable we almost always define $\mathcal {F}$ as the power set of $\Omega$, i.e.\ $\mathcal {F} = 2^{\Omega}$.

Let’s suppose we have $2$ discrete random variables, $x_1,x_2$, with $2$ states each. Therefore $\textbf{x}=(x_1,x_2):\Omega \rightarrow \mathbb{R}^2$. The range of $\textbf{x}$ is limited to $2\times 2=4$ possible values. Any probability distribution for $\textbf{x}$ can be written as a set of 4 probabilities, one for each one of those values. Note that we also have $\mathcal{F}^{tot} = \mathcal{F}_1 \otimes \mathcal{F}_2 =2^{\Omega_1}\otimes 2^{\Omega_2}$:

$\mathcal{F}^{tot} = \Bigg(\emptyset,[x^{(1)}_0],[x^{(1)}_1],[x^{(2)}_0],[x^{(2)}_1],[x^{(1)}_0,x^{(2)}_0],[x^{(1)}_0,x^{(2)}_1],[x^{(1)}_1,x^{(2)}_0],[x^{(1)}_1,x^{(2)}_1],$

$[x^{(1)}_0,\Omega_2],[x^{(1)}_1,\Omega_2],[\Omega_1,x^{(2)}_0],[\Omega_1,x^{(2)}_1],[\Omega_1,\Omega_2]\Bigg)$.