---- > [!definition] Definition ([[independent]]) > Let $(\Omega, \mathcal{F}, \mathbb{P})$ be a [[probability|probability space]]. > > - We call a finite collection $(\mathcal{G}_{i})_{i=1}^{n} \subset \mathcal{F}$ of [[σ-algebra|sub-σ-algebras]] **independent** if $\mathbb{P}(A_{1} \cap \dots \cap A_{n})=\mathbb{P}(A_{1}) \cdots \mathbb{P}(A_{n}) \text{ for all }A_{i} \in \mathcal{G}_{i}, i \leq n.$ > - We call an arbitrary collection $(\mathcal{G}_{i})_{i \in I}$ of sub-$\sigma$-algebras of $\mathcal{F}$ **independent** if any finite subcollection of them is. > [!equivalence] Equivalence for $\sigma$-algebras generated by $\pi$-systems. > If $(\mathcal{G}_{i})_{i \in I}$ is such that each $\mathcal{G}_{i}$ is [[σ-algebra generated by a set collection|generated by]] a [[π-system]] $\mathscr{A}_{i} \subset \mathcal{F}$, $\mathcal{G}_{i}=\sigma(\mathscr{A}_{i})$, then $(\mathcal{G}_{i})_{i \in I}$ are independent if and only if $\mathbb{P}(\bigcap_{i \in J}A_{i})=\prod_{i\in J} \mathbb{P}(A_{i}) \text{ for any }A_{i} \in \mathscr{A}_{i}, i \in J$ > for any finite subset $J \subset I$. > [!proof] > Induction with [[Dynkin's π-𝜆 theorem]]. ^proof > [!specialization] Specializing. ([[independent|independent events]]) > Let $(\Omega, \mathcal{F}, \mathbb{P})$ be a [[probability|probability space]] and $\{ A_{k} \}_{k \in \Gamma} \subset \mathcal{F}$ a family of events in $\mathcal{F}$. We call the family $\{ A_{k} \}_{k \in \Gamma}$ **independent** if their [[σ-algebra generated by a set collection|generated]] [[σ-algebra|σ-algebras]] $\{ \sigma(A_{k}) \}_{k \in \Gamma}$ are independent. One can appeal to the $\pi$-system equivalence or check manually[^3] that this happens if and only if $\mathbb{P}(A_{k_{1}} \cap \dots \cap A_{k_{n}})=\mathbb{P}(A_{k_{1}})\cdots \mathbb{P}(A_{k_{n}}) $ whenever $k_{1},\dots,k_{n}$ are distinct elements of $\Gamma$. > [!basicproperties] > - *(subcollections)* Observing we could have $A_{i}=\Omega$ for some indices $i$, we see that passing to a subcollection preserves independence > - *(subalgebras)* If $(\mathcal{G}_{i})_{i \in I}$ are independent and $\mathcal{H}_{i} \subset \mathcal{G}_{i}$ are further sub-$\sigma$-algebras, then $(\mathcal{H}_{i})_{i \in I}$ are clearly also independent. > - *(countable collection)* If $(\mathcal{G}_{n})_{n \in \mathbb{N}}$ is a [[sequence]] of independent $\sigma$-algebras, then for any $A_{n} \in \mathcal{G}_{n}$, $n \in \mathbb{N}$, we have[^2] $\mathbb{P}(\bigcap_{n=1}^{\infty} A_{n})=\prod_{n=1}^{\infty}\mathbb{P}(A_{n}).$ > >- If $\{ A,B \} \subset \mathcal{F}$ are independent events, then so are: > - $\{ \Omega-A, \Omega-B \}$ > - $\{ A, \Omega-B \}$ > - $\{ \Omega-A, B \}$ > > More generally, one can complement any subcollection of a family of events $\{ A_{k} \}_{k \in \Gamma}$ without breaking independence. > > > [!proof]- Proof. > > Using [[De Morgan's Laws]] and [[the exclusion rule]]: $\begin{align} > > \mathbb{P}((\Omega-A) \cap \Omega-B) &= \mathbb{P}(\Omega-(A \cup B)) \\ > > &= 1-\mathbb{P}(A \cup B) \\ > > &= 1 - \mathbb{P}(A) - \mathbb{P}(B ) +\mathbb{P}(A \cap B) \\ > > &= 1 - \mathbb{P}(A) - \mathbb{P}(B) + \mathbb{P}(A) \mathbb{P}(B) \\ > > &= (1-\mathbb{P}(A)) (1- \mathbb{P}(B)) \\ > > &= \mathbb{P}(\Omega-A) \mathbb{P}(\Omega-B). > > \end{align}$ > > And $\begin{align} > > \mathbb{P}(A \cap (\Omega-B)) &= \mathbb{P}(A- A \cap B) \\ > > &= \mathbb{P}(A) - \mathbb{P}(A \cap B) \\ > > &= \mathbb{P}(A) - \mathbb{P}(A) \mathbb{P}(B) \\ > > &=\mathbb{P}(A) (1-\mathbb{P}(B)) \\ > > &= \mathbb{P}(A) \mathbb{P}(\Omega-B). > > \end{align}$ > > (Symmetric reasoning for $\{ \Omega-A,B \}$). > [^2]: This follows from [[measure|continuity from above]] for [[measure]]: since $A_{1} \supset A_{1} \cap A_{2} \supset\dots$, we have $\mathbb{P}(\bigcap_{n=1}^{\infty} A_{n})= \lim (\mathbb{P}(A_{1}), \mathbb{P}(A_{1} \cap A_{2}), \dots).$ We observe that, by independence, the sequence whose limit we are taking is precisely the sequence of partial products $\prod_{n=1}^{N}\mathbb{P}(A_{n})$. > [!basicexample] > - The [[σ-algebra|trivial σ-algebra]] $\{ \Omega, \emptyset \}$ is independent of any other $\sigma$-algebra. Its information content is null. > - More generally, $\mathcal{G} \leq \mathcal{F}$ is independent of itself iff $\mathbb{P}(A) \in \{ 0,1 \}$ for all $A \in \mathcal{G}$.[^1] ^basic-example [^1]: (This could be cleaner, it ain't that deep) Indeed, suppose $\mathcal{G}$ is independent of itself, i.e., for all pairs $A,B \in \mathcal{G}$ we have $\mathbb{P}(A \cap B)=\mathbb{P}(A)\mathbb{P}(B)$. Then $\mathbb{P}(A)=\mathbb{P}(A \cap A)=\mathbb{P}(A)\mathbb{P}(A)$, implying $\mathbb{P}(A)=1$ or $\mathbb{P}(A)=0$. Conversely, suppose $\mathbb{P}(A) \in \{ 0,1 \}$ for all $A \in \mathcal{G}$. Then $\mathbb{P}(A \cap B) \in \{ 0,1 \}$. First case: $\mathbb{P}(A \cap B)=1$. Since $\mathbb{P}(A \cap B)\leq \mathbb{P}(A)$ and $\mathbb{P}(A \cap B) \leq \mathbb{P}(B)$, $\mathbb{P}(A)=\mathbb{P}(B)=1$ and so $\mathbb{P}(A)\mathbb{P}(B)=1$ also. Second case: $\mathbb{P}(A \cap B)=0$. Then $\begin{align} 1&=\mathbb{P}\big( \Omega - (A \cap B) \big) \\ &=\mathbb{P}\big( (\Omega- A) \cup (\Omega - B) \big) \\ &\leq \mathbb{P}(\Omega - A) + \mathbb{P}(\Omega - B) . \end{align}$ We see that $\mathbb{P}(\Omega - A)$ and $\mathbb{P}(\Omega-B)$ can't both be zero, hence $\mathbb{P}(A)$ and $\mathbb{P}(B)$ can't both be one. So $\mathbb{P}(A)\mathbb{P}(B)=0$ as required. [^3]: One direction is immediate. For the other, assume $\mathbb{P}(A_{k_{1}} \cap \cdots \cap A_{k_{n}})=\mathbb{P}(A_{k_{1}}) \cdots \mathbb{P}(A_{k_{n}})$ holds for all $k_{1},\dots,k_{n}$ distinct elements of $\Gamma$. The task is to show that the $\sigma$-algebras $\{ \Omega, \emptyset, A_{k}, A^{c}_{k} \}_{k \in \Gamma}$ are independent. This follows from the basic properties section of the note and complements preserving independence. > [!basicexample] A cautionary example. > > We roll 2 fair dice. Let > - $A$ be "1st die shows 3" > - $B$ be "2nd die shows 4" > - $C$ be "sum of dice values is 7" > > What is $P(A),P(B), P(C)$? Are the [[event]]s independent? > - $P(A) = \frac{1}{6}$ > - $P(B)=\frac{1}{6}$ > - $P(C) = \frac{1}{6}$. > > $P(A \cap B) = \frac{1}{36}$ > $P(A \cap C) = P(A \vert C)P(C) = \frac{1}{36}$ > $P(B \cap C) = \frac{1}{36}$ > > Now, $A,B$ independent because $P(A \cap B) = \frac{1}{6}\frac{1}{6} = \frac{1}{36} = P(A)P(B)$ . However, *the whole family is still not independent* because $P(A \cap B \cap C) = P(C \vert A \cap B)P(A \cap B) = 1 \cdot \frac{1}{36} = \frac{1}{36} \neq P(A)P(B)P(C) = \frac{1}{6}\frac{1}{6}\frac{1}{6} = \frac{1}{216}$. > > [!basicexample] Classic example: circuit failing problem > ![[CleanShot 2022-09-08 at [email protected]]] ^basic-example ---- #### ---- #### References > [!backlink] > ```dataview > TABLE rows.file.link as "Further Reading" > FROM [[]] > FLATTEN file.tags as Tag > WHERE Tag = "#definition" OR Tag = "#theorem" OR Tag = "#MOC" OR Tag = "#proposition" OR Tag = "#axiom" > GROUP BY Tag > ``` > [!frontlink] > ```dataview > TABLE rows.file.link as "Further Reading" > FROM outgoing([[]]) > FLATTEN file.tags as Tag > WHERE Tag = "#definition" OR Tag = "#theorem" OR Tag = "#MOC" OR Tag = "#proposition" OR Tag = "#axiom" > GROUP BY Tag > ``` #analysis/probability-statistics # probability definition Suppose $\Omega$ a [[sample space]]. Let $A_{i} \subset \Omega$ be a (possibly [[uncountably infinite]]) family of [[event]]s in $\Omega$. We say that $\{ A_{i} \}$ are *independent* provided that for any finite $J \subset A_{i}$, $\mathbb{P}(\bigcap_{i \in J}A_{i}) = \prod_{i\in J}P(A_{i}).$ # Classic Example: Circuit Failing Problem #notFormatted