S 2.2 The rules of probability

• We proceed to establish the rules which probabilities must satisfy, given their definitions.

• Both definitions KP2.2 and KP2.3 lead to the same set of rules, by different routes.

• We exploit the frequentist formulation: the route is shorter.

[A] Range of values

Key Point 2.4

The probability P(A|I) lies in the range

$$0 \le P(A\vert I) \le 1$$

with $P(A\vert I) = 1\, (0)$ denoting certainty that A is true (false).

Proof:

P(A|I) is the long-run fraction of I-instances giving A. By its nature (a `fraction') it must lie in the stated range. The figure shows this schematically. P(A|I)=1 means that all instances of I give A so A is certainly true. P(A|I)=0 means that no instances of I give A so A is certainly false.

[B] AND-combinations: general case

• Consider two assertions A₁ and A₂.

• Let $A_1 \cdot A_2$ (alternative notation: $A_1 \framebox {\sc and} A_2$)denote the composite assertion (`conjunction') that A₁ and A₂ are both true.

  Example:
  

  • I: a card has been drawn from a shuffled pack

  • A₁: the card is an ace

  • A₂: the card is a spade

  • $A_1 \cdot A_2$: the card is the ace of spades

• It follows that:

Key Point 2.5

$$P(A_1 \cdot A_2\vert I) =P(A_2\vert A_1\cdot I)P(A_1\vert I) \hspace{1cm} \mbox{(general case)}$$

where $P(A_2\vert A_1\cdot I)$ is the probability of A₂ given I and A₁.

Proof:

• Suppose we do the I-experiment N(I) times, observing A₁ on N(A₁) occasions.

• In the `long-run limit' we identify

  $$N(A_1) \stackrel{=}{\mbox{{\tiny LR}}}P(A_1\vert I) N(I)$$

• Amongst the N(A₁) instances giving A₁ we observe a number (call it $N(A_1 \cdot A_2)$) which also give A₂.

• We identify (in the long-run limit)

  $$N(A_1 \cdot A_2) \stackrel{=}{\mbox{{\tiny LR}}}P(A_2\vert A_1\cdot I) N(A_1)$$

• Thus

  $$\frac{N(A_1 \cdot A_2)}{N(I)} \stackrel{=}{\mbox{{\tiny LR}}}P(A_2\vert A_1\cdot I) \frac{N(A_1)}{N(I)}$$

  or

  $$P(A_1 \cdot A_2\vert I) =P(A_2\vert A_1\cdot I)P(A_1\vert I)$$

  as claimed.

[C] AND-combinations: mutually-independent results

• Suppose that A₁ and A₂ are mutually independent .

  Meaning of mutually independent:

  • In words: the truthfulness of one statement tells us nothing about the truthfulness of the other.

  • In algebra:

    $$P(A_1\vert A_2\cdot I) = P(A_1\vert I) \hspace*{1cm} \mbox{and} \hspace*{1cm} P(A_2\vert A_1\cdot I) = P(A_2\vert I)$$

  Example:
  

  • I: two cards have been drawn from separate shuffled packs

  • A₁: the card from the 1st pack is an ace

  • A₂: the card from the 2nd pack is an ace

  • We might expect that A₁ and A₂ are mutually independent .

• Then as a special case of KP2.5 :

  Key Point 2.6
  

  $$P(A_1 \cdot A_2\vert I) =P(A_2\vert I) \times P(A_1\vert I) ... ...ce{1cm} \mbox{(for $A_1$, $A_2$\space mutually independent)}$$

[D] OR-combinations: general case

• Consider two assertions A₁ and A₂.

• Let $A_1 \framebox {\sc or} A_2$ denote the composite assertion (`disjunction') that at least one of the two statements A₁ and A₂ is true.

Key Point 2.7

$$P(A_1 \framebox {\sc or} A_2\vert I) =P(A_1\vert I) + P(A_2... ... I) -P(A_1 \cdot A_2\vert I) \hspace{1cm} \mbox{(general case)}$$

Proof:

• The result follows from the identity

  $$N(A_1 \framebox {\sc or} A_2) =N(A_1) + N(A_2) -N(A_1 \cdot A_2)$$

Here $N(A_1 \cdot A_2)$ is the number of occasions in which A1 and A2 are both true. We need to subtract this term because its contribution is double-counted (once in N(A1) and once in N(A2)). The figure should make this clear.

[E] OR-combinations: mutually exclusive results

• Suppose that A₁ and A₂ are mutually exclusive .

  Meaning of mutually exclusive:

  • In words: the two statements cannot both be true.

  • In algebra:

    $$P(A_1\cdot A_2\vert I) = 0$$

  Example:
  

  • I: a card has been drawn from a shuffled pack

  • A₁: the card is the ace of diamonds

  • A₂: the card is a spade

  • It is clear that A₁ and A₂ are mutually exclusive .

• Then as a special case of KP2.7 :

  Key Point 2.8
  

  $$P(A_1 \framebox {\sc or} A_2\vert I) = P(A_1\vert I) + P(A_... ...I) \hspace{1cm} \mbox{($A_1$, $A_2$\space mutually exclusive)}$$

[F] Normalisation: mutually exclusive and exhaustive results

• Suppose that the set of $\Omega$ assertions A₁, A₂ ...$A_\Omega$ are mutually exclusive and exhaustive (MEE for short)

  Meaning of exhaustive:

  • In words: at least one of A₁ ...$A_\Omega$ must be true.

  • In algebra:

    $$P(A_1 \framebox {\sc or} A_2 \framebox {\sc or} A_3 \framebox {\sc or} \ldots \framebox {\sc or} A_\Omega\vert I) = 1$$

  Example:
  

  • I: a card has been drawn from a shuffled pack

  • A₁: it is an ace

  • A₂: it is a face card

  • A₃: it is a plain card

  • Then A₁, A₂ and A₃ are MEE with $\Omega =3$.

• Then for such a set:

  Key Point 2.9
  

  The normalisation condition:

  $$\sum_{r=1}^{\Omega} P(A_r \vert I) = 1 \hspace{1cm}$$

Proof:

• Follows from KP2.2 :

  $$\sum_{r=1}^{\Omega} P(A_r \vert I) \stackrel{=}{\mbox{{\tin... ... \sum_{r=1}^{\Omega} \frac{N(A_r)}{N(I)} = \frac{N(I)}{N(I)} =1$$

• We have used the fact that each of the N(I) instances of I must fall into one of the categories associated with the cases $r=1\ldots \Omega$.

Alternatively think of the slices of a cake!