S 4.3 The Gaussian distribution

[A] Statement and properties:

• The Gaussian (or `normal') distribution is a core ingredient of much of both theoretical and experimental science.

• It is most naturally introduced in the context of a continuous random variable, and is thus described by a probability density.

• In the first place we simply define its form. Then we shall show that it can be be viewed as a special case of the binomial distribution.

• We shall subsequently see why its role extends much further.

Key Point 4.4

The function

$$f(x) = (2 \pi \sigma ^2)^{-1/2}e^{-(x-\mu)^2/(2 \sigma ^2)} \equiv f^{G}(x; \mu , \sigma)$$

is the PDFof a gaussian-distributed random variable of mean $\mu$ and variance $\sigma^2$.

Commentary:

• This is a definition : there is as yet no assertion about the context in which it is useful.

• Thus at this point the only task is to show that KP4.4 is self-consistent.

• The properties to be demonstrated are:

Key Point 4.5

Properties of the gaussian PDF

$$\int_{-\infty}^{\infty} f^{G}(x; \mu , \sigma) dx =1 \hspace... ...nfty}^{\infty} f^{G}(x; \mu , \sigma) (x -\mu)^2 dx = \sigma ^2$$

specifying, respectively, the normalisation, the mean and the variance.

• These properties follow by appeal to the standard-but-significant integral:

  The third most important integral in physics ?

  
  

$$\int_{-\infty}^{\infty} e^{-\alpha z^2} dz = \sqrt{\frac{\pi}{\alpha}} \mbox{\hspace*{1cm}for any } \alpha \gt$$ (4.6)

Proof:

• Normalisation: make the change of variables

  $$z = x-\mu$$

  Then using Eq4.6
  $$\begin{eqnarray*}\int_{-\infty}^{\infty} f^{G}(x;\mu, \sigma) dx &=& (2 \pi \s... ...\ &=&(2 \pi \sigma ^2)^{-1/2}\times \sqrt{2 \pi \sigma ^2} = 1\end{eqnarray*}$$
  
  
  
  
  

• Mean: appeal to symmetry; $\langle x \rangle =\mu$ follows immediately

• Variance: integrate by parts
  $$\begin{eqnarray*}V[x]&=& \int_{-\infty}^{\infty} f^{G}(x;\mu, \sigma)(x-\mu)^2... ...ma^2 \int_{-\infty}^{\infty}e^{-z^2/(2 \sigma ^2)} dz =\sigma^2\end{eqnarray*}$$
  
  
  
  
  

• The last step uses the normalisation condition.

Observe the key features of fG: It is specified by $\mu$ and $\sigma$; $\mu$ locates its (single) maximum and is a centre of symmetry; $\sigma$ measures its `width'.

[B] The standard form

Key Point 4.6

The PDF $f^{G}(x; \mu, \sigma)$ of any gaussian-distributed variable of (arbitrary) mean $\mu$ and variance $\sigma^2$ can be written in the form:

$$f^{G}(x; \mu, \sigma) = \sigma^{-1} \phi(\frac{x-\mu}{\sigma})$$

where

$$\phi(z) \equiv \frac{1}{\sqrt{2\pi}} e^{-z^2/2} \equiv f^{G}(z;0,1)$$

is a gaussian of zero mean and unit variance.

Proof:

• Recall the explicit form

  $$f^{G}(x; \mu, \sigma) = (2 \pi \sigma ^2)^{-1/2}e^{-(x-\mu)^2/(2 \sigma ^2)}$$

• Define the scaled and shifted variable

  $$z\equiv \frac{x-\mu}{\sigma}$$

• Then

  $$f^{G}(x; \mu, \sigma) = \sigma^{-1} \times \frac{1} {\sqrt{2\pi}} e^{-z^2/2} = \sigma^{-1} \phi(z)$$

  as claimed.

• The function $\phi(z)$ is called the standard Gaussian; a variable (like z) with distribution $\phi$ is called a `standard gaussian variable'.

  Note: the symbol here is small -phi

• Its values are tabulated and may be used to evaluate the PDF of any gaussian-distributed variable.

• The related function

  $$\Phi (z) = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^{z} e^{-u^2/2} du =\int_{-\infty}^{z} \phi(u) du$$ (4.7)

  gives the probability that the variable Z has value less than z.

• This function is called the cumulative standard gaussian.

  Note: the symbol here is big -phi

• The following figures show the relationships between the two functions.
  

• The value of $\Phi$ at any z (eg marked point) gives the area under $\phi$ to the left of that z (shaded).

• By inspection $\Phi$ satisfies

  $$\Phi(z) +\Phi(-z) =1$$ (4.8)

• The values of $\Phi(z)$ are tabulated and may be used to characterise any gaussian variable.

  A lot of useful numbers

  
  

Key Point 4.7

For any gaussian-distributed random variable (GRV) X of mean $\mu$ and variance $\sigma^2$

$$P(X < x)= \Phi \left(\frac{x-\mu}{\sigma} \right)$$

gives the (cumulative) probability that the variable X has value less than x.

Proof:

From its definition it follows that

$$\begin{eqnarray*}P(X<x) & =& \int_{-\infty}^{x} f^{G}(x^{\prime}; \mu, \sigma)... ...a} e^{-u^2/2}du \\ &=& \Phi \left(\frac{x-\mu}{\sigma} \right)\end{eqnarray*}$$

• Using the tabulated data and Eq4.8 we deduce that for any GRV:

  • The probability that X lies within $\pm \sigma$ of its mean is
    $$\begin{eqnarray*}P(\mu-\sigma <X < \mu +\sigma)&=& P(X < \mu +\sigma) -P(X < \mu -\sigma)\\ &=&\Phi(1) - \Phi(-1)\\ &=&2\Phi(1)-1 =0.68 \end{eqnarray*}$$
    
    
    
    
    

  • The probability that X lies more than $\pm 3 \sigma$ from its mean is

    $$P(X<\mu-3\sigma) + P(X\gt\mu+3\sigma) = 2 \times [1 -\Phi(3)] = 0.0026$$

• These results underpin experimental science (S6 ) and the award of Nobel Prizes.

[C] The gaussian PDF as a limit of the binomial distribution

• The Gaussian distribution is a special case of the binomial distribution. It is also much more than this.

• It emerges from the binomial distribution in the limit in which the mean number of successes Np is large (tends to infinity)

• This requires that N itself is large enough (while p is arbitrary).

• The explicit claim is:

  Key Point 4.8
  

  In the limit $Np \rightarrow \infty$the binomial distribution may be written as

  $$p(m) = (2 \pi \sigma ^2)^{-1/2}e^{-(m-\mu)^2/(2 \sigma ^2)} = f^{G}(m; \mu , \sigma)$$

  where $\sigma \equiv \sigma[m]$ and $\mu \equiv \langle m \rangle$.

• A formal proof involves some `tedious-but-straightforward algebra'.

• An empirical demonstration follows:

  For $\mu = \langle m \rangle =5$, p(m) and fG(m) are already close to one another at discrete m	For $\mu = \langle m \rangle =20$ they agree to 1 part in 102.