Student’s t Distribution

Definition

1.For inference about the mean of a normal population, the standard normal distribution $\mathcal{N}(0,1)$ is the reference model when the population variance is known. When the variance is unknown and has to be estimated from the sample, the estimate has its own sampling error, and the standard normal distribution understates the spread of the resulting statistic. Student's $t$ distribution is the reference model that accounts for this extra variability.

2.The continuous random variable $T$ is defined as the ratio:

$$T = \frac{Z}{\sqrt{V/k}}$$

4.Since the expected value of $V$ is $E(V)=k,$ the square of the denominator has mean $1.$ The distribution of $T$ remains symmetric about zero and has a wider spread because the denominator is random. Since $Z$ and $V$ are independent, the density of their ratio is computable in closed form.

5.The name comes from the pseudonym of William Sealy Gosset, a chemist at the Guinness brewery in Dublin. His 1908 paper "The Probable Error of a Mean" appeared in Biometrika under the name "Student".

Density of the distribution

6.For a variable $T$ with $k$ degrees of freedom the probability density function is:

$$f(t;k) = \frac{\Gamma\left(\frac{k+1}{2}\right)}{\Gamma\left(\frac{k}{2}\right)\sqrt{k\pi}}\left(1+\frac{t^{2}}{k}\right)^{-\frac{k+1}{2}}$$

7.Using the beta function $B(a,b)=\Gamma(a)\Gamma(b)/\Gamma(a+b)$ and the identity $\Gamma(1/2)=\sqrt{\pi}$ for the gamma function, the density has the equivalent form:

$$f(t;k) = \frac{1}{\sqrt{k}B\left(\frac{k}{2},\frac{1}{2}\right)}\left(1+\frac{t^{2}}{k}\right)^{-\frac{k+1}{2}}$$

8.Both forms depend on $t$ only through $t^{2},$ so the graph is symmetric about the origin. For large $|t|,$ the density decays like $|t|^{-(k+1)},$ a power law instead of the exponential decay $e^{-t^{2}/2}$ of the normal density. The tails are therefore heavier for small values of $k,$ and the shape of the curve depends only on $k.$

9.For $k=1$ the gamma factors are $\Gamma(1)=1$ and $\Gamma(1/2)=\sqrt{\pi},$ and the density is:

$$f(t;1) = \frac{1}{\pi\left(1+t^{2}\right)}$$

10.This is the Cauchy density, whose mean does not exist. At the other end, as $k \to \infty$ the second factor tends to $e^{-t^{2}/2}$ and the normalizing constant tends to $1/\sqrt{2\pi},$ so the density converges pointwise to the standard normal density.

11.

The curves become more concentrated around the origin as the degrees of freedom grow and approach the standard normal curve.

12.The graph of $f(t;k)$ has four properties that follow from the expression above.

Mean, variance, and kurtosis

14.For $T \sim t_k$ the density, mean and variance, and excess kurtosis are:

16.The conditions on $k$ are conditions for convergence of the defining improper integrals. The moment of order $m$ requires the integrability of $|t|^{m}f(t;k),$ which behaves like $|t|^{m-k-1}$ at infinity and is integrable when $k>m.$ The mean exists for $k>1,$ the variance for $k>2,$ and the excess kurtosis, which needs the fourth moment, for $k>4.$ For $1

17.Writing the variance as $1+2/(k-2)$ separates it into the variance of the standard normal distribution and a positive excess that vanishes as $k$ grows. The coefficient $\gamma_{2}$ subtracts $3$ from the fourth standardized moment because that moment is equal to $3$ for every normal distribution, so $\gamma_{2}=0$ is the normal reference. For the $t$ distribution $\gamma_{2}=6/(k-4)$ is positive at every admissible $k$ and tends to $0.$ After a $t$ variable is rescaled to have variance $1,$ its density has a higher value at the origin and heavier tails than the standard normal density, and these differences decrease as $k$ grows. With $k=5,$ a $t$ variable lies more than four standard deviations from its mean with probability $3.6\cdot10^{-3},$ against $6.3\cdot10^{-5}$ for the normal model. The skewness is $0$ by symmetry whenever the third moment exists.

Cumulative distribution function

18.The cumulative distribution function of $T$ is given by:

$$F(t;k) = \int_{-\infty}^{t} f(u;k) \ du$$

19.The integrand has no elementary primitive for general $k,$ and the standard closed form uses the regularized beta function $I_x(a,b)=B(x;a,b)/B(a,b),$ the ratio of the incomplete beta function to the complete one. For $t>0$ the substitution $u=\sqrt{k}\sqrt{(1-x)/x}$ maps the tail integral onto an incomplete beta integral and gives:

$$F(t;k) = 1-\frac{1}{2}I_{\frac{k}{k+t^{2}}}\left(\frac{k}{2},\frac{1}{2}\right)$$

20.Symmetry extends this to negative arguments through $F(-t;k)=1-F(t;k).$ Critical values are quantiles computed numerically and traditionally listed in t-tables.

The t statistic for a sample mean

21.Let $X_1,X_2,\dots,X_n$ be independent observations from $\mathcal{N}(\mu,\sigma^{2}),$ with sample mean $\bar X$ and unbiased sample variance:

$$S^{2} = \frac{1}{n-1}\sum_{i=1}^{n}\left(X_i-\bar X\right)^{2}$$

22.The sample mean standardized with the known $\sigma$ is a standard normal variable, and the scaled sample variance is an independent chi-square variable:

$$Z = \frac{\bar X-\mu}{\sigma/\sqrt{n}} \sim \mathcal{N}(0,1), \quad V = \frac{(n-1)S^{2}}{\sigma^{2}} \sim \chi^{2}_{n-1}$$

23.Substituting these two variables into the definition of $T$ with $k=n-1$ gives:

$$ \begin{align} T &= \frac{Z}{\sqrt{V/(n-1)}} \\[6pt] &= \frac{\left(\bar X-\mu\right)\sqrt{n}/\sigma}{\sqrt{S^{2}/\sigma^{2}}} \\[6pt] &= \frac{\bar X-\mu}{S/\sqrt{n}} \end{align} $$

24.The unknown $\sigma$ cancels, and the random variable on the last line has Student's $t$ distribution with $n-1$ degrees of freedom. In a hypothesis test the null hypothesis specifies $\mu=\mu_0,$ and the statistic $(\bar X-\mu_0)/(S/\sqrt{n})$ is computable from the sample. One degree of freedom is lost with respect to the sample size because the deviations $X_i-\bar X$ satisfy the constraint $\sum_{i=1}^{n}\left(X_i-\bar X\right)=0,$ so only $n-1$ of them can be assigned independently.

Symmetry of the t distribution

25.Symmetry about zero determines how critical values are read. Write $t_{\alpha}$ for the value that leaves probability $\alpha$ in the right tail, so that $P(T>t_{\alpha})=\alpha.$ Reflecting the curve across the vertical axis maps the right tail beyond $t_{\alpha}$ onto the left tail below $-t_{\alpha},$ and the two areas are equal. With this notation:

$$t_{1-\alpha} = -t_{\alpha}$$

26.The value with right-tail area $1-\alpha$ is the opposite of the value with right-tail area $\alpha.$ A single table entry therefore gives both tail boundaries. If $P(T>t_{\alpha})=\alpha,$ then $P(T<-t_{\alpha})=\alpha,$ and the two tails together have probability $2\alpha.$

27.

By symmetry, the shaded areas below $-t_{\alpha}$ and above $t_{\alpha}$ are equal.

28.A z-table usually reports cumulative probabilities for the standard normal distribution, while a t-table reports critical values for specified tail probabilities. Since the shape of the $t$ distribution depends on $k,$ each row of the t-table corresponds to a different number of degrees of freedom, and each column corresponds to a right-tail probability. The entry at their intersection is the critical value $t_{\alpha}$ for those two parameters.

29.

These tables give numerical probabilities or critical values without requiring a direct evaluation of the integral of the density.

Example 1

30.Suppose we want the value of the $t$ statistic with $k=12$ degrees of freedom that leaves an area of $0.01$ in the left tail. A left-tail area of $0.01$ corresponds to a right-tail area of $0.99,$ so the value we are looking for is $t_{0.99},$ and by symmetry it is the opposite of the value with right-tail area $0.01$:

$$t_{0.99} = -t_{0.01}$$

31.The table gives $t_{0.01}$ directly. Reading the row $k=12$ and the column of right-tail probability $0.01,$ the entry at the intersection is:

32.Substituting $t_{0.01}=2.681$ into the symmetry relation gives:

$$t_{0.99} = -2.681$$

33.With $12$ degrees of freedom, the value that leaves $1\%$ of the total probability to its left is $-2.681.$

Note on tail probabilities

34.For symmetric cutoffs $-a$ and $a,$ the probability contained in the two tails is twice the probability in one tail. In the example the right tail beyond $2.681$ has area $0.01,$ the left tail below $-2.681$ has the same area, and the probability outside the interval $[-2.681,2.681]$ is:

$$2 \times 0.01 = 0.02$$

35.The central area between the two critical values is the complement:

$$1-0.02 = 0.98$$

36.Applied to the statistic of the previous section, this pair of critical values determines a confidence interval. A sample of $13$ observations has $n-1=12$ degrees of freedom, and the inequality $-2.681 \le \left(\bar X-\mu\right)/\left(S/\sqrt{13}\right) \le 2.681$ holds with probability $0.98.$ Solving for $\mu$ gives the interval:

$$\bar X \pm 2.681\frac{S}{\sqrt{13}}$$

37.Under repeated sampling, this interval covers the unknown population mean in $98\%$ of the samples.