9.1: The Central Limit Theorem for Sample Means
- Page ID
- 105850
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)
( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\id}{\mathrm{id}}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\kernel}{\mathrm{null}\,}\)
\( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\)
\( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\)
\( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)
\( \newcommand{\vectorA}[1]{\vec{#1}} % arrow\)
\( \newcommand{\vectorAt}[1]{\vec{\text{#1}}} % arrow\)
\( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vectorC}[1]{\textbf{#1}} \)
\( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)
\( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)
\( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\(\newcommand{\avec}{\mathbf a}\) \(\newcommand{\bvec}{\mathbf b}\) \(\newcommand{\cvec}{\mathbf c}\) \(\newcommand{\dvec}{\mathbf d}\) \(\newcommand{\dtil}{\widetilde{\mathbf d}}\) \(\newcommand{\evec}{\mathbf e}\) \(\newcommand{\fvec}{\mathbf f}\) \(\newcommand{\nvec}{\mathbf n}\) \(\newcommand{\pvec}{\mathbf p}\) \(\newcommand{\qvec}{\mathbf q}\) \(\newcommand{\svec}{\mathbf s}\) \(\newcommand{\tvec}{\mathbf t}\) \(\newcommand{\uvec}{\mathbf u}\) \(\newcommand{\vvec}{\mathbf v}\) \(\newcommand{\wvec}{\mathbf w}\) \(\newcommand{\xvec}{\mathbf x}\) \(\newcommand{\yvec}{\mathbf y}\) \(\newcommand{\zvec}{\mathbf z}\) \(\newcommand{\rvec}{\mathbf r}\) \(\newcommand{\mvec}{\mathbf m}\) \(\newcommand{\zerovec}{\mathbf 0}\) \(\newcommand{\onevec}{\mathbf 1}\) \(\newcommand{\real}{\mathbb R}\) \(\newcommand{\twovec}[2]{\left[\begin{array}{r}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\ctwovec}[2]{\left[\begin{array}{c}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\threevec}[3]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\cthreevec}[3]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\fourvec}[4]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\cfourvec}[4]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\fivevec}[5]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\cfivevec}[5]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\mattwo}[4]{\left[\begin{array}{rr}#1 \amp #2 \\ #3 \amp #4 \\ \end{array}\right]}\) \(\newcommand{\laspan}[1]{\text{Span}\{#1\}}\) \(\newcommand{\bcal}{\cal B}\) \(\newcommand{\ccal}{\cal C}\) \(\newcommand{\scal}{\cal S}\) \(\newcommand{\wcal}{\cal W}\) \(\newcommand{\ecal}{\cal E}\) \(\newcommand{\coords}[2]{\left\{#1\right\}_{#2}}\) \(\newcommand{\gray}[1]{\color{gray}{#1}}\) \(\newcommand{\lgray}[1]{\color{lightgray}{#1}}\) \(\newcommand{\rank}{\operatorname{rank}}\) \(\newcommand{\row}{\text{Row}}\) \(\newcommand{\col}{\text{Col}}\) \(\renewcommand{\row}{\text{Row}}\) \(\newcommand{\nul}{\text{Nul}}\) \(\newcommand{\var}{\text{Var}}\) \(\newcommand{\corr}{\text{corr}}\) \(\newcommand{\len}[1]{\left|#1\right|}\) \(\newcommand{\bbar}{\overline{\bvec}}\) \(\newcommand{\bhat}{\widehat{\bvec}}\) \(\newcommand{\bperp}{\bvec^\perp}\) \(\newcommand{\xhat}{\widehat{\xvec}}\) \(\newcommand{\vhat}{\widehat{\vvec}}\) \(\newcommand{\uhat}{\widehat{\uvec}}\) \(\newcommand{\what}{\widehat{\wvec}}\) \(\newcommand{\Sighat}{\widehat{\Sigma}}\) \(\newcommand{\lt}{<}\) \(\newcommand{\gt}{>}\) \(\newcommand{\amp}{&}\) \(\definecolor{fillinmathshade}{gray}{0.9}\)Section 0: Introduction to the Central Limit Theorem
Let’s recall the definitions of the parameter and statistic.
A parameter is a numerical summary of a population. A statistic is a numerical summary of a sample. The following table summarizes the differences between the two.
|
Statistic |
Parameter |
|---|---|
|
sample |
population |
|
size |
size |
|
varies |
constant |
|
can be found |
unknown |
A parameter is something that is computed from a population, while unknown it remains constant. A statistic is something that is computed from a sample of size n, while it may vary from sample to sample it can be easily found when needed.
The following table summarizes the most common statistics along with their corresponding parameters.
|
Statistic |
Parameter |
|---|---|
|
sample mean |
\(\mu\) |
|
sample proportion |
|
|
sample variance |
\(\sigma^2\) |
\(p\)One of the most important applications of a sample statistic is to approximate the unknown parameter of the population such as the mean, proportions, and variance.
To approximate the average age of ALL students on campus, the average height of the class can be used.
To approximate the proportion of males among ALL the students on campus, the proportion of males in the class can be used.
To approximate the variance of the students’ GPA in the entire college, the variance of the GPA of the students in the class.
Let’s measure a few sample statistics. We may assume that our class is our sample, and it should not be too hard to imagine that the average height of our class is 167.5 cm, the proportion of males is 56%, and the variance of GPA is 0.61. All these numbers approximate the corresponding parameters which are unknown.
Let’s take another sample. But before I write the results of the survey, let me ask you a question. Can you predict the outputs of the survey before the survey is performed? Here are the results! The mean is 169.3 cm, the proportion of males is 44% and the variance is 0.52. I am sorry, but I doubt that you guessed any of them correctly!
|
Sample 1 |
Sample 2 |
Population Parameter |
|
|---|---|---|---|
|
average height |
167.5 cm |
169.3 cm |
\(\mu\) |
|
proportion of males |
56% |
44% |
\(p\) |
|
variance of GPA |
0.61 |
0.52 |
\(\sigma^2\) |
It is not hard to realize that a random observation of any sample statistic is almost never the same as another observation from another sample! Nor is it equal to the population parameter which is in general unknown. This error is unavoidable and is called the sampling error. Later, we will learn that the sampling error can be minimized by increasing the sample size, but it cannot be eliminated. However, many times the only way to learn anything about the parameter is through observing a statistic.
Let’s introduce the notation. A single observation of a sample statistic will depend on the sample and therefore for an \(i\)-th sample, let’s label the average as \(\bar{x}_i\), the proportion as \(\hat{p}\), the variance as \(s^2_i\).
|
Sample 1 |
Sample 2 |
Sample |
Population Parameter |
|
|---|---|---|---|---|
|
average height |
\(\bar{x}_1=167.5\) |
\(\bar{x}_2=169.3\) |
\(\bar{x}_i\) |
\(\mu\) |
|
proportion of males |
\(\hat{p}_1=0.56\) |
\(\hat{p}_2=0.44\) |
\(\hat{p}_i\) |
\(p\) |
|
variance of GPA |
\(s^2_1=0.61\) |
\(s^2_2=0.52\) |
\(s^2_i\) |
\(\sigma^2\) |
What all these quantities have in common is the fact they are impossible to guess before the actual sample is drawn, i.e., they are unknown quantities whose values depend on chance! Doesn’t that sound familiar? Yes, that is the precise definition of a random variable!
Recall that a random variable is an unknown quantity whose value depends on chance. A random variable can be classified as discrete or continuous.
Thus, each of the quantities above and some others (such as the sample sum) can be thought of as continuous random variables!
Let’s introduce a random variable notation – the average of a sample is \(\bar{X}\), the proportion of a sample is \(\hat{P}\), and the variance of a sample is \(S^2\).
Why are they all continuous? Because the range of all possible values for a sample statistic is a continuous interval. For example:
- The average height of the students in a random sample can be any number between 140 and 200.
- The proportion of males in a random sample can be any number between 0 and 100%.
- The variance of a random sample can be any positive number.
|
Notation |
Sample |
Population Parameter |
|
|---|---|---|---|
|
Sample mean |
\(\bar{X}\) |
\(\bar{x}_i\) |
\(\mu\) |
|
Sample proportion |
\(\hat{P}\) or \(\hat{p}\) |
\(\hat{p}_i\) |
\(p\) |
|
Sample variance |
\(S^2\) or \(s^2\) |
\(s^2_i\) |
\(\sigma^2\) |
- \(\bar{x}_i\)
is a random observation of a random variable called “sample mean”, \(\bar{X}\). - \(\hat{p}_i\) is a random observation of a random variable called “sample proportion”, \(\hat{p}\).
- \(s^2_i\) is a random observation of a random variable called “sample variance”, \(s^2\).
Next step is to determine the distributions of each of these random variables – are they normal? Student-\(T\)? or maybe \(\chi^2\)? Once we determine the distribution for each of the variables, we would want to know the parameters – mean and standard deviation in case if the distribution is normal, and degrees of freedom if the distribution is T or \(\chi^2\) or \(F\).
In summary, what’s the Central Limit Theorem all about? It answers the exact questions that we asked earlier – what is the distribution of each sample statistics and its parameters? The Central Limit Theorem has three parts – one for each of the following random variables – the mean, the proportion, and the sum. The distribution of the sample variance can also be determined but it is not part of the Central Limit Theorem.
Section 1: The Central Limit Theorem for Means
For a relatively large sample size \(n\) (\(n\geq30\)) from a population \(X\) with parameters \(\mu_X\) and \(\sigma_X\), the sample mean, \(\bar{X}\), is approximately normally distributed with parameters \(\mu_{\bar{X}}=\mu_X\) and \(\sigma_{\bar{X}}=\dfrac{\sigma_X}{\sqrt{n}}\), regardless of the distribution of \(X\). For normally distributed \(X\), the sample doesn’t have to be large.
Symbolically, the statement can be written in the following way:
For \(X\sim?(\mu, \sigma)\) and \(n\geq30\) or for \(X\sim N(\mu, \sigma)\) and any \(n\)
\(\bar{X}\sim N\left(\mu_{\bar{X}}=\mu_X, \sigma_{\bar{X}}=\dfrac{\sigma}{\sqrt{n}}\right)\)
According to a study, the time between taking a pain reliever and getting relief for a randomly selected patient has unknown distribution with a mean of 46 minutes and a standard deviation of 5.9 minutes. Find the probability that the average time between taking a pain reliever and getting relief for a sample of 36 randomly selected patients is between 45 and 48 minutes.
Solution
Let \(X\) be the time between taking a pain reliever and getting relief for a randomly selected patient, then
\(X\sim ?(\mu_X=46, \sigma_X=5.9)\)
Let \(\bar{X}\) be the average time between taking a pain reliever and getting relief for a random sample of 36 patients, then, by CLT,
\(\bar{X}\sim N\left(\mu_{\bar{X}}=\mu_X=46, \sigma_{\bar{X}}=\dfrac{\sigma_X}{\sqrt{n}}=\dfrac{5.9}{\sqrt{36}}=0.9833\right)\)
Then the probability that the average time between taking a pain reliever and getting relief for a sample of 36 randomly selected patients is between 45 and 48 minutes can be expressed as \(P(45<\bar{X}<48\):
\(P(45<\bar{X}<48)=P\left(\dfrac{45-46}{0.9833}<\dfrac{\bar{X}-\mu_{\bar{X}}}{\sigma_{\bar{X}}}<\dfrac{48-46}{0.9833}\right)=P(-1.02<Z<2.03)=0.8250=82.50\%\)
Or using technology
\(P(45<\bar{X}<48)=0.8244=82.44\%\)