4.2: Finding the Probability
- Page ID
- 105828
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)
( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\id}{\mathrm{id}}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\kernel}{\mathrm{null}\,}\)
\( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\)
\( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\)
\( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)
\( \newcommand{\vectorA}[1]{\vec{#1}} % arrow\)
\( \newcommand{\vectorAt}[1]{\vec{\text{#1}}} % arrow\)
\( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vectorC}[1]{\textbf{#1}} \)
\( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)
\( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)
\( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\(\newcommand{\avec}{\mathbf a}\) \(\newcommand{\bvec}{\mathbf b}\) \(\newcommand{\cvec}{\mathbf c}\) \(\newcommand{\dvec}{\mathbf d}\) \(\newcommand{\dtil}{\widetilde{\mathbf d}}\) \(\newcommand{\evec}{\mathbf e}\) \(\newcommand{\fvec}{\mathbf f}\) \(\newcommand{\nvec}{\mathbf n}\) \(\newcommand{\pvec}{\mathbf p}\) \(\newcommand{\qvec}{\mathbf q}\) \(\newcommand{\svec}{\mathbf s}\) \(\newcommand{\tvec}{\mathbf t}\) \(\newcommand{\uvec}{\mathbf u}\) \(\newcommand{\vvec}{\mathbf v}\) \(\newcommand{\wvec}{\mathbf w}\) \(\newcommand{\xvec}{\mathbf x}\) \(\newcommand{\yvec}{\mathbf y}\) \(\newcommand{\zvec}{\mathbf z}\) \(\newcommand{\rvec}{\mathbf r}\) \(\newcommand{\mvec}{\mathbf m}\) \(\newcommand{\zerovec}{\mathbf 0}\) \(\newcommand{\onevec}{\mathbf 1}\) \(\newcommand{\real}{\mathbb R}\) \(\newcommand{\twovec}[2]{\left[\begin{array}{r}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\ctwovec}[2]{\left[\begin{array}{c}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\threevec}[3]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\cthreevec}[3]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\fourvec}[4]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\cfourvec}[4]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\fivevec}[5]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\cfivevec}[5]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\mattwo}[4]{\left[\begin{array}{rr}#1 \amp #2 \\ #3 \amp #4 \\ \end{array}\right]}\) \(\newcommand{\laspan}[1]{\text{Span}\{#1\}}\) \(\newcommand{\bcal}{\cal B}\) \(\newcommand{\ccal}{\cal C}\) \(\newcommand{\scal}{\cal S}\) \(\newcommand{\wcal}{\cal W}\) \(\newcommand{\ecal}{\cal E}\) \(\newcommand{\coords}[2]{\left\{#1\right\}_{#2}}\) \(\newcommand{\gray}[1]{\color{gray}{#1}}\) \(\newcommand{\lgray}[1]{\color{lightgray}{#1}}\) \(\newcommand{\rank}{\operatorname{rank}}\) \(\newcommand{\row}{\text{Row}}\) \(\newcommand{\col}{\text{Col}}\) \(\renewcommand{\row}{\text{Row}}\) \(\newcommand{\nul}{\text{Nul}}\) \(\newcommand{\var}{\text{Var}}\) \(\newcommand{\corr}{\text{corr}}\) \(\newcommand{\len}[1]{\left|#1\right|}\) \(\newcommand{\bbar}{\overline{\bvec}}\) \(\newcommand{\bhat}{\widehat{\bvec}}\) \(\newcommand{\bperp}{\bvec^\perp}\) \(\newcommand{\xhat}{\widehat{\xvec}}\) \(\newcommand{\vhat}{\widehat{\vvec}}\) \(\newcommand{\uhat}{\widehat{\uvec}}\) \(\newcommand{\what}{\widehat{\wvec}}\) \(\newcommand{\Sighat}{\widehat{\Sigma}}\) \(\newcommand{\lt}{<}\) \(\newcommand{\gt}{>}\) \(\newcommand{\amp}{&}\) \(\definecolor{fillinmathshade}{gray}{0.9}\)Section 1: Probability of an Event
We already know that the probability of an impossible event is zero, and the probability of a certain event is one. As a result, the probability of any other event must be between 0 and 1. The question that we will try to answer next is about how to find the probability of any event. Before we start answering this question, we need to understand that for every event there is an experiment for which there exists the sample space which consists of all possible simple outcomes using which we can define an event as a set of simple outcomes.
Consider the following two examples: tossing a coin and driving to work, both experiments have two simple outcomes – tossing a coin may result in heads or tails and driving to work may result in an accident or not. But there is one conceptual difference between these two experiments. To see the difference let us answer the following questions: Is there a reason to believe that the heads are more or less likely than tails? And is there a reason to believe that to get in an accident is as likely as not to get in an accident? Turns out that there is no reason to believe that when tossing a fair coin, the chances of getting heads is any different from getting tails. Such experiment is said to have equally likely outcomes. On the other hand, there is no reason to believe that the chances of getting in an accident are the same as chances of not getting in an accident. Such experiment is said to have not equally likely outcomes.
Consider another two examples: rolling a die and counting credit cards in the wallet of a randomly selected person, both experiments have 6 simple outcomes – rolling a die may result in any number between 1 and 6 and the number of credit cards in the wallet of a randomly selected person can vary from zero to any number. Again, there is one conceptual difference between these two experiments. To see the difference let us answer the following questions: Is there a reason to believe that 1 or 3 are more or less likely than 4 or 6? And is there a reason to believe that it is as likely to have 0 cards as more than 5? It is obvious, that there is no reason to believe that when rolling a fair die the chances of getting any number are different from any other number. On the other hand, there is no reason to believe that the chances of having no credit cards are the same as chances of having more than 5.
Most classical experiments have equally likely simple outcomes but in real life most of the experiments do not have equally likely outcomes. So, it is important to be able to work with both types of experiments. When working with an experiment that has equally likely simple outcomes, we will use an approach called classical, and when working with an experiment that has not equally likely simple outcomes, we will use an approach called empirical.
|
Equally Likely Outcomes |
Not Equally Likely Outcomes |
|---|---|
|
|
Section 2: The Classical Approach
Next, we will discuss the classical approach which is a method of finding the probability of an event defined for an experiment with equally likely simple outcomes. We already understand that for every event there is an experiment for which there exists the sample space which consists of all possible simple outcomes. In this section, we consider only those experiments for which all simple outcomes are equally likely. In such a case, to find the probability we use the following formula:
\(P(E)=\frac{f}{N}\)
where \(E\) is an event, \(f\) is the number of ways that event \(E\) may occur or the size of the set that defines the event, and \(N\) is the total number of outcomes which is the size of the sample space.
So, to find the probability of an event we can just count the number of outcomes in the set definition and divide it by the size of the sample space.
Find the probabilities of the following events when a fair coin is flipped three times.
- There are two Hs
- There are no Ts
- There is an odd number of Ts
- There is at least one H
Solution
To use the formula, we first need to identify the experiment and its sample space. The experiment is tossing a coin three times which has a well-known sample space of size 8:
|
Experiment |
Sample space |
Size |
|---|---|---|
|
flipping a coin thrice |
{TTT, TTH, THT, HTT, THH, HTH, HHT, HHH} |
\(N=8\) |
For every event we will find the set description of an event that is a list all simple outcomes that define it, figure out the size and divided it by 8 - the size of the sample space.
For example, for event \(A\) there are 3 outcomes: THH, HTH, HHT, therefore the probability of event \(A\) is
\(P(A)=\frac{3}{8}=0.375\)
Similarly, for event \(B\) there is only one outcome HHH, therefore the probability of event \(B\) is
\(P(B)=\frac{1}{8}=0.125\)
Similarly, for event \(C\) there are 4 outcomes THH, HTH, HHT, TTT, therefore the probability of event \(C\) is
\(P(C)=\frac{4}{8}=0.500\)
Finally, for event \(D\) there are 7 outcomes, therefore the probability of event \(D\) is
\(P(D)=\frac{7}{8}=0.875\)
As one can see by the definition of the method it is impossible to get a number less than zero or greater than one. This result validates our choice of denoting the probability of impossible events as zero and probabilities of certain events as one.
We discussed the classical approach which allows to find the probability of an event defined for an experiment with equally likely simple outcomes.
Section 3: The Empirical Approach
As we previously said, in real life most of the experiments do not have equally likely outcomes, so what do we do? Next, we will discuss the empirical approach which is a method of finding the probability of an event defined for an experiment with not equally likely simple outcomes. We already understand that for every event there is an experiment for which there exists the sample space which consists of all possible simple outcomes.
Let our experiment consist of throwing a cone in the air and recording how it is landed.
A cone is a simple three-dimensional object, that can only land in two positions: upright (U) or sideways (S). The sample space of the experiment has two simple outcomes U and S. It is pretty clear that these outcomes are not equally likely, so the classical approach wouldn’t apply here. We have a strong intuition that the probability of the cone landing on its side is more than the probability of it landing upright. The goal is to find the probabilities for each of the outcomes!
One way to compute the probability of each outcome is to look at the past experience of tossing a cone. For instance, in this case, last time I flipped a cone 100 times, I had the following frequency distribution table:
|
Outcome |
Frequency |
|---|---|
|
S |
94 |
|
U |
6 |
None of these numbers look like they can represent the probability of an event but if we construct the relative frequency table then we will start seeing that the relative frequencies look a lot like probabilities? Aren’t they?
|
Outcome |
Frequency |
Relative Frequency |
Probability |
|---|---|---|---|
|
S |
94 |
0.94 |
P(S) |
|
U |
6 |
0.06 |
P(U) |
|
Total |
100 |
1 |
|
In summary, the Empirical approach consists of performing the experiment many times and constructing the relative frequency distribution table and then interpreting the relative frequencies as the approximations of the probabilities of the corresponding events. In our example, 0.94 is approximately the probability that a cone will land on its side, and 0.06 is approximately the probability that a cone will land upright. The empirical approach is also called the relative frequency approach.
To explain why the relative frequency is only an approximation of the true probability, let’s consider tossing a coin which also may result in only two outcomes. This time the probability of each outcome is known, both are 
. Last time I flipped a fair coin 100 times I observed the following frequencies and relative frequencies:
|
Outcome |
Frequency |
Relative Frequency |
Probability |
|---|---|---|---|
|
H |
47 |
0.47 |
0.50 |
|
T |
53 |
0.53 |
0.50 |
|
Total |
100 |
1 |
|
Since we know that the true probability of a coin showing heads is 0.5, we conclude that the relative frequency 0.47 is only an approximation to the true value. Now if we toss a coin 1000 times, we would see the following relative frequency distribution table in which the relative frequencies are much closer to the true values of the probabilities:
|
Outcome |
Frequency |
Relative Frequency |
Probability |
|---|---|---|---|
|
H |
487 |
0.487 |
0.50 |
|
T |
513 |
0.513 |
0.50 |
|
Total |
1000 |
1 |
|
This phenomenon is known as The Law of Large Numbers states that the larger the number of experiments the closer the relative frequencies approximate to the true value.
Estimate the probability that a randomly selected student owns 2 cars.
Solution
To answer this question using the empirical approach I would have to check on my past experience with the students owning cars. Last time I asked students how many cars they own I received the following frequencies and relative frequencies.
|
Number of cars |
Frequency |
Relative Frequency |
|---|---|---|
|
0 |
6 |
.24 |
|
1 |
16 |
.64 |
|
2 |
2 |
.08 |
|
3 |
0 |
.00 |
|
4+ |
1 |
.04 |
|
Total |
25 |
1 |
Based on the Empirical approach, we may conclude that the probability that a randomly selected student owns two cars is approximately 8%.
Note that the relative frequencies are only approximations. Knowing that we would not say that the probability of a student owning 3 cars is zero, we would say it is a very small probability. Also, it is worth mentioning that the more representative the sample is of the larger population the closer these approximations are for the entire population. For example, if we were to assume that students are representative of the entire population, we can make a statement that the probability of a random person owning two cars is approximately 8%.
So how does one interpret the probability of an event? Let us say the probability of raining tomorrow is 0.2. What does it mean? Or how was such number produced?
To interpret we must understand where the number came from. If the empirical approach was used then in the past, it rained 20 out of 100 days that were just like tomorrow, so the relative frequency and the probability are both 0.2. We interpret this as, in the future, it will also rain 20 out of 100 days or 2 out of 10 or 1 out of 5 on average. The problem with such interpretation is the implied assumption that the experiment will continue behaving the same way as in the past, which may not be necessarily the case.
Note that in general we may not know the true probability of an event, which is the case with many experiments whose outcomes are not equally likely, but we just discussed the empirical approach which is a great way to approximate it.
Section 4: Probability and Odds
Historically the probability was developed to provide the support for gambling. And historically an alternative way was used to measure the likelihood of an outcome called odds. The odds of an event \(E\) is the ratio of the number of ways the event may occur to the number of ways that the event may not occur:
\(o(E)=|E|:|\text{no }E|\)
For an experiment with a known sample space and event \(E\) is defined by the list of \(x\) outcomes from the sample space then the remaining \(y\) outcomes are not used in the definition of \(E\). We say the odds in favor of \(E\) are “\(x\) to \(y\)", and we write \(x:y\) and the odds against \(E\) are “\(y\) to \(x\)", and we write \(y:x\).
Find the odds that there are exactly two heads out of two tosses of a fair coin.
Solution
In our experiment of tossing a coin two times the sample space is the following:
{HH, HT, TH, TT}
Let \(E\) be an event in which we have exactly two heads out of two tosses of a fair coin. Only one outcome makes up the event \(E\) and the remaining three are not \(E\). We say the odds in favor of \(E\) are 1:3, and against \(E\) are 3:1.
Often the odds are reported as a simplified ratio, that is the ratio in which both sides are reduced by the common factor. The odds in favor of getting an odd number when rolling a die can be expresses as 1:1 instead of 3:3 and the odds in favor of rolling less than 3 when rolling a die can be expressed as 1:2 instead of 2:4.
When there is more than one way to express the same concept, one must know how to convert from one way to another.
If the odds in favor of an event \(E\) are \(x:y\), then the probability of \(E\) is \(\frac{x}{x+y}\) and this table summarizes the conversion rule:
|
\(o(E)\) |
\(P(E)\) |
|---|---|
|
\(x:y\) |
\(\frac{x}{x+y}\) |
If \(o(E)=3:1\), then \(P(E)=\frac{3}{3+1}=0.75=75%\).
If the probability of an event \(E\) is \(p\%\) then the odds in favor of \(E\) are \(p:(100-p)\) and this table summarizes the conversion rule:
|
\(P(E)\) |
\(o(E)\) |
|---|---|
|
\(p\%\) |
\(p:(100-p)\) |
If \(P(E)=20\%\), then \(o(E)=20:(100-20)=20:80=1:4\).
Alternatively, if the probability of an event \(E\) is given by a fraction \(\frac{a}{n}\) then the odds in favor of \(E\) are \(a:(n-a)\) and this table summarizes the conversion rule:
|
\(P(E)\) |
\(o(E)\) |
|---|---|
|
\(\frac{a}{n}\) |
\(a:(n-a)\) |
If \(P(E)=\frac{4}{7}\), then \(o(E)=4:(7-4)=4:3\).
Although it is pretty straightforward how to use odds, the only reason we covered it is to pay respect to the historical relation of probability to gambling.