1.8: Screening Tools–How Effective Are they?
- Page ID
- 147911
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\( \newcommand{\dsum}{\displaystyle\sum\limits} \)
\( \newcommand{\dint}{\displaystyle\int\limits} \)
\( \newcommand{\dlim}{\displaystyle\lim\limits} \)
\( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)
( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\id}{\mathrm{id}}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\kernel}{\mathrm{null}\,}\)
\( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\)
\( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\)
\( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)
\( \newcommand{\vectorA}[1]{\vec{#1}} % arrow\)
\( \newcommand{\vectorAt}[1]{\vec{\text{#1}}} % arrow\)
\( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vectorC}[1]{\textbf{#1}} \)
\( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)
\( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)
\( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\(\newcommand{\longvect}{\overrightarrow}\)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\(\newcommand{\avec}{\mathbf a}\) \(\newcommand{\bvec}{\mathbf b}\) \(\newcommand{\cvec}{\mathbf c}\) \(\newcommand{\dvec}{\mathbf d}\) \(\newcommand{\dtil}{\widetilde{\mathbf d}}\) \(\newcommand{\evec}{\mathbf e}\) \(\newcommand{\fvec}{\mathbf f}\) \(\newcommand{\nvec}{\mathbf n}\) \(\newcommand{\pvec}{\mathbf p}\) \(\newcommand{\qvec}{\mathbf q}\) \(\newcommand{\svec}{\mathbf s}\) \(\newcommand{\tvec}{\mathbf t}\) \(\newcommand{\uvec}{\mathbf u}\) \(\newcommand{\vvec}{\mathbf v}\) \(\newcommand{\wvec}{\mathbf w}\) \(\newcommand{\xvec}{\mathbf x}\) \(\newcommand{\yvec}{\mathbf y}\) \(\newcommand{\zvec}{\mathbf z}\) \(\newcommand{\rvec}{\mathbf r}\) \(\newcommand{\mvec}{\mathbf m}\) \(\newcommand{\zerovec}{\mathbf 0}\) \(\newcommand{\onevec}{\mathbf 1}\) \(\newcommand{\real}{\mathbb R}\) \(\newcommand{\twovec}[2]{\left[\begin{array}{r}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\ctwovec}[2]{\left[\begin{array}{c}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\threevec}[3]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\cthreevec}[3]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\fourvec}[4]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\cfourvec}[4]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\fivevec}[5]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\cfivevec}[5]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\mattwo}[4]{\left[\begin{array}{rr}#1 \amp #2 \\ #3 \amp #4 \\ \end{array}\right]}\) \(\newcommand{\laspan}[1]{\text{Span}\{#1\}}\) \(\newcommand{\bcal}{\cal B}\) \(\newcommand{\ccal}{\cal C}\) \(\newcommand{\scal}{\cal S}\) \(\newcommand{\wcal}{\cal W}\) \(\newcommand{\ecal}{\cal E}\) \(\newcommand{\coords}[2]{\left\{#1\right\}_{#2}}\) \(\newcommand{\gray}[1]{\color{gray}{#1}}\) \(\newcommand{\lgray}[1]{\color{lightgray}{#1}}\) \(\newcommand{\rank}{\operatorname{rank}}\) \(\newcommand{\row}{\text{Row}}\) \(\newcommand{\col}{\text{Col}}\) \(\renewcommand{\row}{\text{Row}}\) \(\newcommand{\nul}{\text{Nul}}\) \(\newcommand{\var}{\text{Var}}\) \(\newcommand{\corr}{\text{corr}}\) \(\newcommand{\len}[1]{\left|#1\right|}\) \(\newcommand{\bbar}{\overline{\bvec}}\) \(\newcommand{\bhat}{\widehat{\bvec}}\) \(\newcommand{\bperp}{\bvec^\perp}\) \(\newcommand{\xhat}{\widehat{\xvec}}\) \(\newcommand{\vhat}{\widehat{\vvec}}\) \(\newcommand{\uhat}{\widehat{\uvec}}\) \(\newcommand{\what}{\widehat{\wvec}}\) \(\newcommand{\Sighat}{\widehat{\Sigma}}\) \(\newcommand{\lt}{<}\) \(\newcommand{\gt}{>}\) \(\newcommand{\amp}{&}\) \(\definecolor{fillinmathshade}{gray}{0.9}\)INTRODUCTION
No medical test is perfect. Some are very good at determining when a patient has a disease, and some are very good at determining when a patient does not have a disease.
For example, you may have seen advertisements for Cologuard, a test for colon cancer, which claims to have a “specificity of 87%.” But what does it mean for a test to have a specificity of 87%? You should be able to answer that question after completing this collaboration.
SPECIFIC OBJECTIVES
By the end of this collaboration, you should understand that
- screening tools for medical conditions or diseases are not 100% effective.
- there are at least two ways to quantify the effectiveness of screening tools.
- when a doctor decides to use a particular screening tool, the doctor has to decide whether it is more important to have a high specificity or a high sensitivity.
By the end of this collaboration, you should be able to
- interpret the meaning of a screening tool’s sensitivity and specificity.
PROBLEM SITUATION: PSA
Prostate cancer is cancer of the prostate gland, which surrounds the neck of the bladder in males. Prostate cancer is a common type of cancer in men. Doctors have developed an initial screening test for detecting prostate cancer which measures Prostate Specific Antigen (PSA). In this initial screening test, the doctor takes a sample of the man’s blood and measures the level of PSA. PSA is a substance which is only produced by the prostate gland. Men who have prostate cancer frequently have high levels of PSA in their blood.
However, PSA is also present in men who do not have prostate cancer. So, it is possible that a man with a high PSA level might not have prostate cancer. It is also possible that a man with a low PSA level might still have prostate cancer. This is why the PSA blood test is only used as the initial screening test. A more effective way to know if prostate cancer is present is to conduct a second screening test using a biopsy. In a biopsy, doctors cut small tissue samples from a man’s prostate gland. The samples can then be tested to see if cancer is present.
Why don’t all men start with the biopsy? Doctors try to avoid unnecessary biopsies because the procedure is invasive, uncomfortable, and expensive. Even though the PSA blood test is less effective than a biopsy, it is simpler to use as an initial screening tool for prostate cancer because the only thing required is a blood sample. Doctors use the results of the PSA blood test to help them decide whether to recommend a second screening (the biopsy). In this collaboration, we will be focusing on the initial screening tool: the PSA blood test.
The table below shows the risk of prostate cancer for men with certain PSA levels in their blood (measured in nanograms per milliliter, or ng/mL):28
|
PSA Level |
Risk of Prostate Cancer |
| < 0.5 | 7% |
| 0.6-1.0 | 10% |
| 1.1-2.0 | 17% |
| 2.1-3.0 | 24% |
| 3.1-4.0 | 27% |
(1) Review the table. Based on this information, do PSA levels seem to be associated with risk of prostate cancer? Record any patterns that you notice.
Imagine you are a doctor. You decide to use the PSA blood test to screen for prostate cancer in your patients. But, at what PSA level will you decide to do a biopsy on a patient? In other words, what is your “cut-off” value?
A PSA “cut-off” value is the PSA level at which you start recommending biopsies for your patients. That is, you will recommend a biopsy for any man whose PSA level is at or above the “cut-off” value. Any man whose PSA level is below the “cut-off” value will not be given a biopsy, because prostate cancer is less likely, though still possible.
For example, suppose the PSA cut-off value is set at 2.0. A man with a PSA level of 1.3 will be told that it is unlikely he has prostate cancer, and a biopsy will not be recommended. A man with a PSA level of 2.5 will be told that, because it is likely enough that he has prostate cancer, a biopsy is recommended. In both cases, it is uncertain whether the men have prostate cancer, but the higher PSA level of 2.5 indicates that it is more likely in that man.
(2) Suppose your goal in choosing a PSA cut-off value is to be certain that you detect almost all individuals with cancer. With that goal, should the cut-off value be high or low? Explain your reasoning.
(3) Suppose your goal is different now. Imagine that your goal in choosing a PSA cut-off value is to limit the number of unnecessary biopsies. Should the cut-off value be high or low? Explain your reasoning.
When determining a cut-off value, doctors may consider how effective the screening tool will be at different values. Doctors will most likely discuss the effectiveness with their patients when administering a screening tool. When discussing how effective a screening tool is, two technical terms are frequently used: sensitivity and specificity. These terms quantify probabilities or likelihoods. They are expressed as percentages.
The sensitivity is the likelihood that a test correctly produces a positive result for those who should test positive. For example, with the PSA blood test, sensitivity is the likelihood that the test will indicate the presence of cancer when cancer is actually present.
The specificity is the likelihood that a test correctly produces a negative result for those who should test negative. For example, with the PSA blood test, specificity is the likelihood that the test will indicate the absence of cancer when cancer is actually absent.
The terms sensitivity and specificity are not just used when screening for prostate cancer, but can be calculated for any test that screens for the presence or absence of a disease or medical condition.
Sometimes students have trouble remembering which of the two terms refers to patients with a disease, and which refers to patients without the disease. It may be helpful to use the mnemonic that doctors should be especially sensitive in how they interact with sick patients, those who actually have the disease.
The following table, from JAMA: The Journal of the American Medical Association29, shows the specificity and sensitivity associated with using three different possible cut-off values when screening for prostate cancer:
| PSA Level (ng/mL) | Sensitivity | Specificity |
| 1.1 | 83% | 39% |
| 2.6 | 40% | 81% |
| 4.1 | 20% | 94% |
(4) Suppose you choose 1.1 as the PSA cut-off value.
(a) Write a complete sentence that explains the sensitivity associated with this screening tool and cut-off value. Imagine that you are writing this to a friend who does not understand sensitivity.
(b) Write a complete sentence that would explain to a friend the specificity associated with this screening tool and cut-off value. Again, imagine your friend does not understand specificity.
(c) With this screening tool and cut-off value, what percentage of men without cancer would get unnecessary biopsies?
(d) With this screening tool and cut-off value, what percentage of men with cancer would not get necessary biopsies?
(5) Suppose you were to choose to use 4.1 as the cut-off value.
(a) Write a complete sentence that would explain to a friend the sensitivity associated with this screening tool.
(b) Write a complete sentence that would explain to a friend the specificity associated with this screening tool.
(c) With this screening tool and cut-off value, what percentage of men without cancer would get unnecessary biopsies?
(d) With this screening tool and cut-off value, what percentage of men with cancer would not get necessary biopsies?
MAKING CONNECTIONS
Record the important mathematical ideas from the discussion.
______________________________________
28 Ian M. Thompson, et al, “Prevalence of Prostate Cancer among Men with a Prostate-Specific Antigen Level ≤4.0 ng per Milliliter,” New England Journal of Medicine 350, no. 22(2004): 2239-46.
29 Ian M. Thompson et al, “Operating Characteristics of Prostate-Specific Antigen in Men with an Initial PSA Level of 3.0 ng/mL or Lower,” JAMA 294, no. 1(2005): 66-70.