4.1: Fitting a Straight Line

Last updated
Save as PDF

Page ID: 96051

\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \) \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)\(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\) \(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\)\(\newcommand{\AA}{\unicode[.8,0]{x212B}}\)

Suppose the fitting curve is a line. We write for the fitting curve

\[y(x)=\alpha x+\beta . \nonumber \]

The distance \(r_{i}\) from the data point \(\left(x_{i}, y_{i}\right)\) and the fitting curve is given by

\[\begin{aligned} r_{i} &=y_{i}-y\left(x_{i}\right) \\ &=y_{i}-\left(\alpha x_{i}+\beta\right) . \end{aligned} \nonumber \]

A least-squares fit minimizes the sum of the squares of the \(r_{i}\) ’s. This minimum can be shown to result in the most probable values of \(\alpha\) and \(\beta\).

We define

\[\begin{aligned} \rho &=\sum_{i=1}^{n} r_{i}^{2} \\ &=\sum_{i=1}^{n}\left(y_{i}-\left(\alpha x_{i}+\beta\right)\right)^{2} \end{aligned} \nonumber \]

To minimize \(\rho\) with respect to \(\alpha\) and \(\beta\), we solve

\[\frac{\partial \rho}{\partial \alpha}=0, \quad \frac{\partial \rho}{\partial \beta}=0 \nonumber \]

Taking the partial derivatives, we have

\[\begin{aligned} &\frac{\partial \rho}{\partial \alpha}=\sum_{i=1}^{n} 2\left(-x_{i}\right)\left(y_{i}-\left(\alpha x_{i}+\beta\right)\right)=0 \\ &\frac{\partial \rho}{\partial \beta}=\sum_{i=1}^{n} 2(-1)\left(y_{i}-\left(\alpha x_{i}+\beta\right)\right)=0 \end{aligned} \nonumber \]

These equations form a system of two linear equations in the two unknowns \(\alpha\) and \(\beta\), which is evident when rewritten in the form

\[\begin{aligned} \alpha \sum_{i=1}^{n} x_{i}^{2}+\beta \sum_{i=1}^{n} x_{i} &=\sum_{i=1}^{n} x_{i} y_{i} \\ \alpha \sum_{i=1}^{n} x_{i}+\beta n &=\sum_{i=1}^{n} y_{i} \end{aligned} \nonumber \]

These equations can be solved either analytically, or numerically in MATLAB, where the matrix form is

\[\left(\begin{array}{ccc} \sum_{i=1}^{n} & x_{i}^{2} & \sum_{i=1}^{n} \\ \sum_{i=1}^{n} & x_{i} & n \end{array}\right)\left(\begin{array}{c} \alpha \\ \beta \end{array}\right)=\left(\begin{array}{c} \sum_{i=1}^{n} x_{i} y_{i} \\ \sum_{i=1}^{n} y_{i} \end{array}\right) . \nonumber \]

A proper statistical treatment of this problem should also consider an estimate of the errors in \(\alpha\) and \(\beta\) as well as an estimate of the goodness-of-fit of the data to the model. We leave these further considerations to a statistics class.