8.9: An Application to Quadratic Forms

An Application to Quadratic Forms

An expression like \(x_{1}^2 + x_{2}^2 + x_{3}^2 - 2x_{1}x_{3} + x_{2}x_{3}\) is called a quadratic form in the variables \(x_{1}\), \(x_{2}\), and \(x_{3}\). In this section we show that new variables \(y_{1}\), \(y_{2}\), and \(y_{3}\) can always be found so that the quadratic form, when expressed in terms of the new variables, has no cross terms \(y_{1}y_{2}\), \(y_{1}y_{3}\), or \(y_{2}y_{3}\). Moreover, we do this for forms involving any finite number of variables using orthogonal diagonalization. This has far-reaching applications; quadratic forms arise in such diverse areas as statistics, physics, the theory of functions of several variables, number theory, and geometry.

Quadratic Form026966 A quadratic form \(q\) in the \(n\) variables \(x_1, x_{2}, \dots, x_{n}\) is a linear combination of terms \(x_{1}^2, x_{2}^2, \dots, x_{n}^2\), and cross terms \(x_{1}x_{2}, x_{1}x_{3}, x_{2}x_{3}, \dots\).

If \(n = 3\), \(q\) has the form

\[q = a_{11}x_{1}^2 + a_{22}x_{2}^2 + a_{33}x_{3}^2 + a_{12}x_{1}x_{2} + a_{21}x_2x_1+ a_{13}x_{1}x_{3} + a_{31}x_{3}x_{1} + a_{23}x_{2}x_{3} + a_{32}x_{3}x_{2} \nonumber \]

In general

\[q = a_{11}x_{1}^2 + a_{22}x_{2}^2 + \cdots + a_{nn}x_{n}^2 + a_{12}x_{1}x_{2} + a_{13}x_{1}x_{3} + \cdots \nonumber \]

This sum can be written compactly as a matrix product

\[q = q(\mathbf{x}) = \mathbf{x}^TA\mathbf{x} \nonumber \]

where \(\mathbf{x} = (x_{1}, x_{2}, \dots, x_{n})\) is thought of as a column, and \(A = \left[ a_{ij} \right]\) is a real \(n \times n\) matrix. Note that if \(i \neq j\), two separate terms \(a_{ij}x_{i}x_{j}\) and \(a_{ji}x_{j}x_{i}\) are listed, each of which involves \(x_{i}x_{j}\), and they can (rather cleverly) be replaced by

\[\frac{1}{2}(a_{ij} + a_{ji})x_{i}x_{j} \quad \mbox{ and } \quad \frac{1}{2}(a_{ij} + a_{ji})x_{j}x_{i} \nonumber \]

respectively, without altering the quadratic form. Hence there is no loss of generality in assuming that \(x_{i}x_{j}\) and \(x_{j}x_{i}\) have the same coefficient in the sum for \(q\). In other words, we may assume that \(A\) is symmetric.

027002 Write \(q = x_1^2 + 3x_3^2+2x_1x_2 - x_1x_3\) in the form \(q(\mathbf{x}) = \mathbf{x}^{T}A\mathbf{x}\), where \(A\) is a symmetric \(3 \times 3\) matrix.

The cross terms are \(2x_{1}x_{2} = x_{1}x_{2} + x_{2}x_{1}\) and \(-x_{1}x_{3} = -\frac{1}{2}x_{1}x_{3} - \frac{1}{2}x_{3}x_{1}\).

Of course, \(x_{2}x_{3}\) and \(x_{3}x_{2}\) both have coefficient zero, as does \(x^{2}_{2}\). Hence

\[q(\mathbf{x}) = \left[ \begin{array}{rrr} x_{1} & x_{2} & x_{3} \end{array}\right] \left[ \begin{array}{rrr} 1 & 1 & -\frac{1}{2} \\ 1 & 0 & 0 \\ -\frac{1}{2} & 0 & 3 \end{array}\right] \left[ \begin{array}{c} x_{1} \\ x_{2} \\ x_{3} \end{array}\right] \nonumber \]

is the required form (verify).

We shall assume from now on that all quadratic forms are given by

\[q(\mathbf{x}) = \mathbf{x}^TA\mathbf{x} \nonumber \]

where \(A\) is symmetric. Given such a form, the problem is to find new variables \(y_{1}, y_{2}, \dots, y_{n}\), related to \(x_{1}, x_{2}, \dots, x_{n}\), with the property that when \(q\) is expressed in terms of \(y_{1}, y_{2}, \dots, y_{n}\), there are no cross terms. If we write

\[\mathbf{y} = (y_{1}, y_{2}, \dots, y_{n})^T \nonumber \]

this amounts to asking that \(q = \mathbf{y}^{T}D\mathbf{y}\) where \(D\) is diagonal. It turns out that this can always be accomplished and, not surprisingly, that \(D\) is the matrix obtained when the symmetric matrix \(A\) is orthogonally diagonalized. In fact, as Theorem [thm:024303] shows, a matrix \(P\) can be found that is orthogonal (that is, \(P^{-1} = P^{T}\)) and diagonalizes \(A\):

\[P^TAP = D = \left[ \begin{array}{cccc} \lambda_{1} & 0 & \cdots & 0 \\ 0 & \lambda_{2} & \cdots & 0 \\ \vdots & \vdots && \vdots \\ 0 & 0 & \cdots & \lambda_{n} \end{array}\right] \nonumber \]

The diagonal entries \(\lambda_{1}, \lambda_{2}, \dots, \lambda_{n}\) are the (not necessarily distinct) eigenvalues of \(A\), repeated according to their multiplicities in \(c_{A}(x)\), and the columns of \(P\) are corresponding (orthonormal) eigenvectors of \(A\). As \(A\) is symmetric, the \(\lambda_{i}\) are real by Theorem [thm:016397].

Now define new variables \(\mathbf{y}\) by the equations

\[\mathbf{x} = P\mathbf{y} \quad \mbox{ equivalently } \quad \mathbf{y} = P^T\mathbf{x} \nonumber \]

Then substitution in \(q(\mathbf{x}) = \mathbf{x}^{T}A\mathbf{x}\) gives

\[q = (P\mathbf{y})^TA(P\mathbf{y}) = \mathbf{y}^T(P^TAP)\mathbf{y} = \mathbf{y}^TD\mathbf{y} = \lambda_{1}y_{1}^2 + \lambda_{2}y_{2}^2 + \cdots + \lambda_{n}y_{n}^2 \nonumber \]

Hence this change of variables produces the desired simplification in \(q\).

Diagonalization Theorem027060 Let \(q = \mathbf{x}^{T}A\mathbf{x}\) be a quadratic form in the variables \(x_{1}, x_{2}, \dots, x_{n}\), where \(\mathbf{x} = (x_{1}, x_{2}, \dots, x_{n})^{T}\) and \(A\) is a symmetric \(n \times n\) matrix. Let \(P\) be an orthogonal matrix such that \(P^{T}AP\) is diagonal, and define new variables \(\mathbf{y} = (y_{1}, y_{2}, \dots, y_{n})^{T}\) by

\[\mathbf{x} = P\mathbf{y} \quad \mbox{ equivalently } \quad \mathbf{y} = P^T\mathbf{x} \nonumber \]

If \(q\) is expressed in terms of these new variables \(y_{1}, y_{2}, \dots, y_{n}\), the result is

\[q = \lambda_{1}y_{1}^2 + \lambda_{2}y_{2}^2 + \cdots + \lambda_{n}y_{n}^2 \nonumber \]

where \(\lambda_{1}, \lambda_{2}, \dots, \lambda_{n}\) are the eigenvalues of \(A\) repeated according to their multiplicities.

Let \(q = \mathbf{x}^{T}A\mathbf{x}\) be a quadratic form where \(A\) is a symmetric matrix and let \(\lambda_{1}, \dots, \lambda_{n}\) be the (real) eigenvalues of \(A\) repeated according to their multiplicities. A corresponding set \(\{\mathbf{f}_{1}, \dots, \mathbf{f}_{n}\}\) of orthonormal eigenvectors for \(A\) is called a set of principal axes for the quadratic form \(q\). (The reason for the name will become clear later.) The orthogonal matrix \(P\) in Theorem [thm:027060] is given as \(P = \left[ \begin{array}{ccc} \mathbf{f}_{1} & \cdots & \mathbf{f}_{n} \end{array} \right]\), so the variables \(X\) and \(Y\) are related by

\[\mathbf{x} = P\mathbf{y} = \left[ \begin{array}{cccc} \mathbf{f}_{1} & \mathbf{f}_{2}& \cdots & \mathbf{f}_{n} \end{array}\right] \left[ \begin{array}{c} y_{1} \\ y_{2} \\ \vdots \\ y_{n} \end{array}\right] = y_{1}\mathbf{f}_{1} + y_{2}\mathbf{f}_{2} + \cdots + y_{n}\mathbf{f}_{n} \nonumber \]

Thus the new variables \(y_{i}\) are the coefficients when \(\mathbf{x}\) is expanded in terms of the orthonormal basis \(\{\mathbf{f}_{1}, \dots, \mathbf{f}_{n}\}\) of \(\mathbb{R}^n\). In particular, the coefficients \(y_{i}\) are given by \(y_{i} = \mathbf{x} \cdot \mathbf{f}_{i}\) by the expansion theorem (Theorem [thm:015082]). Hence \(q\) itself is easily computed from the eigenvalues \(\lambda_{i}\) and the principal axes \(\mathbf{f}_{i}\):

\[q = q(\mathbf{x}) = \lambda_{1}(\mathbf{x}\bullet \mathbf{f}_{1})^2 + \cdots + \lambda_{n}(\mathbf{x}\bullet \mathbf{f}_{n})^2 \nonumber \]

027107 Find new variables \(y_{1}\), \(y_{2}\), \(y_{3}\), and \(y_{4}\) such that

\[q = 3(x_{1}^2 + x_{2}^2 + x_{3}^2 +x_{4}^2) + 2x_{1}x_{2} - 10x_{1}x_{3} + 10x_{1}x_{4} + 10x_{2}x_{3} - 10x_{2}x_{4} + 2x_{3}x_{4} \nonumber \]

has diagonal form, and find the corresponding principal axes.

The form can be written as \(q = \mathbf{x}^{T}A\mathbf{x}\), where

\[\mathbf{x} = \left[ \begin{array}{c} x_{1} \\ x_{2} \\ x_{3} \\ x_{4} \end{array}\right] \quad \mbox{ and } \quad A = \left[ \begin{array}{rrrr} 3 & 1 & -5 & 5 \\ 1 & 3 & 5 & -5 \\ -5 & 5 & 3 & 1 \\ 5 & -5 & 1 & 3 \end{array}\right] \nonumber \]

A routine calculation yields

\[c_{A}(x) = \det (xI - A) = (x - 12)(x + 8)(x - 4)^2 \nonumber \]

so the eigenvalues are \(\lambda_{1} = 12\), \(\lambda_{2} = -8\), and \(\lambda_{3} = \lambda_{4} = 4\). Corresponding orthonormal eigenvectors are the principal axes:

\[\mathbf{f}_{1} = \frac{1}{2} \left[ \begin{array}{r} 1 \\ -1 \\ -1 \\ 1 \end{array}\right] \quad \mathbf{f}_{2} = \frac{1}{2} \left[ \begin{array}{r} 1 \\ -1 \\ 1 \\ -1 \end{array}\right] \quad \mathbf{f}_{3} = \frac{1}{2} \left[ \begin{array}{r} 1 \\ 1 \\ 1 \\ 1 \end{array}\right] \quad \mathbf{f}_{4} = \frac{1}{2} \left[ \begin{array}{r} 1 \\ 1 \\ -1 \\ -1 \end{array}\right] \nonumber \]

The matrix

\[P = \left[ \begin{array}{cccc} \mathbf{f}_{1} & \mathbf{f}_{2} & \mathbf{f}_{3} & \mathbf{f}_{4} \end{array}\right] = \frac{1}{2} \left[ \begin{array}{rrrr} 1 & 1 & 1 & 1 \\ -1 & -1 & 1 & 1 \\ -1 & 1 & 1 & -1 \\ 1 & -1 & 1 & -1 \end{array}\right] \nonumber \]

is thus orthogonal, and \(P^{-1}AP = P^{T}AP\) is diagonal. Hence the new variables \(\mathbf{y}\) and the old variables \(\mathbf{x}\) are related by \(\mathbf{y} = P^{T}\mathbf{x}\) and \(\mathbf{x} = P\mathbf{y}\). Explicitly,

\[\begin{aligned} y_{1} = \frac{1}{2}(x_{1} - x_{2} - x_{3} + x_{4}) & &x_{1} &= \frac{1}{2}(y_{1} + y_{2} + y_{3} + y_{4}) \\ y_{2} = \frac{1}{2}(x_{1} - x_{2} + x_{3} - x_{4}) & &x_{2} &= \frac{1}{2}(-y_{1} - y_{2} + y_{3} + y_{4}) \\ y_{3} = \frac{1}{2}(x_{1} + x_{2} + x_{3} + x_{4}) & &x_{3} &= \frac{1}{2}(-y_{1} + y_{2} + y_{3} - y_{4}) \\ y_{4} = \frac{1}{2}(x_{1} + x_{2} - x_{3} - x_{4}) & &x_{4} &= \frac{1}{2}(y_{1} - y_{2} + y_{3} - y_{4}) \end{aligned} \nonumber \]

If these \(x_{i}\) are substituted in the original expression for \(q\), the result is

\[q = 12y_{1}^2 - 8y_{2}^2 + 4y_{3}^2 + 4y_{4}^2 \nonumber \]

This is the required diagonal form.

It is instructive to look at the case of quadratic forms in two variables \(x_{1}\) and \(x_{2}\). Then the principal axes can always be found by rotating the \(x_{1}\) and \(x_{2}\) axes counterclockwise about the origin through an angle \(\theta\). This rotation is a linear transformation \(R_{\theta} : \mathbb{R}^2 \to \mathbb{R}^2\), and it is shown in Theorem [thm:006021] that \(R_{\theta}\) has matrix \(P = \left[ \begin{array}{rr} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{array}\right]\). If \(\{\mathbf{e}_{1}, \mathbf{e}_{2}\}\) denotes the standard basis of \(\mathbb{R}^2\), the rotation produces a new basis \(\{\mathbf{f}_{1}, \mathbf{f}_{2}\}\) given by

\[\label{rotationEq} \mathbf{f}_{1} = R_{\theta}(\mathbf{e}_{1}) = \left[ \begin{array}{r} \cos\theta \\ \sin\theta \end{array}\right] \quad \mbox{ and } \quad \mathbf{f}_{2} = R_{\theta}(\mathbf{e}_{2}) = \left[ \begin{array}{r} -\sin\theta \\ \cos\theta \end{array}\right] \]

Given a point \(\mathbf{p} = \left[ \begin{array}{c} x_{1} \\ x_{2} \end{array}\right] = x_{1}\mathbf{e}_{1} + x_{2}\mathbf{e}_{2}\) in the original system, let \(y_{1}\) and \(y_{2}\) be the coordinates of \(\mathbf{p}\) in the new system (see the diagram). That is,

\[\label{rotationEq2} \left[ \begin{array}{c} x_{1} \\ x_{2} \end{array}\right] = \mathbf{p} = y_{1}\mathbf{f}_{1} + y_{2}\mathbf{f}_{2} = \left[ \begin{array}{rr} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{array}\right] \left[ \begin{array}{c} y_{1} \\ y_{2} \end{array}\right] \]

Writing \(\mathbf{x} = \left[ \begin{array}{c} x_{1} \\ x_{2} \end{array}\right]\) and \(\mathbf{y} = \left[ \begin{array}{c} y_{1} \\ y_{2} \end{array}\right]\), this reads \(\mathbf{x} = P\mathbf{y}\) so, since \(P\) is orthogonal, this is the change of variables formula for the rotation as in Theorem [thm:027060].

If \(r \neq 0 \neq s\), the graph of the equation \(rx_{1}^2 + sx_{2}^2 = 1\) is called an ellipse if \(rs > 0\) and a hyperbola if \(rs < 0\). More generally, given a quadratic form

\[q = ax_{1}^2 + bx_{1}x_{2} + cx_{2}^2 \quad \mbox{ where not all of } a, b, \mbox{ and } c \mbox{ are zero} \nonumber \]

the graph of the equation \(q = 1\) is called a conic. We can now completely describe this graph. There are two special cases which we leave to the reader.

1. If exactly one of \(a\) and \(c\) is zero, then the graph of \(q = 1\) is a parabola.

So we assume that \(a \neq 0\) and \(c \neq 0\). In this case, the description depends on the quantity \(b^{2} - 4ac\), called the discriminant of the quadratic form \(q\).

2. If \(b^{2} - 4ac = 0\), then either both \(a \geq 0\) and \(c \geq 0\), or both \(a \leq 0\) and \(c \leq 0\).
   Hence \(q = (\sqrt{a}x_{1} + \sqrt{c}x_{2})^2\) or \(q = (\sqrt{-a}x_{1} + \sqrt{-c}x_{2})^2\), so the graph of \(q = 1\) is a pair of straight lines in either case.

So we also assume that \(b^{2} - 4ac \neq 0\). But then the next theorem asserts that there exists a rotation of the plane about the origin which transforms the equation \(ax_{1}^2 + bx_{1}x_{2} + cx_{2}^2 = 1\) into either an ellipse or a hyperbola, and the theorem also provides a simple way to decide which conic it is.

027179 Consider the quadratic form \(q = ax_{1}^2 + bx_{1}x_{2} + cx_{2}^2\) where \(a\), \(c\), and \(b^{2} - 4ac\) are all nonzero.

1. There is a counterclockwise rotation of the coordinate axes about the origin such that, in the new coordinate system, \(q\) has no cross term.

2. is an ellipse if \(b^{2} - 4ac < 0\) and an hyperbola if \(b^{2} - 4ac > 0\).

If \(b = 0\), \(q\) already has no cross term and (1) and (2) are clear. So assume \(b \neq 0\). The matrix \(A = \left[ \begin{array}{cc} a & \frac{1}{2}b \\ \frac{1}{2}b & c \end{array}\right]\) of \(q\) has characteristic polynomial \(c_A(x) = x^2 - (a + c)x - \frac{1}{4}(b^2 - 4ac)\). If we write \(d = \sqrt{b^2 + (a - c)^2}\) for convenience; then the quadratic formula gives the eigenvalues

\[\lambda_{1} = \frac{1}{2}[a + c - d] \quad \mbox{ and } \quad \lambda_{2} = \frac{1}{2}[a + c + d] \nonumber \]

with corresponding principal axes

\[\begin{aligned} \mathbf{f}_{1} &= \frac{1}{\sqrt{b^2 + (a - c - d)^2}}\left[ \begin{array}{c} a - c - d \\ b \end{array}\right] \quad \mbox{ and } \\ \mathbf{f}_{2} &= \frac{1}{\sqrt{b^2 + (a - c - d)^2}}\left[ \begin{array}{c} -b \\ a - c - d \end{array}\right]\end{aligned} \nonumber \]

as the reader can verify. These agree with equation ([rotationEq]) above if \(\theta\) is an angle such that

\[\cos\theta = \frac{a - c - d}{\sqrt{b^2 + (a - c - d)^2}} \quad \mbox{ and } \quad \sin\theta = \frac{b}{\sqrt{b^2 + (a - c - d)^2}} \nonumber \]

Then \(P = \left[ \begin{array}{cc} \mathbf{f}_{1} & \mathbf{f}_{2} \end{array}\right] = \left[ \begin{array}{rr} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{array}\right]\) diagonalizes \(A\) and equation ([rotationEq2]) becomes the formula \(\mathbf{x} = P\mathbf{y}\) in Theorem [thm:027060]. This proves (1).

Finally, \(A\) is similar to \(\left[ \begin{array}{cc} \lambda_{1} & 0 \\ 0 & \lambda_{2} \end{array}\right]\) so \(\lambda_{1}\lambda_{2} = \det A = \frac{1}{4}(4ac - b^2)\). Hence the graph of \(\lambda_{1}y_{1}^2 + \lambda_{2}y_{2}^2 = 1\) is an ellipse if \(b^{2} < 4ac\) and an hyperbola if \(b^{2} > 4ac\). This proves (2).

027215 Consider the equation \(x^{2} + xy + y^{2} = 1\). Find a rotation so that the equation has no cross term.

Here \(a = b = c = 1\) in the notation of Theorem [thm:027179], so \(\cos\theta = \frac{-1}{\sqrt{2}}\) and \(\sin\theta = \frac{1}{\sqrt{2}}\). Hence \(\theta = \frac{3\pi}{4}\) will do it. The new variables are \(y_{1} = \frac{1}{\sqrt{2}}(x_{2} - x_{1})\) and \(y_{2} = \frac{-1}{\sqrt{2}}(x_{2} + x_{1})\) by ([rotationEq2]), and the equation becomes \(y_{1}^2 + 3y_{2}^2 = 2\). The angle \(\theta\) has been chosen such that the new \(y_{1}\) and \(y_{2}\) axes are the axes of symmetry of the ellipse (see the diagram). The eigenvectors \(\mathbf{f}_{1} = \frac{1}{\sqrt{2}} \left[ \begin{array}{r} -1 \\ 1 \end{array}\right]\) and \(\mathbf{f}_{2} = \frac{1}{\sqrt{2}} \left[ \begin{array}{r} -1 \\ -1 \end{array}\right]\) point along these axes of symmetry, and this is the reason for the name principal axes.

The determinant of any orthogonal matrix \(P\) is either \(1\) or \(-1\) (because \(PP^{T} = I\)). The orthogonal matrices \(\left[ \begin{array}{rr} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{array}\right]\) arising from rotations all have determinant \(1\). More generally, given any quadratic form \(q = \mathbf{x}^{T}A\mathbf{x}\), the orthogonal matrix \(P\) such that \(P^{T}AP\) is diagonal can always be chosen so that \(\det P = 1\) by interchanging two eigenvalues (and hence the corresponding columns of \(P\)). It is shown in Theorem [thm:032199] that orthogonal \(2 \times 2\) matrices with determinant \(1\) correspond to rotations. Similarly, it can be shown that orthogonal \(3 \times 3\) matrices with determinant \(1\) correspond to rotations about a line through the origin. This extends Theorem [thm:027179]: Every quadratic form in two or three variables can be diagonalized by a rotation of the coordinate system.

Congruence

We return to the study of quadratic forms in general.

027243 If \(q(\mathbf{x}) = \mathbf{x}^{T}A\mathbf{x}\) is a quadratic form given by a symmetric matrix \(A\), then \(A\) is uniquely determined by \(q\).

Let \(q(\mathbf{x}) = \mathbf{x}^{T}B\mathbf{x}\) for all \(\mathbf{x}\) where \(B^{T} = B\). If \(C = A - B\), then \(C^{T} = C\) and \(\mathbf{x}^{T}C\mathbf{x} = 0\) for all \(\mathbf{x}\). We must show that \(C = 0\). Given \(\mathbf{y}\) in \(\mathbb{R}^n\),

\[\begin{aligned} 0 = (\mathbf{x} + \mathbf{y})^TC(\mathbf{x} + \mathbf{y}) &= \mathbf{x}^TC\mathbf{x} + \mathbf{x}^TC\mathbf{y} + \mathbf{y}^TC\mathbf{x} + \mathbf{y}^TC\mathbf{y} \\ &= \mathbf{x}^TC\mathbf{y} + \mathbf{y}^TC\mathbf{x}\end{aligned} \nonumber \]

But \(\mathbf{y}^{T}C\mathbf{x} = (\mathbf{x}^{T}C\mathbf{y})^{T} = \mathbf{x}^{T}C\mathbf{y}\) (it is \(1 \times 1\)). Hence \(\mathbf{x}^{T}C\mathbf{y} = 0\) for all \(\mathbf{x}\) and \(\mathbf{y}\) in \(\mathbb{R}^n\). If \(\mathbf{e}_{j}\) is column \(j\) of \(I_{n}\), then the \((i, j)\)-entry of \(C\) is \(\mathbf{e}_{i}^{T}C\mathbf{e}_{j} = 0\). Thus \(C = 0\).

Hence we can speak of the symmetric matrix of a quadratic form.

On the other hand, a quadratic form \(q\) in variables \(x_{i}\) can be written in several ways as a linear combination of squares of new variables, even if the new variables are required to be linear combinations of the \(x_{i}\). For example, if \(q = 2x_{1}^2 - 4x_{1}x_{2} + x_{2}^2\) then

\[q = 2(x_{1} - x_{2})^2 - x_{2}^2 \quad \mbox{ and } \quad q = -2x_{1}^2 + (2x_{1} - x_{2})^2 \nonumber \]

The question arises: How are these changes of variables related, and what properties do they share? To investigate this, we need a new concept.

Let a quadratic form \(q = q(\mathbf{x}) = \mathbf{x}^{T}A\mathbf{x}\) be given in terms of variables \(\mathbf{x} = (x_{1}, x_{2}, \dots, x_{n})^{T}\). If the new variables \(\mathbf{y} = (y_{1}, y_{2}, \dots, y_{n})^{T}\) are to be linear combinations of the \(x_{i}\), then \(\mathbf{y} = A\mathbf{x}\) for some \(n \times n\) matrix \(A\). Moreover, since we want to be able to solve for the \(x_{i}\) in terms of the \(y_{i}\), we ask that the matrix \(A\) be invertible. Hence suppose \(U\) is an invertible matrix and that the new variables \(\mathbf{y}\) are given by

\[\mathbf{y} = U^{-1}\mathbf{x}, \quad \mbox{ equivalently } \mathbf{x} = U\mathbf{y} \nonumber \]

In terms of these new variables, \(q\) takes the form

\[q = q(\mathbf{x}) = (U\mathbf{y})^TA(U\mathbf{y}) = \mathbf{y}^T(U^TAU)\mathbf{y} \nonumber \]

That is, \(q\) has matrix \(U^{T}AU\) with respect to the new variables \(\mathbf{y}\). Hence, to study changes of variables in quadratic forms, we study the following relationship on matrices: Two \(n \times n\) matrices \(A\) and \(B\) are called congruent, written \(A \stackrel{c}{\sim} B\), if \(B = U^{T}AU\) for some invertible matrix \(U\). Here are some properties of congruence:

1. \(A \stackrel{c}{\sim} A\) for all \(A\).

2. If \(A \stackrel{c}{\sim} B\), then \(B \stackrel{c}{\sim} A\).

3. If \(A \stackrel{c}{\sim} B\) and \(B \stackrel{c}{\sim} C\), then \(A \stackrel{c}{\sim} C\).

4. If \(A \stackrel{c}{\sim} B\), then \(A\) is symmetric if and only if \(B\) is symmetric.

5. If \(A \stackrel{c}{\sim} B\), then \(rank \;A = rank \;B\).

The converse to (5) can fail even for symmetric matrices.

027315 The symmetric matrices \(A = \left[ \begin{array}{rr} 1 & 0 \\ 0 & 1 \end{array}\right]\) and \(B = \left[ \begin{array}{rr} 1 & 0 \\ 0 & -1 \end{array}\right]\) have the same \(rank \;\) but are not congruent. Indeed, if \(A \stackrel{c}{\sim} B\), an invertible matrix \(U\) exists such that \(B= U^{T}AU = U^{T}U\). But then \(-1 = \det B = (\det U)^{2}\), a contradiction.

The key distinction between \(A\) and \(B\) in Example [exa:027315] is that \(A\) has two positive eigenvalues (counting multiplicities) whereas \(B\) has only one.

Sylvester’s Law of Inertia027325 If \(A \stackrel{c}{\sim} B\), then \(A\) and \(B\) have the same number of positive eigenvalues, counting multiplicities.

The proof is given at the end of this section.

The index of a symmetric matrix \(A\) is the number of positive eigenvalues of \(A\). If \(q = q(\mathbf{x}) = \mathbf{x}^{T}A\mathbf{x}\) is a quadratic form, the index and rank of \(q\) are defined to be, respectively, the index and \(rank \;\) of the matrix \(A\). As we saw before, if the variables expressing a quadratic form \(q\) are changed, the new matrix is congruent to the old one. Hence the index and \(rank \;\) depend only on \(q\) and not on the way it is expressed.

Now let \(q = q(\mathbf{x}) = \mathbf{x}^{T}A\mathbf{x}\) be any quadratic form in \(n\) variables, of index \(k\) and \(rank \;r\), where \(A\) is symmetric. We claim that new variables \(\mathbf{z}\) can be found so that \(q\) is completely diagonalized—that is,

\[q(\mathbf{z}) = z_{1}^2 + \cdots + z_{k}^2 - z_{k+1}^2 - \cdots - z_{r}^2 \nonumber \]

If \(k \leq r \leq n\), let \(D_{n}(k, r)\) denote the \(n \times n\) diagonal matrix whose main diagonal consists of \(k\) ones, followed by \(r - k\) minus ones, followed by \(n - r\) zeros. Then we seek new variables \(\mathbf{z}\) such that

\[q(\mathbf{z}) = \mathbf{z}^TD_{n}(k, r)\mathbf{z} \nonumber \]

To determine \(\mathbf{z}\), first diagonalize \(A\) as follows: Find an orthogonal matrix \(P_{0}\) such that

\[P_{0}^TAP_{0} = D = \func{diag}(\lambda_{1}, \lambda_{2}, \dots, \lambda_{r}, 0, \dots, 0) \nonumber \]

is diagonal with the nonzero eigenvalues \(\lambda_{1}, \lambda_{2}, \dots, \lambda_{r}\) of \(A\) on the main diagonal (followed by \(n - r\) zeros). By reordering the columns of \(P_{0}\), if necessary, we may assume that \(\lambda_{1}, \dots, \lambda_{k}\) are positive and \(\lambda_{k + 1}, \dots, \lambda_{r}\) are negative. This being the case, let \(D_{0}\) be the \(n \times n\) diagonal matrix

\[D_{0} = \func{diag} \left( \frac{1}{\sqrt{\lambda_{1}}}, \dots, \frac{1}{\sqrt{\lambda_{k}}}, \frac{1}{\sqrt{-\lambda_{k+1}}}, \dots, \frac{1}{\sqrt{-\lambda_{r}}}, 1, \dots, 1 \right) \nonumber \]

Then \(D_{0}^{T}DD_{0} = D_{n}(k, r)\), so if new variables \(\mathbf{z}\) are given by \(\mathbf{x} = (P_{0}D_{0})\mathbf{z}\), we obtain

\[q(\mathbf{z}) = \mathbf{z}^TD_{n}(k, r)\mathbf{z} = z_{1}^2 + \cdots + z_{k}^2 - z_{k+1}^2 - \cdots - z_{r}^2 \nonumber \]

as required. Note that the change-of-variables matrix \(P_{0}D_{0}\) from \(\mathbf{z}\) to \(\mathbf{x}\) has orthogonal columns (in fact, scalar multiples of the columns of \(P_{0}\)).

027363 Completely diagonalize the quadratic form \(q\) in Example [exa:027107] and find the index and \(rank \;\).

In the notation of Example [exa:027107], the eigenvalues of the matrix \(A\) of \(q\) are \(12\), \(-8\), \(4\), \(4\); so the index is \(3\) and the \(rank \;\) is \(4\). Moreover, the corresponding orthogonal eigenvectors are \(\mathbf{f}_{1}\), \(\mathbf{f}_{2}\), \(\mathbf{f}_{3}\) (see Example [exa:027107]), and \(\mathbf{f}_{4}\). Hence \(P_{0} = \left[ \begin{array}{cccc} \mathbf{f}_{1} & \mathbf{f}_{3} & \mathbf{f}_{4} & \mathbf{f}_{2} \end{array} \right]\) is orthogonal and

\[P_{0}^TAP_{0} = \func{diag}(12, 4, 4, -8) \nonumber \]

As before, take \(D_{0} = \func{diag}(\frac{1}{\sqrt{12}}, \frac{1}{2}, \frac{1}{2}, \frac{1}{\sqrt{8}})\) and define the new variables \(\mathbf{z}\) by \(\mathbf{x} = (P_{0}D_{0})\mathbf{z}\). Hence the new variables are given by \(\mathbf{z} = D_{0}^{-1}P_{0}^T\mathbf{x}\). The result is

\[\begin{aligned} z_{1} &= \sqrt{3}(x_{1} - x_{2} - x_{3} + x_{4}) \\ z_{2} &= x_{1} + x_{2} + x_{3} + x_{4} \\ z_{3} &= x_{1} + x_{2} - x_{3} - x_{4} \\ z_{4} &= \sqrt{2}(x_{1} - x_{2} + x_{3} - x_{4}) \end{aligned} \nonumber \]

This discussion gives the following information about symmetric matrices.

027386 Let \(A\) and \(B\) be symmetric \(n \times n\) matrices, and let \(0 \leq k \leq r \leq n\).

1. \(A\) has index \(k\) and \(rank \;r\) if and only if \(A \stackrel{c}{\sim} D_{n}(k, r)\).

2. \(A \stackrel{c}{\sim} B\) if and only if they have the same \(rank \;\) and index.

1. If \(A\) has index \(k\) and \(rank \;r\), take \(U = P_{0}D_{0}\) where \(P_{0}\) and \(D_{0}\) are as described prior to Example [exa:027363]. Then \(U^{T}AU = D_{n}(k, r)\). The converse is true because \(D_{n}(k, r)\) has index \(k\) and \(rank \;r\) (using Theorem [thm:027325]).

2. If \(A\) and \(B\) both have index \(k\) and \(rank \;r\), then \(A \stackrel{c}{\sim} D_{n}(k, r) \stackrel{c}{\sim} B\) by (1). The converse was given earlier.

By Theorem [thm:027060], \(A \stackrel{c}{\sim} D_{1}\) and \(B \stackrel{c}{\sim} D_{2}\) where \(D_{1}\) and \(D_{2}\) are diagonal and have the same eigenvalues as \(A\) and \(B\), respectively. We have \(D_{1} \stackrel{c}{\sim} D_{2}\), (because \(A \stackrel{c}{\sim} B\)), so we may assume that \(A\) and \(B\) are both diagonal. Consider the quadratic form \(q(\mathbf{x}) = \mathbf{x}^{T}A\mathbf{x}\). If \(A\) has \(k\) positive eigenvalues, \(q\) has the form

\[q(\mathbf{x}) = a_{1}x_{1}^2 + \cdots + a_{k}x_{k}^2 - a_{k+1}x_{k+1}^2 - \cdots - a_{r}x_{r}^2, \ \ a_{i} > 0 \nonumber \]

where \(r = rank \;A = rank \;B\). The subspace \(W_{1} = \{\mathbf{x} \mid x_{k + 1} = \cdots = x_{r} = 0\}\) of \(\mathbb{R}^n\) has dimension \(n - r + k\) and satisfies \(q(\mathbf{x}) > 0\) for all \(\mathbf{x} \neq \mathbf{0}\) in \(W_{1}\).

On the other hand, if \(B = U^{T}AU\), define new variables \(\mathbf{y}\) by \(\mathbf{x} = U\mathbf{y}\). If \(B\) has \(k^\prime\) positive eigenvalues, \(q\) has the form

\[q(\mathbf{x}) = b_{1}y_{1}^2 + \cdots + b_{k^\prime}y_{k^\prime}^2 - b_{k^\prime+1}y_{k^\prime+1}^2 - \cdots - b_{r}y_{r}^2, \ \ b_{i} > 0 \nonumber \]

Let \(\mathbf{f}_{1}, \dots, \mathbf{f}_{n}\) denote the columns of \(U\). They are a basis of \(\mathbb{R}^n\) and

\[\mathbf{x} = U\mathbf{y} = \left[ \begin{array}{ccc} \mathbf{f}_{1} & \cdots & \mathbf{f}_{n} \end{array}\right] \left[ \begin{array}{c} y_{1} \\ \vdots \\ y_{n} \end{array}\right] = y_{1}\mathbf{f}_{1} + \cdots + y_{n}\mathbf{f}_{n} \nonumber \]

Hence the subspace \(W_{2} = span \;\{\mathbf{f}_{k^{\prime}+1}, \dots, \mathbf{f}_{r}\}\) satisfies \(q(\mathbf{x}) < 0\) for all \(\mathbf{x} \neq \mathbf{0}\) in \(W_{2}\). Note \(dim \;W_{2} = r - k^{\prime}\). It follows that \(W_{1}\) and \(W_{2}\) have only the zero vector in common. Hence, if \(B_{1}\) and \(B_{2}\) are bases of \(W_{1}\) and \(W_{2}\), respectively, then (Exercise [ex:ex6_3_33]) \(B_{1} \cup B_{2}\) is an independent set of \((n - r + k) + (r - k^{\prime}) = n + k - k^{\prime}\) vectors in \(\mathbb{R}^n\). This implies that \(k \leq k^{\prime}\), and a similar argument shows \(k^{\prime} \leq k\).

Exercises for 1

solutions

2

In each case, find a symmetric matrix \(A\) such that \(q = \mathbf{x}^{T}B\mathbf{x}\) takes the form \(q = \mathbf{x}^{T}A\mathbf{x}\).

\(\left[ \begin{array}{rr} 1 & 1 \\ 0 & 1 \end{array}\right]\) \(\left[ \begin{array}{rr} 1 & 1 \\ -1 & 2 \end{array}\right]\) \(\left[ \begin{array}{rrr} 1 & 0 & 1 \\ 1 & 1 & 0 \\ 0 & 1 & 1 \end{array}\right]\) \(\left[ \begin{array}{rrr} 1 & 2 & -1 \\ 4 & 1 & 0 \\ 5 & -2 & 3 \end{array}\right]\)

2. \(A = \left[ \begin{array}{rr} 1 & 0 \\ 0 & 2 \end{array}\right]\)

3. \(A = \left[ \begin{array}{rrr} 1 & 3 & 2 \\ 3 & 1 & -1 \\ 2 & -1 & 3 \end{array}\right]\)

In each case, find a change of variables that will diagonalize the quadratic form \(q\). Determine the index and \(rank \;\) of \(q\).

1. \(q = x_{1}^2 + 2x_{1}x_{2} + x_{2}^2\)

2. \(q = x_{1}^2 + 4x_{1}x_{2} + x_{2}^2\)

3. \(q = x_{1}^2 + x_{2}^2 + x_{3}^2 - 4(x_{1}x_{2} + x_{1}x_{3} + x_{2}x_{3})\)

4. \(q = 7x_{1}^2 + x_{2}^2 + x_{3}^2 + 8x_{1}x_{2} + 8x_{1}x_{3} - 16x_{2}x_{3}\)

5. \(q = 2(x_{1}^2 + x_{2}^2 + x_{3}^2 - x_{1}x_{2} + x_{1}x_{3} - x_{2}x_{3})\)

6. \(q = 5x_{1}^2 + 8x_{2}^2 + 5x_{3}^2 - 4(x_{1}x_{2} + 2x_{1}x_{3} + x_{2}x_{3})\)

7. \(q = x_{1}^2 - x_{3}^2 - 4x_{1}x_{2} + 4x_{2}x_{3}\)

8. \(q = x_{1}^2 + x_{3}^2 - 2x_{1}x_{2} + 2x_{2}x_{3}\)

2. \(P = \frac{1}{\sqrt{2}}\left[ \begin{array}{rr} 1 & 1 \\ 1 & -1 \end{array}\right]\);
   \(\mathbf{y} = \frac{1}{\sqrt{2}}\left[ \begin{array}{r} x_{1} + x_{2} \\ x_{1} - x_{2} \end{array}\right]\);
   \(q = 3y_{1}^2 - y_{2}^2\); \(1\), \(2\)

3. \(P = \frac{1}{3}\left[ \begin{array}{rrr} 2 & 2 & -1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{array}\right]\);
   \(\mathbf{y} = \frac{1}{3}\left[ \begin{array}{rcrcr} 2x_{1} & + & 2x_{2} & - & x_{3} \\ 2x_{1} & - & x_{2} & + & 2x_{3} \\ -x_{1} & + & 2x_{2} & + & 2x_{3} \end{array}\right]\);
   \(q = 9y_{1}^2 + 9y_{2}^2 - 9y_{3}^2\); \(2\), \(3\)

4. \(P = \frac{1}{3}\left[ \begin{array}{rrr} -2 & 1 & 2 \\ 2 & 2 & 1 \\ 1 & -2 & 2 \end{array}\right]\);
   \(\mathbf{y} = \frac{1}{3}\left[ \begin{array}{rcrcr} -2x_{1} & + & 2x_{2} & + & x_{3} \\ x_{1} & + & 2x_{2} & - & 2x_{3} \\ 2x_{1} & + & x_{2} & + & 2x_{3} \end{array}\right]\);
   \(q = 9y_{1}^2 + 9y_{2}^2\); \(2\), \(2\)

5. \(P = \frac{1}{\sqrt{6}}\left[ \begin{array}{rrr} -\sqrt{2} & \sqrt{3} & 1 \\ \sqrt{2} & 0 & 2 \\ \sqrt{2} & \sqrt{3} & -1 \end{array}\right]\);
   \(\mathbf{y} = \frac{1}{\sqrt{6}}\left[ \begin{array}{rcrcr} -\sqrt{2}x_{1} & + & \sqrt{2}x_{2} & + & \sqrt{2}x_{3} \\ \sqrt{3}x_{1} & & & + & \sqrt{3}x_{3} \\ x_{1} & + & 2x_{2} & - & x_{3} \end{array}\right]\);
   \(q = 2y_{1}^2 + y_{2}^2 - y_{3}^2\); \(2\), \(3\)

For each of the following, write the equation in terms of new variables so that it is in standard position, and identify the curve.

\(xy = 1\) \(3x^{2} - 4xy = 2\) \(6x^{2} + 6xy - 2y^{2} = 5\) \(2x^{2} + 4xy + 5y^{2} = 1\)

2. \(x_{1} = \frac{1}{\sqrt{5}}(2x - y)\), \(y_{1} = \frac{1}{\sqrt{5}}(x + 2y)\); \(4x_{1}^2 - y_{1}^2 = 2\); hyperbola

3. \(x_{1} = \frac{1}{\sqrt{5}}(x + 2y)\), \(y_{1} = \frac{1}{\sqrt{5}}(2x - y)\); \(6x_{1}^2 + y_{1}^2 = 1\); ellipse

Consider the equation \(ax^{2} + bxy + cy^{2} = d\), where \(b \neq 0\). Introduce new variables \(x_{1}\) and \(y_{1}\) by rotating the axes counterclockwise through an angle \(\theta\). Show that the resulting equation has no \(x_{1}y_{1}\)-term if \(\theta\) is given by

\[\begin{aligned} & \cos2\theta = \frac{a - c}{\sqrt{b^2+(a-c)^2}} \\ & \sin2\theta = \frac{b}{\sqrt{b^2+(a-c)^2}}\end{aligned} \nonumber \]

[Hint: Use equation ([rotationEq2]) preceding Theorem [thm:027179] to get \(x\) and \(y\) in terms of \(x_{1}\) and \(y_{1}\), and substitute.]

2. Basis \(\{(i, 0, i), (1, 0, -1)\}\), dimension \(2\)

3. Basis \(\{(1, 0, -2i), (0, 1, 1 - i)\}\), dimension \(2\)

Prove properties (1)–(5) preceding Example [exa:027315].

If \(A \stackrel{c}{\sim} B\) show that \(A\) is invertible if and only if \(B\) is invertible.

If \(\mathbf{x} = (x_{1}, \dots, x_{n})^{T}\) is a column of variables, \(A = A^{T}\) is \(n \times n\), \(B\) is \(1 \times n\), and \(c\) is a constant, \(\mathbf{x}^{T}A\mathbf{x} + B\mathbf{x} = c\) is called a quadratic equation in the variables \(x_{i}\).

2. Put \(x_{1}^2 + 3x_{2}^2 + 3x_{3}^2 + 4x_{1}x_{2} - 4x_{1}x_{3} + 5x_{1} - 6x_{3} = 7\) in this form and find variables \(y_{1}\), \(y_{2}\), \(y_{3}\) as in (a).

2. \(3y_{1}^2 + 5y_{2}^2 - y_{3}^2 - 3\sqrt{2}y_{1} + \frac{11}{3}\sqrt{3}y_{2} + \frac{2}{3}\sqrt{6}y_{3} = 7\)
   \(y_{1} = \frac{1}{\sqrt{2}}(x_{2} + x_{3})\), \(y_{2} = \frac{1}{\sqrt{3}}(x_{1} + x_{2} - x_{3})\), \(y_{3} = \frac{1}{\sqrt{6}}(2x_{1} - x_{2} + x_{3})\)

Given a symmetric matrix \(A\), define \(q_{A}(\mathbf{x}) = \mathbf{x}^{T}A\mathbf{x}\). Show that \(B \stackrel{c}{\sim} A\) if and only if \(B\) is symmetric and there is an invertible matrix \(U\) such that \(q_{B}(\mathbf{x}) = q_{A}(U\mathbf{x})\) for all \(\mathbf{x}\). [Hint: Theorem [thm:027243].]

Let \(q(\mathbf{x}) = \mathbf{x}^{T}A\mathbf{x}\) be a quadratic form where \(A = A^{T}\).

1. Show that \(q(\mathbf{x}) > 0\) for all \(\mathbf{x} \neq \mathbf{0}\), if and only if \(A\) is positive definite (all eigenvalues are positive). In this case, \(q\) is called positive definite.

2. Show that new variables \(\mathbf{y}\) can be found such that \(q = \|\mathbf{y}\|^{2}\) and \(\mathbf{y} = U\mathbf{x}\) where \(U\) is upper triangular with positive diagonal entries. [Hint: Theorem [thm:024907].]

2. By Theorem [thm:024907] let \(A = U^{T}U\) where \(U\) is upper triangular with positive diagonal entries. Then \(q = \mathbf{x}^{T}(U^{T}U)\mathbf{x} = (U\mathbf{x})^{T}U\mathbf{x} = \| U\mathbf{x} \|^{2}\).

A bilinear form \(\beta\) on \(\mathbb{R}^n\) is a function that assigns to every pair \(\mathbf{x}\), \(\mathbf{y}\) of columns in \(\mathbb{R}^n\) a number \(\beta(\mathbf{x}, \mathbf{y})\) in such a way that

\[\begin{aligned} \beta(r\mathbf{x} + s\mathbf{y}, \mathbf{z}) = r\beta(\mathbf{x}, \mathbf{z}) + s\beta(\mathbf{y}, \mathbf{z}) \\ \beta(\mathbf{x}, r\mathbf{y} + s\mathbf{z}) = r\beta(\mathbf{x}, \mathbf{z}) + s\beta(\mathbf{x}, \mathbf{z})\end{aligned} \nonumber \]

for all \(\mathbf{x}\), \(\mathbf{y}\), \(\mathbf{z}\) in \(\mathbb{R}^n\) and \(r\), \(s\) in \(\mathbb{R}\). If \(\beta(\mathbf{x}, \mathbf{y}) = \beta(\mathbf{y}, \mathbf{x})\) for all \(\mathbf{x}\), \(\mathbf{y}\), \(\beta\) is called symmetric.

1. If \(\beta\) is a bilinear form, show that an \(n \times n\) matrix \(A\) exists such that \(\beta(\mathbf{x}, \mathbf{y}) = \mathbf{x}^{T}A\mathbf{y}\) for all \(\mathbf{x}\), \(\mathbf{y}\).

2. Show that \(A\) is uniquely determined by \(\beta\).

3. Show that \(\beta\) is symmetric if and only if \(A = A^{T}\).