3.3 The inverse of a matrix

\( \def\Span#1{\text{Span}\left\lbrace #1\right\rbrace} \def\vect#1{\mathbf{#1}} \def\ip{\boldsymbol{\cdot}} \def\iff{\Longleftrightarrow} \def\cp{\times} \)

 

Introduction

In the Section Sec:MatrixOps we defined the sum and product of matrices (of compatible sizes), and we saw that to a certain extent matrix algebra is guided by the same rules as the arithmetic of real numbers. We can also subtract two matrices via \[ A - B = A + (-1)B, \nonumber \nonumber\] but we did not mention division of matrices. For two numbers \(a \) and \(b \), with \(a \neq 0 \), the equation \[ ax = b \nonumber \nonumber\] has the unique solution \[ x = \frac{b}{a} = a^{-1}b = ba^{-1} \nonumber \nonumber\] where \[ a^{-1} = \frac1a \nonumber \nonumber\] is the (unique) solution of the equation \[ ax = 1 \nonumber \nonumber\] The bad news: \[ \frac{A}{B} \text{ cannot be defined in any useful way!} \nonumber \nonumber\] First of all the corresponding matrix equation \[ AX = B, A\neq O \nonumber \nonumber\] does not always have a solution, or the solution is not unique, not even in the case of two \(n \times n \) matrices \(A \) and \(B \). Two examples to illustrate this:

Example

 

The matrix equation \[ AX = B \nonumber \nonumber\] where \[ A = \left[\begin{array}{rr} 1 & 2 \\ 1 & 2 \end{array}\right], \quad B = \left[\begin{array}{rr} 1 & 1 \\ 0 & 0 \end{array}\right] \nonumber \nonumber\] does not have a solution. Why? Well, any column of \(AX \) is a linear combination of the columns of \(A \), and the columns of \(B \) obviously cannot be written as such linear combinations: \[ \left[\begin{array}{r} 1 \\ 0 \end{array}\right] \neq c_1 \left[\begin{array}{r} 1 \\ 1 \end{array}\right] + c_2 \left[\begin{array}{r} 2 \\ 2 \end{array}\right] \quad \text{for all } c_1,c_2 \quad \text{ in } \mathbb{R}. \nonumber \nonumber\]

Example

 

The matrix equation \[ AX = B \nonumber \nonumber\] where \[ A = \left[\begin{array}{rr} 1 & 2 \\ 1 & 2 \end{array}\right], \quad B = \left[\begin{array}{rr} 1 & 4 \\ 1 & 4 \end{array}\right] \nonumber \nonumber\] has infinitely many solutions. Two of those are for instance \[ X_1 = \left[\begin{array}{rr} 1 & 0 \\ 0 & 2 \end{array}\right] \quad \text{and} \quad X_2 = \left[\begin{array}{rr} -1 & 2 \\ 1 & 1 \end{array}\right]. \nonumber \nonumber\]

And lastly, if there is a matrix \(C \) for which \[ CA = I \nonumber \nonumber\] and we would adopt the notation \[ C = A^{-1} \nonumber \nonumber\] then \[ \begin{array}{rcl} AX = B &\Rightarrow& C(AX) = CB \\ &\Rightarrow& (CA)X = IX = \underline{\underline{X =}} CB = \underline{\underline{A^{-1}B}}. \end{array} \nonumber \nonumber\] So \(X = A^{-1}B \). However, it is in no way clear why \(A^{-1}B \) and \(BA^{-1} \) should be equal, and in general indeed they are not. So the notation \[ \dfrac{B}{A} \nonumber \nonumber\] will still be ambiguous. For non-square matrices things are even worse. In this section we will only consider square matrices.

Definition and basic properties of the inverse

 

Definition

 

A square matrix \(A \) is called invertible if there exists a matrix \(B \) for which \[ AB = BA = I. \nonumber \nonumber\] In this situation the matrix \(B \) is called the inverse of \(A \) and we write \[ B = A^{-1}. \nonumber \nonumber\] A matrix that is invertible is also called a regular matrix, and a non-invertible matrix is also called a singular matrix.

Note the use of the definite article the in the sentence `\(B \) is called the inverse of \(A \)'. The following proposition justifies this choice of word.

Proposition

 

If an inverse of a matrix \(A \) exists, then it is unique.

The proof is very short, when we plug in the right idea at the right place:

Skip/Read the proof

Proof

Suppose \(B \) and \(C \) are two matrices that satisfy the properties of being an inverse of \(A \), i.e. \[ AB = BA = I \quad \text{and} \quad AC = CA = I. \nonumber \nonumber\] Then the following chain of identities proves that \(B \) and \(C \) must be equal: \[ B = B I = B (AC) = (BA) C= I C = C. \nonumber \nonumber\]

Note

Actually, the proof shows slightly more, as the assumptions \[ CA= I, \quad AB = I \nonumber \nonumber\] are not used. In fact it shows that for three \(n \times n \) matrices \(A \), \(B \) and \(C \) \[ \text{if} \quad BA = I \quad \text{ and }\quad AC = I \quad\text{ then } \quad B = C. \nonumber \nonumber\]

Example

 

For the matrices \[ A = \left[\begin{array}{rr} 1 & 2 \\ 3 & 5 \end{array}\right] \quad \text{and} \quad B = \left[\begin{array}{rr} -5 & 2 \\ 3 & -1 \end{array}\right] \nonumber \nonumber\] we see \[ \left[\begin{array}{rr} 1 & 2 \\ 3 & 5 \end{array}\right] \left[\begin{array}{rr} -5 & 2 \\ 3 & -1 \end{array}\right] = \left[\begin{array}{rr} -5 & 2 \\ 3 & -1 \end{array}\right] \left[\begin{array}{rr} 1 & 2 \\ 3 & 5 \end{array}\right] = \left[\begin{array}{rr} 1 & 0 \\ 0 & 1 \end{array}\right]. \nonumber \nonumber\] So \(A \) and \(B \) are each other's inverse. Another example: \[ \left[\begin{array}{rrr} 1 & 1 & 0 \\ 1 & 1 & 1 \\ 0 & 1 & 1 \end{array}\right] \left[\begin{array}{rrr} 0 & 1 & -1 \\ 1 & -1 & 1 \\ -1 & 1 & 0 \end{array}\right] = \left[\begin{array}{rrr} 1 & 0 & 0\\ 0 & 1 & 0 \\ 0 & 0 & 1\end{array}\right]. \nonumber \nonumber\] You may check for yourself that the product in the other order also gives \(I \), so \[ \left[\begin{array}{rrr} 1 & 1 & 0 \\ 1 & 1 & 1 \\ 0 & 1 & 1 \end{array}\right]^{-1} = \left[\begin{array}{rrr} 0 & 1 & -1 \\ 1 & -1 & 1 \\ -1 & 1 & 0 \end{array}\right] \nonumber \nonumber\] It will appear (Remark 22) that for square matrices, a one-sided inverse is automatically a two-sided inverse, by which we mean \[ \text{if } AB = I \quad \text{then also}\quad BA = I. \nonumber \nonumber\]

The first example can be generalized:

Proposition

 

If \(A = \left[\begin{array}{rr} a & b \\ c & d \end{array}\right] \), then \(A^{-1} \) exists if and only if \[ ad - bc \neq 0. \nonumber \nonumber\] In that case \[ A^{-1} = \left[\begin{array}{rr} a & b \\ c & d \end{array}\right]^{-1} = \left[\begin{array}{rr} \dfrac{d}{ad - bc} & \dfrac{-b}{ad - bc} \\ \dfrac{-c}{ad - bc} & \dfrac{a}{ad - bc} \end{array}\right] = \frac{1}{ad-bc} \left[\begin{array}{rr} d &- b \\ -c & a \end{array}\right]. \nonumber \nonumber\]

We leave the verification as an exercise.

Exercise

 

Show that matrix \(B \) proposed in Proposition 7 indeed satisfies \[ AB = BA = I. \nonumber \nonumber\] Also check that the first matrix in Example 6 illustrates the formula.

Note

 

Note that the condition \[ ad - bc \neq 0 \nonumber \nonumber\] is equivalent to the statement \[ \text{the vectors } \left[\begin{array}{r} a \\ c \end{array}\right] \text{ and } \left[\begin{array}{r} b \\ d \end{array}\right] \text{ are linearly independent.} \nonumber \nonumber\] First we show that \[ ad - bc = 0 \quad \text{ implies } \left[\begin{array}{rr} a &b\\ c&d \end{array}\right] \text{ has linearly dependent columns.} \nonumber \nonumber\] It is best to split this in two cases: \[ ad = 0 \quad \quad \text{and} \quad ad \neq 0. \nonumber \nonumber\] If we assume \[ ad - bc = 0 \quad \quad \text{and} \quad ad = 0, \nonumber \nonumber\] then we have \[ b = 0 \quad \quad \text{or} \quad c = 0 \nonumber \nonumber\] which leads to a matrix \[ \left[\begin{array}{rr} a & b \\ c & d \end{array}\right] \nonumber \nonumber\] with either a zero row or a zero column, which will indeed have linearly dependent columns. Second, if we assume \[ ad - bc = 0 \quad \quad\text{and} \quad ad \neq 0 \nonumber \nonumber\] then both \(a \neq 0 \) and \(d \neq 0 \), in which case \[ d = \frac{bc}{a}, \text{ so } \left[\begin{array}{r} b \\ d \end{array}\right] = \left[\begin{array}{r} b \\ \frac{bc}{a} \end{array}\right] = \dfrac{b}{a} \left[\begin{array}{r} a \\ c \end{array}\right], \nonumber \nonumber\] hence the columns are again linearly dependent Thus we have shown: \[ ad-bc = 0 \quad \quad \longrightarrow \quad \left[\begin{array}{rr} a & b \\ c & d \end{array}\right] \text{ has linearly dependent columns.} \nonumber \nonumber\] Next let us consider the converse, i.e. \[ \left[\begin{array}{rr} a & b \\ c&d \end{array}\right] \text{ has linearly dependent columns} \text{ implies } ad - bc = 0. \nonumber \nonumber\] If a \(2 \times 2 \) matrix has two linearly dependent columns, then one of the columns will be a multiple of the other column, e.g. \[ \text{either } \quad \left[\begin{array}{r} a \\ c \end{array}\right] = k \left[\begin{array}{r} b \\ d \end{array}\right] \quad \text{or}\quad \left[\begin{array}{r} b \\ d \end{array}\right] = k \left[\begin{array}{r} a \\ c \end{array}\right] . \nonumber \nonumber\] In both cases it is easily checked that \[ ad-bc = 0. \nonumber \nonumber\]

The following proposition shows that the above considerations can be generalized.

Proposition

 

If \(A \) is a square matrix, then \[ AX = I \nonumber \nonumber\] has a unique solution if and only if \[ A \text{ has linearly independent columns.} \nonumber \nonumber\]

Skip/Read the proof

Proof

As in the proof in Remark Rem:DetZeroDependentColumns we have to prove two implications: \[ AX = I \text{ has a unique solution } \Longrightarrow A \text{ has linearly independent columns} \nonumber \nonumber\] and \[ A \text{ has linearly independent columns} \Longrightarrow AX = I \text{ has a unique solution. } \nonumber \nonumber\] For the first part, assume that \[ AX = I \text{ has a (unique) solution. } \nonumber \nonumber\] That means that every column \(\vect{e}_j \) of the identity matrix is a linear combination of columns \(\vect{a}_1, \ldots, \vect{a}_n \) of \(A \). So the span of the columns of \(A \) contains the span of the columns of \(\vect{e}_1, \ldots, \vect{e}_n \), which is the whole \(\mathbb{R}^n \). Thus every linear system \[ A\vect{x} =\vect{b}, \vect{b} \in \mathbb{R}^{n} \nonumber \nonumber\] has a solution. Then the reduced echelon form of \(A \) must have a pivot in every row, and, since it is a square matrix, it must be the identity matrix. Consequently, it has a pivot in every column, so the linear system \[ A\vect{x} =\vect{0} \nonumber \nonumber\] only has the trivial solution, which proves that indeed the columns of \(A \) are linearly independent. For the converse, suppose that \(A \) has linearly independent columns. Then the reduced echelon form of \(A \) must be the identity matrix. This implies that for each \(\vect{b} \) in \(\mathbb{R}^n \) \[ [ A | \vect{b} ] \sim [ I | \vect{b'} ], \nonumber \nonumber\] and in particular, each linear system \[ A\vect{x} =\vect{e}_j \nonumber \nonumber\] has a unique solution. If we denote this solution by \(\vect{c}_j \) we have that \[ A[ \vect{c}_1 \quad \vect{c}_2 \quad \ldots \quad \vect{c}_n ] = [ A\vect{c}_1 \quad A\vect{c}_2 \quad \ldots \quad A\vect{c}_n ] = [ \vect{e}_1 \quad \vect{e}_2 \quad \ldots \quad \vect{e}_n ] = I. \nonumber \nonumber\] Since all solutions \(\vect{c}_j \) are unique, the solution of the equation \[ A\vect{x} = I \nonumber \nonumber\] is unique as well.

It makes sense that the solution \(B \) of this matrix equation will be the inverse of \(A \), and it is, but it takes some effort to show that the other requirement, \[ BA = I \nonumber \nonumber\] is also fulfilled. In the next subsection we will see that the matrix equation \[ AX = I \nonumber \nonumber\] will lead the way to an algorithm to compute the inverse of a matrix. Before we go there we will look at some general properties of invertible matrices.

Proposition

 

If the \(n \times n \) matrix \(A \) is invertible and \(B \) is an \(n \times p \) matrix, then the solution of the matrix equation \[ AX = B \nonumber \nonumber\] is unique, and given by \[ X = A^{-1}B. \nonumber \nonumber\] Often the matrix \(B \) has only one column: \[ AX = \vect{b} \quad \text{has the unique solution} \quad X = A^{-1}\vect{b}. \nonumber \nonumber\]

Skip/Read the proof

Proof

We multiply both sides of the equation \[ AX = B \nonumber \nonumber\] by \(A^{-1} \) and use the fact that the matrix product has the associative property: \[ \begin{array}{rl} AX = B \Longrightarrow A^{-1}(AX) = A^{-1}B & \Longrightarrow (A^{-1}A)X = IX = A^{-1}B \\ &\Longrightarrow X = A^{-1}B. \end{array} \nonumber \nonumber\]

We illustrate the proposition by an example.

Example

 

Suppose the matrix \(A \) and the vectors \(\vect{b}_1 \) and \(\vect{b}_1 \) are given by \[ A= \left[\begin{array}{rr}1 & 2 \\ 3 & 4 \end{array}\right], \quad \vect{b}_1= \left[\begin{array}{r}-1 \\ 1 \end{array}\right] \quad \text{and} \quad \vect{b}_2= \left[\begin{array}{r}2 \\ 10 \end{array}\right]. \nonumber \nonumber\] Consider the two linear systems \[ A\vect{x} =\vect{b}_1\quad \text{and} \quad A\vect{x} = \vect{b}_2. \nonumber \nonumber\] Using the inverse matrix \[ A^{-1} = \frac{1}{-2} \left[\begin{array}{rr}4 & -2 \\ -3 & 1 \end{array}\right] = \frac{1}{2} \left[\begin{array}{rr}-4 & 2 \\ 3 & -1 \end{array}\right], \nonumber \nonumber\] the two solutions are quickly written down: \[ \vect{x}_1= A^{-1}\vect{b}_1= \frac{1}{2} \left[\begin{array}{rr}-4 & 2 \\ 3 & -1 \end{array}\right] \left[\begin{array}{r}-1 \\ 1 \end{array}\right] = \left[\begin{array}{r}3 \\ -2 \end{array}\right] \nonumber \nonumber\] and likewise \[ \vect{x}_2= \frac{1}{2} \left[\begin{array}{rr}-4 & 2 \\ 3 & -1 \end{array}\right] \left[\begin{array}{r}2 \\ 10 \end{array}\right] = \left[\begin{array}{r}6 \\ -2 \end{array}\right]. \nonumber \nonumber\]

A note of warning: the proof of Proposition 11 is based on the existence of the inverse of the matrix \(A \). Beware of this: never start using the expression \(A^{-1} \) unless you have made sure first that the matrix \(A \) is indeed invertible. If not, you may lead yourself into inconsistencies like in the following example:

Example

 

What goes wrong in the following `proof' of the statement: \[ \text{ if } A^2 = A \text{ and } A\neq O, \text{ then } A = I. \nonumber \nonumber\] `Fallacious proof': \[ A^2 = A \Longrightarrow A^{-1}A^2 = A^{-1}A = I \Longrightarrow A = I, \nonumber \nonumber\] since \[ A^{-1}A^2 = A^{-1}(A A) = (A^{-1}A)A = IA = A. \nonumber \nonumber\] Somewhere something must have gone wrong, as the following counterexample shows: \[ \left[\begin{array}{rr} \frac12 & \frac12 \\ \frac12 & \frac12 \end{array}\right]^2 = \left[\begin{array}{rr} \frac12 & \frac12 \\ \frac12 & \frac12 \end{array}\right] \quad\text{whereas} \quad \left[\begin{array}{rr} \frac12 & \frac12 \\ \frac12 & \frac12 \end{array}\right] \neq \left[\begin{array}{rr} 1 & 0 \\ 0 & 1 \end{array}\right] \nonumber \nonumber\] So, where exactly did it go wrong?!

The next proposition contains a few rules to manipulate inverse matrices.

Proposition

 

If \(A \) is invertible and \(c \neq 0 \), then the following is true

1. The matrix \(cA \) is invertible, and \[ (cA)^{-1} = \dfrac1c A^{-1}. \nonumber \nonumber\]
2. The matrix \(A^T \) is invertible, and \[ (A^T)^{-1} = (A^{-1})^T. \nonumber \nonumber\]
    
3. The matrix \(A^{-1} \) is invertible, and \[ (A^{-1})^{-1} = A. \nonumber \nonumber\]

Skip/Read the proof

Proof

All statements can be proved by verifying that the relevant products are equal to \(I \).

1. The matrix \(A^{-1} \) exists, and so does \(\dfrac1c A^{-1} \). We find: \[ (cA) \cdot \dfrac1c A^{-1} = c\cdot \dfrac1c A\cdot A^{-1} = 1 \quad \cdot I = I, \nonumber \nonumber\] and likewise \(\dfrac1c A^{-1}\cdot (cA) = I \), which proves that indeed \(\dfrac1c A^{-1} = (cA)^{-1} \).
2. Since it is given that \(A^{-1} \) exists we can proceed as follows, where we make use of the characteristic property \( B^TA^T = (AB)^T \): \[ (A^{-1})^TA^T = ( AA^{-1})^T = I^T = I \text{ and } A^T(A^{-1})^T =( A^{-1}A)^T = I^T = I, \nonumber \nonumber\] which settles the second statement. To prove (iii), see Exercise 15.

Exercise

 

Prove the last statement of the previous proposition.

The next example gives an illustration of property ii.

Example

 

We consider the matrix \[ A = \left[\begin{array}{rrr} 2 & 6 & 5 \\ 0 & 2 & 2 \\ 0 & 0 & 3 \end{array}\right]. \nonumber \nonumber\] It has the inverse matrix \[ B = \left[\begin{array}{rrr} 1/2 & -3/2 & 1/6 \\ 0 & 1/2 & -1/3 \\ 0 & 0 & 1/3 \end{array}\right], \nonumber \nonumber\] which can be checked by showing that \(AB \) and \(BA \) are equal to \(I \). So \(B = A^{-1} \), and \(B^T = (A^{-1})^T \). We also have \[ A^TB^T = \left[\begin{array}{rrr} 2 & 0 & 0 \\ 6 & 2 & 0 \\ 5 & 2 & 3 \end{array}\right] \left[\begin{array}{rrr} 1/2 & 0 & 0 \\ -3/2 & 1/2 & 0 \\ 1/6 & -1/3 & 1/3 \end{array}\right] = \left[\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1\end{array}\right] \nonumber \nonumber\] as well as \(B^TA^T = I \), which proves that \(B^T = (A^{-1})^T = (A^T)^{-1} \).

The last property we mention and prove is the product rule for the matrix inverse:

Proposition

 

If \(A \) and \(B \) are invertible \(n \times n \) matrices then the matrix \(AB \) is also invertible, and \[ (AB)^{-1} = B^{-1}A^{-1}. \nonumber \nonumber\]

Skip/Read the proof

Proof

Again we just check that the properties of the definition hold: suppose that \(A \) and \(B \) are invertible with inverses \(A^{-1} \) and \(B^{-1} \). Then using the associative property we find \[ (B^{-1}A^{-1})(AB) = B^{-1}A^{-1}AB = B^{-1}(A^{-1}A)B = B^{-1}IB = B^{-1}B = I, \nonumber \nonumber\] and along the same lines \[ (AB) B^{-1}A^{-1} = I. \nonumber \nonumber\] This shows that \(B^{-1}A^{-1} \) is indeed the inverse of \(AB \).

Exercise

 

Is the identity \[ ((AB)^T)^{-1} = (A^T)^{-1}(B^T)^{-1} \nonumber \nonumber\] true or false? In case it is true, give an argument, when false, give a counterexample.

Exercise

 

Prove the following converse of Proposition 17: if \(A \) and \(B \) are \(n\times n \) matrices for which the product \(AB \) is invertible, then \(A \) and \(B \) are both invertible. Make sure that you do not use \(A^{-1} \) or \(B^{-1} \) prematurely, i.e., before you have established that they exist.

How to compute the inverse

 

In fact the construction of the inverse of a matrix was already present implicitly in Propositon 10. The inverse of the matrix \(A \) must satisfy the equation \(AX = I \). Written out column by column this means that \[ AX = I \quad \iff \quad A[ \vect{x}_1 \quad \vect{x}_2 \quad \ldots \quad \vect{x}_n ] = [ \vect{e}_1 \quad \vect{e}_2 \quad \ldots \quad \vect{e}_n ]. \nonumber \nonumber\] For the existence of a solution of this equation Prop. 10 tells us it is \underline{necessary} that \(A \) has independent columns, and we can furthermore read off that the columns of the matrix \(X \) will be the (unique) solutions of the linear systems \[ A\vect{x}_k = \vect{e}_k, \text{ where } k = 1,2,\ldots, n \nonumber \nonumber\] So let us first focus on this equation by considering a fairly general \(3\times 3 \) matrix \(A \).

Example

 

For the matrix \[ A = \left[\begin{array}{rrr} 1 & 1 & 4 \\ 1 & -1 & -1 \\ 2 & -2 & -4 \end{array}\right] \nonumber \nonumber\] we find the solution \(B \) of the matrix equation (which will appear to exist) \[ AX = I \nonumber \nonumber\] and then check whether \[ BA = I \nonumber \nonumber\] also holds. In which case we can truthfully assert that \(B = A^{-1} \). Instead of finding the solution \(X \) column by column, which gives three linear systems with the same coefficient matrix, \[ \left[\begin{array}{rrrr | r} 1 & 1 & 4 & 1 \\ 1 & -1 & -1 & 0 \\ 2 & -2 & -4 & 0 \end{array}\right], \quad \left[\begin{array}{rrrr | r} 1 & 1 & 4 & 0 \\ 1 & -1 & -1 & 1 \\ 2 & -2 & -4 & 0 \end{array}\right], \text{ and } \left[\begin{array}{rrrr | r} 1 & 1 & 4 & 0 \\ 1 & -1 & -1 & 0 \\ 2 & -2 & -4 & 1 \end{array}\right], \nonumber \nonumber\] we can solve the three linear systems simultaneously using a combined augmented matrix which we may denote by either \[ \left[\begin{array}{rrrrrrrr | r} 1 & 1 & 4 & 1 & 0 & 0 \\ 1 & -1 & -1 & 0 & 1 & 0 \\ 2 & -2 & -4 & 0 & 0 & 1 \end{array}\right] \quad \text{or} \quad \left[\begin{array}{rrrrrr | r} 1 & 1 & 4 & 1 & 0 & 0 \\ 1 & -1 & -1 & 0 & 1 & 0 \\ 2 & -2 & -4 & 0 & 0 & 1 \end{array}\right] = \left[ A \, \vert \, I \right]. \nonumber \nonumber\] Let us first row reduce this matrix and then draw conclusions: \[ \left[ A \, \vert \, I \right] = \left[\begin{array}{rrrrrr | r} 1 & 1 & 4 & 1 & 0 & 0 \\ 1 & -1 & -1 & 0 & 1 & 0 \\ 2 & -2 & -4 & 0 & 0 & 1 \end{array}\right] \quad \begin{array}{r} [R_{1}] \\ [R_{2} -1 R_{1}] \\ [R_{3} -2 R_{1}] \end{array} \nonumber \nonumber\] \[ \sim \left[\begin{array}{rrrrrr | r} 1 & 1 & 4 & 1 & 0 & 0 \\ 0 & -2 & -5 & -1 & 1 & 0 \\ 0 & -4 & -12 & -2 & 0 & 1 \end{array}\right] \quad \begin{array}{r} [R_{1}+ R_{2}] \\ [R_{2}] \\ [R_{3} -2 R_{2}] \end{array} \nonumber \nonumber\] \[ \sim \left[\begin{array}{rrrrrr | r} 1 & 0 & 3/2 & 1/2 & 1/2 & 0 \\ 0 & -2 & -5 & -1 & 1 & 0 \\ 0 & 0 & -2 & 0 & -2 & 1 \end{array}\right] \quad \begin{array}{r} [R_{1}+ R_{3}] \\ [R_{2} -R_{3}] \\ [R_{3}] \end{array} \nonumber \nonumber\] \[ \sim \left[\begin{array}{rrrrrr | r} 1 & 0 & 0 & 1/2 & -1 & 3/4 \\ 0 & -2 & 0 & -1 & 6 & -5/2 \\ 0 & 0 & -2 & 0 & -2 & 1 \end{array}\right] \quad \begin{array}{r} [R_{1}] \\ [ -\frac12 R_{2}] \\ [ -\frac12 R_{3}] \end{array} \nonumber \nonumber\] \[ \sim \left[\begin{array}{rrrrrr | r} 1 & 0 & 0 & 1/2 & -1 & 3/4 \\ 0 & 1 & 0 & 1/2 & -3 & 5/4 \\ 0 & 0 & 1 & 0 & 1 & -1/2 \end{array}\right] = \left[ I \, \vert \, B \right] \nonumber \nonumber\] By construction we have that the matrix \[ B = \left[\begin{array} {rrr} 1/2 & -1 & 3/4 \\ 1/2 & -3 & 5/4 \\ 0 & 1 & -1/2 \end{array}\right] = \frac14 \left[\begin{array} {rrr} 2 & -4 & 3 \\ 2 & -12 & 5 \\ 0 & 4 & -2 \end{array}\right] \nonumber \nonumber\] satisfies \[ AB = I. \nonumber \nonumber\] Let us check the product in the other order \[ BA = \frac14 \left[\begin{array} {rrr} 2 & -4 & 3 \\ 2 & -12 & 5 \\ 0 & 4 & -2 \end{array}\right] \left[\begin{array} {rrr} 1 & 1 & 4 \\ 1 & -1 & -1 \\ 2 & -2 & -4 \end{array}\right] = \frac14 \left[\begin{array} {rrr} 4 & 0 & 0 \\0 & 4 & 0 \\ 0 & 0 & 4 \end{array}\right] = I \nonumber \nonumber\] So indeed we can conclude \[ \left[\begin{array} {rrr} 1 & 1 & 4 \\ 1 & -1 & -1 \\ 2 & -2 & -4 \end{array}\right]^{-1} = \frac14 \left[\begin{array} {rrr} 2 & -4 & 3 \\ 2 & -12 & 5 \\ 0 & 4 & -2 \end{array}\right] . \nonumber \nonumber\]

Now was this just beginners' luck? It wasn't, as the next proposition shows

Proposition

 

A square matrix \(A \) is invertible if and only it has independent columns. In that case the inverse can be found by reducing the matrix \[ \left[ A \, \vert \, I \right] \nonumber \nonumber\] to the reduced echelon form \[ \left[ I \, \vert \, B \right], \nonumber \nonumber\] and then \[ B = A^{-1}. \nonumber \nonumber\]

Skip/Read the proof

Proof

We have already seen (Proposition 10) that an invertible matrix has independent columns, which implies that the reduced echelon form of \(A \) is indeed the identity matrix. And then it is clear that via row operations we get \[ \left[ A \, \vert \, I \right] \sim \quad . . . . . \quad \sim \left[ I \, \vert \, B \right], \nonumber \nonumber\] where the matrix \(B \) satisfies \(AB = I \). What we have to show is that \[ BA = I \nonumber \nonumber\] as well. To understand that this is indeed true, we recall (Definition Dfn:Matrixproduct:ElementaryMatrix) that row operations can be effectuated via multiplications with elementary matrices. Furthermore, since the matrix product is defined column by column, i.e. \[ MX = M\left[\begin{array}{cccc}\vect{x}_1 &\vect{x}_2 &\ldots &\vect{x}_p \end{array}\right] = \left[\begin{array}{cccc}M\vect{x}_1 &M\vect{x}_2 &\ldots &M\vect{x}_p \end{array}\right], \nonumber \nonumber\] we also have \[ E\left[ A_1 \quad \, \vert \, A_2 \quad \right] = \left[ EA_1 \quad \, \vert \, EA_2 \quad \right]. \nonumber \nonumber\] A series of \(k \) row operations can be mimicked by \(k \) multiplications with elementary matrices: \[ \begin{array}{ccl} \left[ A \, \vert \, I \right] &\sim& \left[ E_1A \, \vert \, E_1I \right] \sim \left[ E_2 E_1A \, \vert \, E_2E_1I \right] \sim \ldots \quad \sim \\ &\sim& \left[ E_k\cdots E_2 E_1A \, \vert \, E_k\cdots E_2E_1I \right] = \left[ I \, \vert \, B \right]. \end{array} \nonumber \nonumber\] So the matrix \(B \) that was found as the solution of the matrix equation \[ AX = I \nonumber \nonumber\] is the product of all the elementary matrices by which \(A \) is reduced to the identity matrix. Thus we have shown that indeed \[ BA = (E_k\cdots E_2 E_1)A = I. \nonumber \nonumber\]

Note

 

In the proof we in fact showed that for a square matrix \(A \): \[ \text{if } AB = I \quad \text{then} \quad BA = I. \nonumber \nonumber\] For non-square matrices this statement is not correct. See exercises in Grasple (#60136, #61170).

Note

If \(A \) is not invertible, then the outcome of the row reduction of \[ \left[ A \, \vert \, I \right] \nonumber \nonumber\] will also lead to the correct answer: as soon as it is clear that \(A \) cannot be row reduced to \(I \) we can conclude that \(A \) is not invertible.

To help understand the above exposition let us run through the whole procedure for a specific matrix .

Example

 

We want to compute the inverse of the matrix \[ A = \left[\begin{array} {rr} 1 & 4 \\ 2 & 6 \end{array}\right]. \nonumber \nonumber\] The short way: \[ \begin{array}{rcl} \left[\begin{array}{rrrr | r} 1 & 4 & 1 & 0 \\ 2 & 6 & 0 & 1 \end{array}\right]\quad \begin{array}{r} [R_{1}] \\ [R_{2} -2 R_{1}] \end{array} \!\!\! &\sim& \left[\begin{array}{rrrr | r} 1 & 4 & 1 & 0 \\ 0 & -2 & -2 & 1 \end{array}\right] \quad \begin{array}{r} [R_{1}+ 2 R_{2}] \\ [R_{2}] \end{array} \\ &\sim& \left[\begin{array}{rrrr | r} 1 & 0 & -3 & 2 \\ 0 & -2 & -2 & 1 \end{array}\right] \quad \begin{array}{r} [R_{1}] \\ [ -\frac12 R_{2}] \end{array} \sim \left[\begin{array}{rrrr | r} 1 & 0 & -3 & 2 \\ 0 & 1 & 1 & -\frac12 \end{array}\right] \end{array} \nonumber \nonumber\] So: \[ A^{-1} = \left[\begin{array} {rr} -3 & 2 \\ 1 & -\frac12 \end{array}\right] = \frac12 \left[\begin{array} {rr} -6 & 4 \\ 2 & -1 \end{array}\right]. \nonumber \nonumber\] End of story. To see how the proof of Proposition 21 works for this specific matrix, we will give a derivation using elementary matrices. First step: row replacement with entry on position (1,1) as a first pivot: \[ \left[\begin{array} {rr} 1 & 0 \\ -2 & 1 \end{array}\right] \left[\begin{array}{rrrr | r} 1 & 4 & 1 & 0 \\ 2 & 6 & 0 & 1 \end{array}\right] = \left[\begin{array}{rrrr | r} 1 & 4 & 1 & 0 \\ 0 & -2 & -2 & 1 \end{array}\right], \quad E_1 = \left[\begin{array}{rr} 1 & 0 \\ -2 & 1 \end{array}\right]. \nonumber \nonumber\] Second step: another row replacement, using the entry on position (2,2) as pivot: \[ \left[\begin{array} {rr} 1 & 2 \\ 0 & 1 \end{array}\right] \left[\begin{array}{rrrr | r} 1 & 4 & 1 & 0 \\ 0 & -2 & -2 & 1 \end{array}\right] = \left[\begin{array}{rrrr | r} 1 & 0 & -3 & 2 \\ 0 & -2 & -2 & 1 \end{array}\right], \quad E_2 = \left[\begin{array}{rr} 1 & 2 \\ 0 & 1 \end{array}\right]. \nonumber \nonumber\] Third step: the scaling of the second row: \[ \left[\begin{array} {rr} 1 & 0 \\ 0 & -\frac12 \end{array}\right] \left[\begin{array}{rrrr | r} 1 & 0 & -3 & 2 \\ 0 & -2 & -2 & 1 \end{array}\right] = \left[\begin{array}{rrrr | r} 1 & 0 & -3 & 2 \\ 0 & 1 & 1 & -\frac12 \end{array}\right],\quad E_3 = \left[\begin{array}{rr} 1 & 0 \\ 0 & -\frac12 \end{array}\right]. \nonumber \nonumber\] All in all \[ (E_3E_2E_1)A = \left( \left[\begin{array}{rr} 1 & 0 \\ 0 & -\frac12 \end{array}\right] \left[\begin{array}{rr} 1 & 2 \\ 0 & 1 \end{array}\right] \left[\begin{array}{rr} 1 & 0 \\ -2 & 1 \end{array}\right]\right) A = \left[\begin{array}{rr} -3 & 2 \\ 1 & -\frac12 \end{array}\right]A = I. \nonumber \nonumber\]

Characterizations of invertibility

 

In the previous subsections quite a few properties of invertible matrices came along, either explicitly or implicitly. For future reference we list them in a theorem. Recall: by definition a (square) matrix \(A \) is invertible (or: regular) if and only if there exists a matrix \(B \) for which \[ AB = BA= I. \nonumber \nonumber\]

Theorem

 

For an \(n\times n \) matrix \(A \), the following statements are equivalent. So, each of the following properties is a characterization of invertibility of a square matrix \(A \):

1. \(A \) is invertible;
2. there exists a matrix \(B \) for which \(AB = I \);
3. for each \(\vect{b}\in\mathbb{R}^n \) the linear system \(A\vect{x} = \vect{b} \) has a unique solution;
4. \(A \) is row equivalent to the identity matrix \(I_n \);
5. \(A \) has independent columns;
6. \(A \) can be written as a product of elementary matrices: \[ A = E_1E_2\cdots E_k. \nonumber \nonumber\]

Skip/Read the proof

Proof

It is a good exercise to find out where the evidence of each characterization is found, and wherever necessary to fill in the missing details.

There are many variations on Theorem 25. The following exercise contains a few.

Exercise

 

Show that invertibility of an \(n\times n \) matrix \(A \) is also equivalent to

1. There is a matrix \(B \) such that \(BA = I \) ;
2. \(A \) has independent rows;
3. each column of the matrix \(A \) is a pivot column;
4. the columns of \(A \) span the whole \(\mathbb{R}^n \). Again it may very well be that you have to resort to previous sections.