2.1: Matrix Operations

Learning Objectives

• Perform the matrix operations of matrix addition, scalar multiplication, transposition and matrix multiplication. Identify when these operations are not defined. Represent these operations in terms of the entries of a matrix.
• Prove algebraic properties for matrix addition, scalar multiplication, transposition, and matrix multiplication. Apply these properties to manipulate an algebraic expression involving matrices.

You have now solved systems of equations by writing them in terms of an augmented matrix and then doing row operations on this augmented matrix. It turns out that matrices are important not only for systems of equations but also in many applications.

Recall that a matrix is a rectangular array of numbers. Several of them are referred to as matrices. For example, here is a matrix.

$$\left[ \begin{array}{rrrr} 1 & 2 & 3 & 4 \\ 5 & 2 & 8 & 7 \\ 6 & -9 & 1 & 2 \end{array} \right] \label{matrix}$$

Recall that the size or dimension of a matrix is defined as $m\times n$ where $m$ is the number of rows and $n$ is the number of columns. The above matrix is a $3\times 4$ matrix because there are three rows and four columns. You can remember the columns are like columns in a Greek temple. They stand upright while the rows lay flat like rows made by a tractor in a plowed field.

When specifying the size of a matrix, you always list the number of rows before the number of columns. You might remember that you always list the rows before the columns by using the phrase Rowman Catholic.

Consider the following definition.

Definition $\PageIndex{1}$: Square Matrix

A matrix $A$ which has size $n \times n$ is called a square matrix . In other words, $A$ is a square matrix if it has the same number of rows and columns.

There is some notation specific to matrices which we now introduce. We denote the columns of a matrix $A$ by $A_{j}$ as follows

$$A = \left[ \begin{array}{rrrr} A_{1} & A_{2} & \cdots & A_{n} \end{array} \right]\nonumber$$ Therefore, $A_{j}$ is the $j^{th}$ column of $A$, when counted from left to right.

The individual elements of the matrix are called entries or components of $A$. Elements of the matrix are identified according to their position. The $\mathbf{\left( i, j \right)}$-entry of a matrix is the entry in the $i^{th}$ row and $j^{th}$ column. For example, in the matrix $\eqref{matrix}$ above, $8$ is in position $\left(2,3 \right)$ (and is called the $\left(2,3 \right)$-entry) because it is in the second row and the third column.

In order to remember which matrix we are speaking of, we will denote the entry in the $i^{th}$ row and the $j^{th}$ column of matrix $A$ by $a_{ij}$. Then, we can write $A$ in terms of its entries, as $A= \left[ a_{ij} \right]$. Using this notation on the matrix in $\eqref{matrix}$, $a_{23}=8, a_{32}=-9, a_{12}=2,$ etc.

There are various operations which are done on matrices of appropriate sizes. Matrices can be added to and subtracted from other matrices, multiplied by a scalar, and multiplied by other matrices. We will never divide a matrix by another matrix, but we will see later how matrix inverses play a similar role.

In doing arithmetic with matrices, we often define the action by what happens in terms of the entries (or components) of the matrices. Before looking at these operations in depth, consider a few general definitions.

Definition $\PageIndex{2}$: The Zero Matrix

The $m\times n$ zero matrix is the $m\times n$ matrix having every entry equal to zero. It is denoted by $0.$

One possible zero matrix is shown in the following example.

Example $\PageIndex{1}$: The Zero Matrix

The $2\times 3$ zero matrix is $0= \left[ \begin{array}{ccc} 0 & 0 & 0 \\ 0 & 0 & 0 \end{array} \right]$.

Note there is a $2\times 3$ zero matrix, a $3\times 4$ zero matrix, etc. In fact there is a zero matrix for every size!

Definition $\PageIndex{3}$: Equality of Matrices

Let $A$ and $B$ be two $m\times n$ matrices. Then $A=B$ means that for $A=[a_{ij}]$ and $B=[b_{ij}]$, $a_{ij}=b_{ij}$ for all $1\leq i\leq m$ and $1\leq j\leq n$.

In other words, two matrices are equal exactly when they are the same size and the corresponding entries are identical. Thus $$\left[ \begin{array}{rr} 0 & 0 \\ 0 & 0 \\ 0 & 0 \end{array} \right] \neq \left[ \begin{array}{rr} 0 & 0 \\ 0 & 0 \end{array} \right]\nonumber$$ because they are different sizes. Also, $$\left[ \begin{array}{rr} 0 & 1 \\ 3 & 2 \end{array} \right] \neq \left[ \begin{array}{rr} 1 & 0 \\ 2 & 3 \end{array} \right]\nonumber$$ because, although they are the same size, their corresponding entries are not identical.

In the following section, we explore addition of matrices.

Addition of Matrices

When adding matrices, all matrices in the sum need have the same size. For example, $$\left[ \begin{array}{rr} 1 & 2 \\ 3 & 4 \\ 5 & 2 \end{array} \right]\nonumber$$ and $$\left[ \begin{array}{rrr} -1 & 4 & 8\\ 2 & 8 & 5 \end{array} \right]\nonumber$$ cannot be added, as one has size $3 \times 2$ while the other has size $2 \times 3$.

However, the addition $$\left[ \begin{array}{rrr} 4 & 6 & 3\\ 5 & 0 & 4\\ 11 & -2 & 3 \end{array} \right] + \left[ \begin{array}{rrr} 0 & 5 & 0 \\ 4 & -4 & 14 \\ 1 & 2 & 6 \end{array} \right]\nonumber$$ is possible.

The formal definition is as follows.

Definition $\PageIndex{4}$: Addition of Matrices

Let $A=\left[ a_{ij}\right]$ and $B=\left[ b_{ij}\right]$ be two $m\times n$ matrices. Then $A+B=C$ where $C$ is the $m \times n$ matrix $C=\left[ c_{ij}\right]$ defined by $$c_{ij}=a_{ij}+b_{ij}\nonumber$$

This definition tells us that when adding matrices, we simply add corresponding entries of the matrices. This is demonstrated in the next example.

Example $\PageIndex{2}$: Addition of Matrices of Same Size

Add the following matrices, if possible. $$A = \left[ \begin{array}{ccc} 1 & 2 & 3 \\ 1 & 0 & 4 \end{array} \right], B = \left[ \begin{array}{rrr} 5 & 2 & 3 \\ -6 & 2 & 1 \end{array} \right]\nonumber$$

Solution

Notice that both $A$ and $B$ are of size $2 \times 3$. Since $A$ and $B$ are of the same size, the addition is possible. Using Definition $\PageIndex{4}$, the addition is done as follows. $$A + B = \left[ \begin{array}{rrr} 1 & 2 & 3 \\ 1 & 0 & 4 \end{array} \right] + \left[ \begin{array}{rrr} 5 & 2 & 3 \\ -6 & 2 & 1 \end{array} \right] = \left[ \begin{array}{rrr} 1+5 & 2+2 & 3+3 \\ 1+ -6 & 0+2 & 4+1 \end{array} \right] = \left[ \begin{array}{rrr} 6 & 4 & 6 \\ -5 & 2 & 5 \end{array} \right]\nonumber$$

Addition of matrices obeys very much the same properties as normal addition with numbers. Note that when we write for example $A+B$ then we assume that both matrices are of equal size so that the operation is indeed possible.

Proposition $\PageIndex{1}$: Properties of Matrix Addition

Let $A,B$ and $C$ be matrices. Then, the following properties hold.

• Commutative Law of Addition $$A+B=B+A \label{mat1}$$
• Associative Law of Addition $$\left( A+B\right) +C=A+\left( B+C\right) \label{mat2}$$
• Existence of an Additive Identity $$\begin{array}{c} \mbox{There exists a zero matrix 0 such that}\\ A+0=A \label{mat3} \end{array}$$
• Existence of an Additive Inverse $$\begin{array}{c} \mbox{There exists a matrix $-A$ such that} \\ A+\left( -A\right) =0 \label{mat4} \end{array}$$

Proof

Consider the Commutative Law of Addition given in $\eqref{mat1}$. Let $A,B,C,$ and $D$ be matrices such that $A+B=C$ and $B+A=D.$ We want to show that $D=C$. To do so, we will use the definition of matrix addition given in Definition $\PageIndex{4}$. Now, $$c_{ij}=a_{ij}+b_{ij}=b_{ij}+a_{ij}=d_{ij}\nonumber$$ Therefore, $C=D$ because the $ij^{th}$ entries are the same for all $i$ and $j$. Note that the conclusion follows from the commutative law of addition of numbers, which says that if $a$ and $b$ are two numbers, then $a+b = b+a$. The proof of the other results are similar, and are left as an exercise.

We call the zero matrix in $\eqref{mat3}$ the additive identity. Similarly, we call the matrix $-A$ in $\eqref{mat4}$ the additive inverse. $-A$ is defined to equal $\left( -1\right) A = [-a_{ij}].$ In other words, every entry of $A$ is multiplied by $-1$.

In the next section we will study scalar multiplication in more depth to understand what is meant by $\left( -1\right) A.$

Scalar Multiplication of Matrices

Recall that we use the word scalar when referring to numbers. Therefore, scalar multiplication of a matrix is the multiplication of a matrix by a number. To illustrate this concept, consider the following example in which a matrix is multiplied by the scalar $3$. $$3\left[ \begin{array}{rrrr} 1 & 2 & 3 & 4 \\ 5 & 2 & 8 & 7 \\ 6 & -9 & 1 & 2 \end{array} \right] = \left[ \begin{array}{rrrr} 3 & 6 & 9 & 12 \\ 15 & 6 & 24 & 21 \\ 18 & -27 & 3 & 6 \end{array} \right]\nonumber$$

The new matrix is obtained by multiplying every entry of the original matrix by the given scalar.

The formal definition of scalar multiplication is as follows.

Definition $\PageIndex{5}$: Scalar Multiplication of Matrices

If $A=\left[ a_{ij}\right]$ and $k$ is a scalar, then $kA=\left[ ka_{ij}\right] .$

Consider the following example.

Example $\PageIndex{3}$: Effect of Multiplication by a Scalar

Find the result of multiplying the following matrix $A$ by $7$. $$A=\left[ \begin{array}{rr} 2 & 0 \\ 1 & -4 \end{array} \right]\nonumber$$

Solution

By Definition $\PageIndex{5}$, we multiply each element of $A$ by $7$. Therefore, $$7A = 7\left[ \begin{array}{rr} 2 & 0 \\ 1 & -4 \end{array} \right] = \left[ \begin{array}{rr} 7(2) & 7(0) \\ 7(1) & 7(-4) \end{array} \right] = \left[ \begin{array}{rr} 14 & 0 \\ 7 & -28 \end{array} \right]\nonumber$$

Similarly to addition of matrices, there are several properties of scalar multiplication which hold.

Proposition $\PageIndex{2}$: Properties of Scalar Multiplication

Let $A, B$ be matrices, and $k, p$ be scalars. Then, the following properties hold.

• Distributive Law over Matrix Addition $$k \left( A+B\right) =k A+ kB\nonumber$$
• Distributive Law over Scalar Addition $$\left( k +p \right) A= k A+p A\nonumber$$
• Associative Law for Scalar Multiplication $$k \left( p A\right) = \left( k p \right) A\nonumber$$
• Rule for Multiplication by $1$ $$1A=A\nonumber$$

Proof

The proof of this proposition is similar to the proof of Proposition $\PageIndex{1}$ and is left an exercise to the reader.

Matrix Multiplication

The next important matrix operation we will explore is multiplication of matrices. The operation of matrix multiplication is one of the most important and useful of the matrix operations. Throughout this section, we will also demonstrate how matrix multiplication relates to linear systems of equations.

First, we provide a formal definition of row and column vectors.

Definition $\PageIndex{6}$: Row and Column Vectors

Matrices of size $n\times 1$ or $1\times n$ are called vectors. If $X$ is such a matrix, then we write $x_{i}$ to denote the entry of $X$ in the $i^{th}$ row of a column matrix, or the $i^{th}$ column of a row matrix.

The $n\times 1$ matrix $$X=\left[ \begin{array}{c} x_{1} \\ \vdots \\ x_{n} \end{array} \right]\nonumber$$ is called a column vector. The $1\times n$ matrix $$X = \left[ \begin{array}{ccc} x_{1} & \cdots & x_{n} \end{array} \right]\nonumber$$ is called a row vector.

We may simply use the term vector throughout this text to refer to either a column or row vector. If we do so, the context will make it clear which we are referring to.

A linear combination of vectors is a sum consisting of those vectors multiplied by scalars. For example, $$\left[ \begin{array}{r} 50 \\ 122 \end{array} \right] = 7\left[ \begin{array}{r} 1 \\ 4 \end{array} \right] +8\left[ \begin{array}{r} 2 \\ 5 \end{array} \right] +9\left[ \begin{array}{r} 3 \\ 6 \end{array} \right]\nonumber$$ is a linear combination of three vectors.

It turns out that we can express any system of linear equations as a linear combination of vectors. In fact, the vectors that we will use are just the columns of the corresponding augmented matrix!

Definition $\PageIndex{7}$: The Vector Form of a System of Linear Equations

Suppose we have a system of equations given by $$\begin{array}{c} a_{11}x_{1}+\cdots +a_{1n}x_{n}=b_{1} \\ \vdots \\ a_{m1}x_{1}+\cdots +a_{mn}x_{n}=b_{m} \end{array}\nonumber$$ We can express this system in vector form which is as follows: $$x_1 \left[ \begin{array}{c} a_{11}\\ a_{21}\\ \vdots \\ a_{m1} \end{array} \right] + x_2 \left[ \begin{array}{c} a_{12}\\ a_{22}\\ \vdots \\ a_{m2} \end{array} \right] + \cdots + x_n \left[ \begin{array}{c} a_{1n}\\ a_{2n}\\ \vdots \\ a_{mn} \end{array} \right] = \left[ \begin{array}{c} b_1\\ b_2\\ \vdots \\ b_m \end{array} \right]\nonumber$$

Notice that each vector used here is one column from the corresponding augmented matrix. There is one vector for each variable in the system, along with the constant vector.

The first important form of matrix multiplication is multiplying a matrix by a vector. Consider the product given by $$\left[ \begin{array}{rrr} 1 & 2 & 3 \\ 4 & 5 & 6 \end{array} \right] \left[ \begin{array}{r} 7 \\ 8 \\ 9 \end{array} \right]\nonumber$$ We will soon see that this equals $$7\left[ \begin{array}{c} 1 \\ 4 \end{array} \right] +8\left[ \begin{array}{c} 2 \\ 5 \end{array} \right] +9\left[ \begin{array}{c} 3 \\ 6 \end{array} \right] =\left[ \begin{array}{c} 50 \\ 122 \end{array} \right]\nonumber$$

In general terms, $$\begin{aligned} \left[ \begin{array}{ccc} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \end{array} \right] \left[ \begin{array}{c} x_{1} \\ x_{2} \\ x_{3} \end{array} \right] &= \ x_{1}\left[ \begin{array}{c} a_{11} \\ a_{21} \end{array} \right] +x_{2}\left[ \begin{array}{c} a_{12} \\ a_{22} \end{array} \right] +x_{3}\left[ \begin{array}{c} a_{13} \\ a_{23} \end{array} \right] \\ &=\left[ \begin{array}{c} a_{11}x_{1}+a_{12}x_{2}+a_{13}x_{3} \\ a_{21}x_{1}+a_{22}x_{2}+a_{23}x_{3} \end{array} \right] \end{aligned}$$ Thus you take $x_{1}$ times the first column, add to $x_{2}$ times the second column, and finally $x_{3}$ times the third column. The above sum is a linear combination of the columns of the matrix. When you multiply a matrix on the left by a vector on the right, the numbers making up the vector are just the scalars to be used in the linear combination of the columns as illustrated above.

Here is the formal definition of how to multiply an $m\times n$ matrix by an $n\times 1$ column vector.

Definition $\PageIndex{8}$: Multiplication of Vector by Matrix

Let $A=\left[ a_{ij} \right]$ be an $m\times n$ matrix and let $X$ be an $n\times 1$ matrix given by $$A=\left[ A_{1} \cdots A_{n}\right], X = \left[ \begin{array}{r} x_{1} \\ \vdots \\ x_{n} \end{array} \right]\nonumber$$

Then the product $AX$ is the $m\times 1$ column vector which equals the following linear combination of the columns of $A$: $$x_{1}A_{1}+x_{2}A_{2}+\cdots +x_{n}A_{n} = \sum_{j=1}^{n}x_{j}A_{j}\nonumber$$

If we write the columns of $A$ in terms of their entries, they are of the form $$A_{j} = \left[ \begin{array}{c} a_{1j} \\ a_{2j} \\ \vdots \\ a_{mj} \end{array} \right]\nonumber$$ Then, we can write the product $AX$ as $$AX = x_{1}\left[ \begin{array}{c} a_{11} \\ a_{21} \\ \vdots \\ a_{m1} \end{array} \right] + x_{2}\left[ \begin{array}{c} a_{12} \\ a_{22} \\ \vdots \\ a_{m2} \end{array} \right] +\cdots + x_{n}\left[ \begin{array}{c} a_{1n} \\ a_{2n} \\ \vdots \\ a_{mn} \end{array} \right]\nonumber$$

Note that multiplication of an $m \times n$ matrix and an $n \times 1$ vector produces an $m \times 1$ vector.

Here is an example.

Example $\PageIndex{4}$: A Vector Multiplied by a Matrix

Compute the product $AX$ for $$A = \left[ \begin{array}{rrrr} 1 & 2 & 1 & 3 \\ 0 & 2 & 1 & -2 \\ 2 & 1 & 4 & 1 \end{array} \right], X = \left[ \begin{array}{r} 1 \\ 2 \\ 0 \\ 1 \end{array} \right]\nonumber$$

Solution

We will use Definition $\PageIndex{3}$ to compute the product. Therefore, we compute the product $AX$ as follows. $$\begin{aligned} & 1\left[ \begin{array}{r} 1 \\ 0 \\ 2 \end{array} \right] + 2\left[ \begin{array}{r} 2 \\ 2 \\ 1 \end{array} \right] + 0\left[ \begin{array}{r} 1 \\ 1 \\ 4 \end{array} \right] + 1 \left[ \begin{array}{r} 3 \\ -2\\ 1 \end{array} \right] \\ &= \left[ \begin{array}{r} 1 \\ 0 \\ 2 \end{array} \right] + \left[ \begin{array}{r} 4 \\ 4 \\ 2 \end{array} \right] + \left[ \begin{array}{r} 0 \\ 0 \\ 0 \end{array} \right] + \left[ \begin{array}{r} 3 \\ -2\\ 1 \end{array} \right] \\ &= \left[ \begin{array}{r} 8 \\ 2 \\ 5 \end{array} \right]\end{aligned}$$

Using the above operation, we can also write a system of linear equations in matrix form. In this form, we express the system as a matrix multiplied by a vector. Consider the following definition.

Definition $\PageIndex{9}$: The Matrix Form of a System of Linear Equations

Suppose we have a system of equations given by $$\begin{array}{c} a_{11}x_{1}+\cdots +a_{1n}x_{n}=b_{1} \\ a_{21}x_{1}+ \cdots + a_{2n}x_{n} = b_{2} \\ \vdots \\ a_{m1}x_{1}+\cdots +a_{mn}x_{n}=b_{m} \end{array}\nonumber$$ Then we can express this system in matrix form as follows. $$\left[ \begin{array}{cccc} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \end{array} \right] \left[ \begin{array}{c} x_{1} \\ x_{2} \\ \vdots \\ x_{n} \end{array} \right] = \left[ \begin{array}{c} b_{1}\\ b_{2}\\ \vdots \\ b_{m} \end{array} \right]\nonumber$$

The expression $AX=B$ is also known as the Matrix Form of the corresponding system of linear equations. The matrix $A$ is simply the coefficient matrix of the system, the vector $X$ is the column vector constructed from the variables of the system, and finally the vector $B$ is the column vector constructed from the constants of the system. It is important to note that any system of linear equations can be written in this form.

Notice that if we write a homogeneous system of equations in matrix form, it would have the form $AX=0$, for the zero vector $0$.

You can see from this definition that a vector $$X = \left[ \begin{array}{c} x_{1} \\ x_{2} \\ \vdots \\ x_{n} \end{array} \right]\nonumber$$ will satisfy the equation $AX=B$ only when the entries $x_{1}, x_{2}, \cdots, x_{n}$ of the vector $X$ are solutions to the original system.

Now that we have examined how to multiply a matrix by a vector, we wish to consider the case where we multiply two matrices of more general sizes, although these sizes still need to be appropriate as we will see. For example, in Example $\PageIndex{1}$, we multiplied a $3 \times 4$ matrix by a $4 \times 1$ vector. We want to investigate how to multiply other sizes of matrices.

We have not yet given any conditions on when matrix multiplication is possible! For matrices $A$ and $B$, in order to form the product $AB$, the number of columns of $A$ must equal the number of rows of $B.$ Consider a product $AB$ where $A$ has size $m\times n$ and $B$ has size $n \times p$. Then, the product in terms of size of matrices is given by $$(m\times\overset{\text{these must match!}}{\widehat{n)\;(n}\times p})=m\times p\nonumber$$

Note the two outside numbers give the size of the product. One of the most important rules regarding matrix multiplication is the following. If the two middle numbers don’t match, you can’t multiply the matrices!

When the number of columns of $A$ equals the number of rows of $B$ the two matrices are said to be conformable and the product $AB$ is obtained as follows.

Definition $\PageIndex{10}$: Multiplication of Two Matrices

Let $A$ be an $m\times n$ matrix and let $B$ be an $n\times p$ matrix of the form $$B=\left[ B_{1} \cdots B_{p}\right]\nonumber$$ where $B_{1},...,B_{p}$ are the $n\times 1$ columns of $B$. Then the $m\times p$ matrix $AB$ is defined as follows: $$AB = A \left[ B_{1} \cdots B_{p}\right] = \left[ (A B)_{1} \cdots (AB)_{p}\right]\nonumber$$ where $(AB)_{k}$ is an $m\times 1$ matrix or column vector which gives the $k^{th}$ column of $AB$.

Consider the following example.

Example $\PageIndex{5}$: Multiplying Two Matrices

Find $AB$ if possible. $$A = \left[ \begin{array}{rrr} 1 & 2 & 1 \\ 0 & 2 & 1 \end{array} \right], B = \left[ \begin{array}{rrr} 1 & 2 & 0 \\ 0 & 3 & 1 \\ -2 & 1 & 1 \end{array} \right]\nonumber$$

Solution

The first thing you need to verify when calculating a product is whether the multiplication is possible. The first matrix has size $2\times 3$ and the second matrix has size $3\times 3$. The inside numbers are equal, so $A$ and $B$ are conformable matrices. According to the above discussion $AB$ will be a $2\times 3$ matrix. Definition $\PageIndex{5}$ gives us a way to calculate each column of $AB$, as follows.

$$\left[ \overset{ \text{First column}}{\overbrace{\left[ \begin{array}{rrr} 1 & 2 & 1 \\ 0 & 2 & 1 \end{array} \right] \left[ \begin{array}{r} 1 \\ 0 \\ -2 \end{array} \right] }},\overset{\text{Second column}}{\overbrace{\left[ \begin{array}{rrr} 1 & 2 & 1 \\ 0 & 2 & 1 \end{array} \right] \left[ \begin{array}{r} 2 \\ 3 \\ 1 \end{array} \right] }},\overset{\text{Third column}}{\overbrace{\left[ \begin{array}{rrr} 1 & 2 & 1 \\ 0 & 2 & 1 \end{array} \right] \left[ \begin{array}{r} 0 \\ 1 \\ 1 \end{array} \right] }}\right]\nonumber$$ You know how to multiply a matrix times a vector, using Definition $\PageIndex{3}$ for each of the three columns. Thus $$\left[ \begin{array}{rrr} 1 & 2 & 1 \\ 0 & 2 & 1 \end{array} \right] \left[ \begin{array}{rrr} 1 & 2 & 0 \\ 0 & 3 & 1 \\ -2 & 1 & 1 \end{array} \right] = \ \left[ \begin{array}{rrr} -1 & 9 & 3 \\ -2 & 7 & 3 \end{array} \right]\nonumber$$

Since vectors are simply $n \times 1$ or $1 \times m$ matrices, we can also multiply a vector by another vector.

Example $\PageIndex{6}$: Vector Times Vector Multiplication

Multiply if possible $\left[ \begin{array}{r} 1 \\ 2 \\ 1 \end{array} \right] \left[ \begin{array}{rrrr} 1 & 2 & 1 & 0 \end{array} \right] .$

Solution

In this case we are multiplying a matrix of size $3 \times 1$ by a matrix of size $1 \times 4.$ The inside numbers match so the product is defined. Note that the product will be a matrix of size $3 \times 4$. Using Definition $\PageIndex{5}$, we can compute this product as follows $\:$ $$\left[ \begin{array}{r} 1 \\ 2 \\ 1 \end{array} \right] \left[ \begin{array}{rrrr} 1 & 2 & 1 & 0 \end{array} \right] = \left[ \overset{ \text{First column}}{\overbrace{\left[ \begin{array}{r} 1 \\ 2 \\ 1 \end{array} \right] \left[ \begin{array}{r} 1 \end{array} \right] }},\overset{\text{Second column}}{\overbrace{\left[ \begin{array}{r} 1 \\ 2\\ 1 \end{array} \right] \left[ \begin{array}{r} 2 \end{array} \right] }},\overset{\text{Third column}}{\overbrace{\left[ \begin{array}{r} 1 \\ 2 \\ 1 \end{array} \right] \left[ \begin{array}{r} 1 \end{array} \right] }}, \overset {\text{Fourth column}}{\overbrace{\left[ \begin{array}{r} 1\\ 2\\ 1 \end{array} \right] \left[ \begin{array}{r} 0 \end{array} \right]}} \right]\nonumber$$

You can use Definition $\PageIndex{3}$ to verify that this product is $$\left[ \begin{array}{cccc} 1 & 2 & 1 & 0 \\ 2 & 4 & 2 & 0 \\ 1 & 2 & 1 & 0 \end{array} \right]\nonumber$$

Example $\PageIndex{7}$: A Multiplication Which is Not Defined

Find $BA$ if possible. $$B = \left[ \begin{array}{ccc} 1 & 2 & 0 \\ 0 & 3 & 1 \\ -2 & 1 & 1 \end{array} \right], A = \left[ \begin{array}{ccc} 1 & 2 & 1 \\ 0 & 2 & 1 \end{array} \right]\nonumber$$

Solution

First check if it is possible. This product is of the form $\left( 3\times 3\right) \left( 2\times 3\right) .$ The inside numbers do not match and so you can’t do this multiplication.

In this case, we say that the multiplication is not defined. Notice that these are the same matrices which we used in Example $\PageIndex{2}$. In this example, we tried to calculate $BA$ instead of $AB$. This demonstrates another property of matrix multiplication. While the product $AB$ maybe be defined, we cannot assume that the product $BA$ will be possible. Therefore, it is important to always check that the product is defined before carrying out any calculations.

Earlier, we defined the zero matrix $0$ to be the matrix (of appropriate size) containing zeros in all entries. Consider the following example for multiplication by the zero matrix.

Example $\PageIndex{8}$: Multiplication by the Zero Matrix

Compute the product $A0$ for the matrix $$A= \left[ \begin{array}{rr} 1 & 2 \\ 3 & 4 \end{array} \right]\nonumber$$ and the $2 \times 2$ zero matrix given by $$0= \left[ \begin{array}{rr} 0 & 0 \\ 0 & 0 \end{array} \right]\nonumber$$

Solution

In this product, we compute $$\left[ \begin{array}{rr} 1 & 2 \\ 3 & 4 \end{array} \right] \left[ \begin{array}{rr} 0 & 0 \\ 0 & 0 \end{array} \right] = \left[ \begin{array}{rr} 0 & 0 \\ 0 & 0 \end{array} \right]\nonumber$$

Hence, $A0=0$.

Notice that we could also multiply $A$ by the $2 \times 1$ zero vector given by $\left[ \begin{array}{r} 0 \\ 0 \end{array} \right]$. The result would be the $2 \times 1$ zero vector. Therefore, it is always the case that $A0=0$, for an appropriately sized zero matrix or vector.

The ij-th Entry of a Matrix Product

Up to this point, we have used the entries of a matrix to describe the action of matrix addition and scalar multiplication. We can also study matrix multiplication using the entries of matrices.

What is the $ij^{th}$ entry of $AB?$ It is the entry in the $i^{th}$ row and the $j^{th}$ column of the product $AB$.

Now if $A$ is $m \times n$ and $B$ is $n \times p$, then we know that the product $AB$ has the form $$\left[ \begin{array}{cccc} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \end{array} \right] \left[ \begin{array}{cccccc} b_{11} & b_{12} & \cdots & b_{1j} & \cdots & b_{1p} \\ b_{21} & b_{22} & \cdots & b_{2j} & \cdots & b_{2p} \\ \vdots & \vdots & & \vdots & & \vdots\\ b_{n1} & b_{n2} & \cdots & b_{nj} & \cdots & b_{np} \end{array} \right]\nonumber$$

The $j^{th}$ column of $AB$ is of the form $$\left[ \begin{array}{cccc} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \end{array} \right] \left[ \begin{array}{c} b_{1j} \\ b_{2j} \\ \vdots \\ b_{nj} \end{array} \right]\nonumber$$ which is an $m\times 1$ column vector. It is calculated by $$b_{1j} \left[ \begin{array}{c} a_{11} \\ a_{21} \\ \vdots \\ a_{m1} \end{array} \right] + b_{2j}\left[ \begin{array}{c} a_{12} \\ a_{22} \\ \vdots \\ a_{m2} \end{array} \right] +\cdots + b_{nj}\left[ \begin{array}{c} a_{1n} \\ a_{2n} \\ \vdots \\ a_{mn} \end{array} \right]\nonumber$$

Therefore, the $ij^{th}$ entry is the entry in row $i$ of this vector. This is computed by $$a_{i1}b_{1j}+a_{i2}b_{2j}+\cdots + a_{in}b_{nj}=\sum_{k=1}^{n}a_{ik}b_{kj}\nonumber$$

The following is the formal definition for the $ij^{th}$ entry of a product of matrices.

Definition $\PageIndex{11}$: The $ij^{th}$ Entry of a Product

Let $A=\left[ a_{ij}\right]$ be an $m\times n$ matrix and let $B=\left[ b_{ij}\right]$ be an $n\times p$ matrix. Then $AB$ is an $m\times p$ matrix and the $\left( i, j \right)$-entry of $AB$ is defined as $$(AB)_{ij}=\sum_{k=1}^{n}a_{ik}b_{kj} \nonumber$$ Another way to write this is $$(AB)_{ij}=\left[ \begin{array}{cccc} a_{i1} & a_{i2} & \cdots & a_{in} \end{array} \right] \left[ \begin{array}{c} b_{1j} \\ b_{2j} \\ \vdots \\ b_{nj} \end{array} \right] = a_{i1}b_{1j} + a_{i2}b_{2j} + \cdots + a_{in}b_{nj}\nonumber$$

In other words, to find the $\left( i, j \right)$-entry of the product $AB$, or $(AB)_{ij}$, you multiply the $i^{th}$ row of $A,$ on the left by the $j^{th}$ column of $B$. To express $AB$ in terms of its entries, we write $AB = \left[ (AB)_{ij} \right]$.

Consider the following example.

Example $\PageIndex{9}$: The Entries of a Product

Compute $AB$ if possible. If it is, find the $\left( 3,2 \right)$-entry of $AB$ using Definition $\PageIndex{1}$. $$A = \left[ \begin{array}{cc} 1 & 2 \\ 3 & 1 \\ 2 & 6 \end{array} \right], B = \left[ \begin{array}{ccc} 2 & 3 & 1 \\ 7 & 6 & 2 \end{array} \right]\nonumber$$

Solution

First check if the product is possible. It is of the form $\left( 3\times 2\right) \left( 2\times 3\right)$ and since the inside numbers match, it is possible to do the multiplication. The result should be a $3\times 3$ matrix. We can first compute $AB$: $$\left[ \left[ \begin{array}{rr} 1 & 2 \\ 3 & 1 \\ 2 & 6 \end{array} \right] \left[ \begin{array}{r} 2 \\ 7 \end{array} \right] ,\left[ \begin{array}{rr} 1 & 2 \\ 3 & 1 \\ 2 & 6 \end{array} \right] \left[ \begin{array}{r} 3 \\ 6 \end{array} \right] ,\left[ \begin{array}{rr} 1 & 2 \\ 3 & 1 \\ 2 & 6 \end{array} \right] \left[ \begin{array}{r} 1 \\ 2 \end{array} \right] \right]\nonumber$$ where the commas separate the columns in the resulting product. Thus the above product equals $$\left[ \begin{array}{rrr} 16 & 15 & 5 \\ 13 & 15 & 5 \\ 46 & 42 & 14 \end{array} \right]\nonumber$$ which is a $3\times 3$ matrix as desired. Thus, the $\left( 3,2 \right)$-entry equals 42.

Now using Definition $\PageIndex{1}$, we can find that the $\left( 3,2 \right)$-entry equals $$\begin{aligned} \sum_{k=1}^{2}a_{3k}b_{k2} &=a_{31}b_{12}+a_{32}b_{22} \\ &=2\times 3+6\times 6=42\\\end{aligned}$$ Consulting our result for $AB$ above, this is correct!

You may wish to use this method to verify that the rest of the entries in $AB$ are correct.

Here is another example.

Example $\PageIndex{10}$: Finding the Entries of a Product

Determine if the product $AB$ is defined. If it is, find the $\left( 2, 1 \right)$-entry of the product. $$A= \left[ \begin{array}{rrr} 2 & 3 & 1 \\ 7 & 6 & 2 \\ 0 & 0 & 0 \end{array} \right], B =\left[ \begin{array}{rr} 1 & 2 \\ 3 & 1 \\ 2 & 6 \end{array} \right]\nonumber$$

Solution

This product is of the form $\left( 3\times 3\right) \left( 3\times 2\right)$. The middle numbers match so the matrices are conformable and it is possible to compute the product.

We want to find the $\left( 2, 1 \right)$-entry of $AB$, that is, the entry in the second row and first column of the product. We will use Definition $\PageIndex{1}$, which states $$(AB)_{ij}=\sum_{k=1}^{n}a_{ik}b_{kj}\nonumber$$ In this case, $n=3$, $i=2$ and $j=1$. Hence the $\left( 2, 1 \right)$-entry is found by computing $$(AB)_{21} = \sum_{k=1}^{3}a_{2k}b_{k1} = \left[ \begin{array}{ccc} a_{21} & a_{22} & a_{23} \end{array} \right] \left[ \begin{array}{c} b_{11} \\ b_{21} \\ b_{31} \end{array} \right]\nonumber$$ Substituting in the appropriate values, this product becomes $$\left[ \begin{array}{ccc} a_{21} & a_{22} & a_{23} \end{array} \right] \left[ \begin{array}{c} b_{11} \\ b_{21} \\ b_{31} \end{array} \right] = \left[ \begin{array}{ccc} 7 & 6 & 2 \end{array} \right] \left[ \begin{array}{c} 1 \\ 3 \\ 2 \end{array} \right] = 1 \times 7 + 3 \times 6 + 2 \times 2 = 29\nonumber$$

Hence, $(AB)_{21} = 29$.

You should take a moment to find a few other entries of $AB$. You can multiply the matrices to check that your answers are correct. The product $AB$ is given by $$AB = \left[ \begin{array}{cc} 13 & 13 \\ 29 & 32 \\ 0 & 0 \end{array} \right]\nonumber$$

Properties of Matrix Multiplication

As pointed out above, it is sometimes possible to multiply matrices in one order but not in the other order. However, even if both $AB$ and $BA$ are defined, they may not be equal.

Example $\PageIndex{11}$: Matrix Multiplication is Not Commutative

Compare the products $AB$ and $BA$, for matrices $A = \left[ \begin{array}{rr} 1 & 2 \\ 3 & 4 \end{array} \right], B= \left[ \begin{array}{rr} 0 & 1 \\ 1 & 0 \end{array} \right]$

Solution

First, notice that $A$ and $B$ are both of size $2 \times 2$. Therefore, both products $AB$ and $BA$ are defined. The first product, $AB$ is

$$AB = \left[ \begin{array}{rr} 1 & 2 \\ 3 & 4 \end{array} \right] \left[ \begin{array}{rr} 0 & 1 \\ 1 & 0 \end{array} \right] = \left[ \begin{array}{rr} 2 & 1 \\ 4 & 3 \end{array} \right] \nonumber$$

The second product, $BA$ is

$$\left[ \begin{array}{rr} 0 & 1 \\ 1 & 0 \end{array} \right] \left[ \begin{array}{rr} 1 & 2 \\ 3 & 4 \end{array} \right] = \left[ \begin{array}{rr} 3 & 4 \\ 1 & 2 \end{array} \right] \nonumber$$

Therefore, $AB \neq BA$.

This example illustrates that you cannot assume $AB=BA$ even when multiplication is defined in both orders. If for some matrices $A$ and $B$ it is true that $AB=BA$, then we say that $A$ and $B$ commute. This is one important property of matrix multiplication.

The following are other important properties of matrix multiplication. Notice that these properties hold only when the size of matrices are such that the products are defined.

Proposition $\PageIndex{3}$: Properties of Matrix Multiplication

The following hold for matrices $A,B,$ and $C$ and for scalars $r$ and $s$,

$$\begin{align} A\left( rB+sC\right) &= r\left( AB\right) +s\left( AC\right) \label{matrixproperties1} \\[4pt] \left( B+C\right) A &=BA+CA \label{matrixproperties2} \\[4pt] A\left( BC\right) &=\left( AB\right) C \label{matrixproperties3} \end{align}$$

Proof

First we will prove $\eqref{matrixproperties1}$. We will use Definition 2.3.1 and prove this statement using the $ij^{th}$ entries of a matrix. Therefore,

$$\begin{aligned} \left( A\left( rB+sC\right) \right) _{ij} &=\sum_{k}a_{ik}\left( rB+sC\right) _{kj} \\[4pt] &= \sum_{k}a_{ik}\left( rb_{kj}+sc_{kj}\right) \\[4pt] &=r\sum_{k}a_{ik}b_{kj}+s\sum_{k}a_{ik}c_{kj} \\[4pt] &=r\left( AB\right) _{ij}+s\left( AC\right) _{ij} \\[4pt] &=\left( r\left( AB\right) +s\left( AC\right) \right) _{ij} \end{aligned}$$

Thus $A\left( rB+sC\right) =r(AB)+s(AC)$ as claimed.

The proof of Equation $\eqref{matrixproperties2}$ follows the same pattern and is left as an exercise.

Statement Equation $\eqref{matrixproperties3}$ is the associative law of multiplication. Using Definition 2.3.1,

$$\begin{align*}\left( A\left( BC\right) \right) _{ij} &=\sum_{k}a_{ik}\left( BC\right) _{kj} \\[4pt] &=\sum_{k}a_{ik}\sum_{l}b_{kl}c_{lj} \\[4pt] &=\sum_{l}\left( AB\right) _{il}c_{lj}=\left( \left( AB\right) C\right) _{ij}. \end{align*}$$

This proves $\eqref{matrixproperties3}$.

The Transpose of a Matrix

Another important operation on matrices is that of taking the transpose. For a matrix $A$, we denote the transpose of $A$ by $A^T$. Before formally defining the transpose, we explore this operation on the following matrix.

$$\left[ \begin{array}{cc} 1 & 4 \\ 3 & 1 \\ 2 & 6 \end{array} \right] ^{T}= \ \ \left[ \begin{array}{ccc} 1 & 3 & 2 \\ 4 & 1 & 6 \end{array} \right] \nonumber$$

What happened? The first column became the first row and the second column became the second row. Thus the $3\times 2$ matrix became a $2\times 3$ matrix. The number $4$ was in the first row and the second column and it ended up in the second row and first column.

The definition of the transpose is as follows.

Definition $\PageIndex{12}$: The Transpose of a Matrix

Let $A$ be an $m\times n$ matrix. Then $A^{T}$, the transpose of $A$, denotes the $n\times m$ matrix given by

$$A^{T} = \left[ a _{ij}\right] ^{T}= \left[ a_{ji} \right]\nonumber$$

The $\left( i, j \right)$-entry of $A$ becomes the $\left( j,i \right)$-entry of $A^T$.

Consider the following example.

Example $\PageIndex{12}$: The Transpose of a Matrix

Calculate $A^T$ for the following matrix

$$A = \left[ \begin{array}{rrr} 1 & 2 & -6 \\ 3 & 5 & 4 \end{array} \right] \nonumber$$

Solution

By Definition $\PageIndex{1}$, we know that for $A = \left[ a_{ij} \right]$, $A^T = \left[ a_{ji} \right]$. In other words, we switch the row and column location of each entry. The $\left( 1, 2 \right)$-entry becomes the $\left( 2,1 \right)$-entry.

Thus, $$A^T = \left[ \begin{array}{rr} 1 & 3 \\ 2 & 5 \\ -6 & 4 \end{array} \right] \nonumber$$

Notice that $A$ is a $2 \times 3$ matrix, while $A^T$ is a $3 \times 2$ matrix.

The transpose of a matrix has the following important properties.

Proposition $\PageIndex{4}$: Properties of the Transpose of a Matrix

Let $A$ be an $m\times n$ matrix, $B$ an $n\times p$ matrix, and $r$ and $s$ scalars. Then

1. $$\left(A^{T}\right)^{T} = A\nonumber$$
2. $$\left( AB\right) ^{T}=B^{T}A^{T} \nonumber$$
3. $$\left( rA+ sB\right) ^{T}=rA^{T}+ sB^{T} \nonumber$$

Proof

First we prove 2. From Definition $\PageIndex{1}$,

$$\begin{aligned} \left(AB\right)^{T} &= \left[ (AB) _{ij} \right] ^{T}=\left[ (AB)_{ji} \right]=\sum_{k}a_{jk}b_{ki}= \sum_{k}b_{ki}a_{jk} \\[4pt] &= \sum_{k}\left[ b_{ik}\right]^{T}\left[ a_{kj}\right]^{T}=\left[ b_{ij}\right] ^{T} \left[ a_{ij}\right]^{T} = B^{T}A^{T} \end{aligned}$$

The proof of Formula 3 is left as an exercise.

The transpose of a matrix is related to other important topics. Consider the following definition.

Definition $\PageIndex{13}$: Symmetric and Skew Symmetric Matrices

An $n\times n$ matrix $A$ is said to be symmetric if $A=A^{T}.$ It is said to be skew symmetric if $A=-A^{T}.$

We will explore these definitions in the following examples.

Example $\PageIndex{13}$: Symmetric Matrices

Let

$$A=\left[ \begin{array}{rrr} 2 & 1 & 3 \\ 1 & 5 & -3 \\ 3 & -3 & 7 \end{array} \right] \nonumber$$

Use Definition $\PageIndex{2}$ to show that $A$ is symmetric.

Solution

By Definition $\PageIndex{2}$, we need to show that $A = A^T$. Now, using Definition $\PageIndex{1}$,

$$A^{T} = \left[ \begin{array}{rrr} 2 & 1 & 3 \\ 1 & 5 & -3 \\ 3 & -3 & 7 \end{array} \right]\nonumber$$

Hence, $A = A^{T}$, so $A$ is symmetric.

Below is a video example that goes over the the product of a matrix with its transpose.

Example $\PageIndex{14}$

Let $A=\left[ \begin{array}{rrr} 6 & -1 \\ -2 & 4 \\ 1 & 3 \end{array} \right]$. Find $A^TA$.

Solution

• Video Length: 4 minutes 46 seconds.
• Context: This video demonstrates using the transpose of a matrix in a calculation.

Proposition $\PageIndex{5}$

Let $A$ be an $m\times n$ matrix. Then

1. $A^T A$ is a symmetric matrix.
2. $A A^T$ is a symmetric matrix.

Proof

We prove statement 1. To show that $A^T A$ is symmetric, we must show that it is equal to its transpose. Notice that

$$(A^T A)^T = A^T (A^T)^T = A^T A$$

Since $A^T A)^T = A^T A$, $A^T A$ is symmetric.

Proof of statement 2 is similar.

Example $\PageIndex{15}$: A Skew Symmetric Matrix

Let

$$A=\left[ \begin{array}{rrr} 0 & 1 & 3 \\ -1 & 0 & 2 \\ -3 & -2 & 0 \end{array} \right] \nonumber$$

Show that $A$ is skew symmetric.

Solution

By Definition $\PageIndex{2}$,

$$A^{T} = \left[ \begin{array}{rrr} 0 & -1 & -3\\ 1 & 0 & -2\\ 3 & 2 & 0 \end{array} \right] \nonumber$$

You can see that each entry of $A^T$ is equal to $-1$ times the same entry of $A$. Hence, $A^{T} = - A$ and so by Definition $\PageIndex{2}$, $A$ is skew symmetric.