11.2: Matrix Multiply

Read Section 10.1 of the Stephen Boyd and Lieven Vandenberghe Applied Linear algebra book which covers Matrix Multiplication. Here is a quick review:

Two matrices \(A\) and \(B\) can be multiplied together if and only if their “inner dimensions” are the same, i.e. \(A\) is \(n \times d \) and \(B\) is \(d \times m\) (note that the columns of \(A\) and the rows of \(B\) are both \(d\)). Multiplication of these two matrices results in a third matrix \(C\) with the dimension of \(n \times m\). Note that \(C\) has the same first dimension as \(A\) and the same second dimension as \(B\). i.e \(n \times m\).

The (\(i\),\(j\)) element in \(C\) is the dot product of the \(i\)th row of \(A\) and the \(j\)th column of \(B\).

The \(i\)th row of \(A\) is:

\[ [ a_{i1}, a_{i2}, \dots , a_{id} ], \nonumber \]

and the \(j\)th column of \(B\) is:

\[\begin{split}
\left[
\begin{matrix}
b_{1j}\\
b_{2j}\\
\vdots \\
b_{dj}
\end{matrix}
\right]
\end{split} \nonumber \]

So, the dot product of these two vectors is:

\[c_{ij} = a_{i1}b_{1j} + a_{i2}b_{2j} + \dots + a_{id}b_{dj} \nonumber \]

Consider the simple \(2 \times 2\) example below:

\[\begin{split}
\left[
\begin{matrix}
a & b\\
c & d
\end{matrix}
\right]
\left[
\begin{matrix}
w & x\\
y & z
\end{matrix}
\right]
=
\left[
\begin{matrix}
aw+by & ax+bz\\
cw + dy & cx + dz
\end{matrix}
\right]
\end{split} \nonumber \]

Let’s do an example using numpy and show the results using sympy:

import numpy as np
import sympy as sym
sym.init_printing(use_unicode=True) # Trick to make matrixes look nice in jupyter

A = np.matrix([[1,1], [2,2]])
sym.Matrix(A)
# 'Run' this cell to see the output

B = np.matrix([[3,4], [3,4]])
sym.Matrix(B)
# 'Run' this cell to see the output

sym.Matrix(A*B)
# 'Run' this cell to see the output

Do This

Given two matrices; \(A\) and \(B\), show that order matters when doing a matrix multiply. That is, in general, \(AB \neq BA\). Show this with an example using two \(3 \times 3\) matrices and numpy.

# Put your code here.

Now consider the following set of linear equations:

\[ 3x_1-3x_2+9x_3 =~24 \nonumber \]

\[ 2x_1-2x_2+7x_3 =~17 \nonumber \]

\[ -x_1+2x_2-4x_3 = -11 \nonumber \]

We typically write this in the following form:

\[\begin{split}
\left[
\begin{matrix}
3 & -3 & 9\\
2 & -2 & 7 \\
-1 & 2 & -4
\end{matrix}
\right]
\left[
\begin{matrix}
x_1 \\
x_2 \\
x_3
\end{matrix}
\right]
=
\left[
\begin{matrix}
24\\
17 \\
-11
\end{matrix}
\right]
\end{split} \nonumber \]

Notice how doing the matrix multiplication results back into the original system of equations. If we rename the three matrices from above to \(A\), \(x\), and \(b\) (note \(x\) and \(b\) are lowercase because they are column vectors) then we get the main equation for this class, which is:

\[Ax=b \nonumber \]

Note the information about the equation doesn’t change when you change formats. For example, the equation format, the augmented format and the \(Ax=b\) format contain the same information. However, we use the different formats for different applications. Consider the numpy.linalg.solve function which assumes the format \(Ax=b\)

A = np.matrix([[3, -3,9], [2, -2, 7], [-1, 2, -4]])
sym.Matrix(A)
# 'Run' this cell to see the output

b = np.matrix([[24], [17], [-11]])
sym.Matrix(b)
# 'Run' this cell to see the output

#Calculate answer to x using numpy
x = np.linalg.solve(A,b)
sym.Matrix(x)
# 'Run' this cell to see the output

Question

What is the size of the matrix resulting from multiplying a \(10 \times 40\) matrix with a \(40 \times 3\) matrix?