Search

Text Color

Margin Size

Font Type

Enable Dyslexic Font

11.2: Matrix Multiply

Last updated

Sep 17, 2022
Save as PDF
- 11.1: Dot Product Review
- 11.3: Identity Matrix

$\newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} }$

$\newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}}$

$\newcommand{\id}{\mathrm{id}}$ $\newcommand{\Span}{\mathrm{span}}$

( \newcommand{\kernel}{\mathrm{null}\,}\) $\newcommand{\range}{\mathrm{range}\,}$

$\newcommand{\RealPart}{\mathrm{Re}}$ $\newcommand{\ImaginaryPart}{\mathrm{Im}}$

$\newcommand{\Argument}{\mathrm{Arg}}$ $\newcommand{\norm}[1]{\| #1 \|}$

$\newcommand{\inner}[2]{\langle #1, #2 \rangle}$

$\newcommand{\Span}{\mathrm{span}}$

$\newcommand{\id}{\mathrm{id}}$

$\newcommand{\Span}{\mathrm{span}}$

$\newcommand{\kernel}{\mathrm{null}\,}$

$\newcommand{\range}{\mathrm{range}\,}$

$\newcommand{\RealPart}{\mathrm{Re}}$

$\newcommand{\ImaginaryPart}{\mathrm{Im}}$

$\newcommand{\Argument}{\mathrm{Arg}}$

$\newcommand{\norm}[1]{\| #1 \|}$

$\newcommand{\inner}[2]{\langle #1, #2 \rangle}$

$\newcommand{\Span}{\mathrm{span}}$ $\newcommand{\AA}{\unicode[.8,0]{x212B}}$

$\newcommand{\vectorA}[1]{\vec{#1}} % arrow$

$\newcommand{\vectorAt}[1]{\vec{\text{#1}}} % arrow$

$\newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} }$

$\newcommand{\vectorC}[1]{\textbf{#1}}$

$\newcommand{\vectorD}[1]{\overrightarrow{#1}}$

$\newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}}$

$\newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}}$

$\newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} }$

$\newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}}$

$\newcommand{\avec}{\mathbf a}$

$\newcommand{\bvec}{\mathbf b}$

$\newcommand{\cvec}{\mathbf c}$

$\newcommand{\dvec}{\mathbf d}$

$\newcommand{\dtil}{\widetilde{\mathbf d}}$

$\newcommand{\evec}{\mathbf e}$

$\newcommand{\fvec}{\mathbf f}$

$\newcommand{\nvec}{\mathbf n}$

$\newcommand{\pvec}{\mathbf p}$

$\newcommand{\qvec}{\mathbf q}$

$\newcommand{\svec}{\mathbf s}$

$\newcommand{\tvec}{\mathbf t}$

$\newcommand{\uvec}{\mathbf u}$

$\newcommand{\vvec}{\mathbf v}$

$\newcommand{\wvec}{\mathbf w}$

$\newcommand{\xvec}{\mathbf x}$

$\newcommand{\yvec}{\mathbf y}$

$\newcommand{\zvec}{\mathbf z}$

$\newcommand{\rvec}{\mathbf r}$

$\newcommand{\mvec}{\mathbf m}$

$\newcommand{\zerovec}{\mathbf 0}$

$\newcommand{\onevec}{\mathbf 1}$

$\newcommand{\real}{\mathbb R}$

$\newcommand{\twovec}[2]{\left[\begin{array}{r}#1 \\ #2 \end{array}\right]}$

$\newcommand{\ctwovec}[2]{\left[\begin{array}{c}#1 \\ #2 \end{array}\right]}$

$\newcommand{\threevec}[3]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \end{array}\right]}$

$\newcommand{\cthreevec}[3]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \end{array}\right]}$

$\newcommand{\fourvec}[4]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}$

$\newcommand{\cfourvec}[4]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}$

$\newcommand{\fivevec}[5]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}$

$\newcommand{\cfivevec}[5]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}$

$\newcommand{\mattwo}[4]{\left[\begin{array}{rr}#1 \amp #2 \\ #3 \amp #4 \\ \end{array}\right]}$

$\newcommand{\laspan}[1]{\text{Span}\{#1\}}$

$\newcommand{\bcal}{\cal B}$

$\newcommand{\ccal}{\cal C}$

$\newcommand{\scal}{\cal S}$

$\newcommand{\wcal}{\cal W}$

$\newcommand{\ecal}{\cal E}$

$\newcommand{\coords}[2]{\left\{#1\right\}_{#2}}$

$\newcommand{\gray}[1]{\color{gray}{#1}}$

$\newcommand{\lgray}[1]{\color{lightgray}{#1}}$

$\newcommand{\rank}{\operatorname{rank}}$

$\newcommand{\row}{\text{Row}}$

$\newcommand{\col}{\text{Col}}$

$\renewcommand{\row}{\text{Row}}$

$\newcommand{\nul}{\text{Nul}}$

$\newcommand{\var}{\text{Var}}$

$\newcommand{\corr}{\text{corr}}$

$\newcommand{\len}[1]{\left|#1\right|}$

$\newcommand{\bbar}{\overline{\bvec}}$

$\newcommand{\bhat}{\widehat{\bvec}}$

$\newcommand{\bperp}{\bvec^\perp}$

$\newcommand{\xhat}{\widehat{\xvec}}$

$\newcommand{\vhat}{\widehat{\vvec}}$

$\newcommand{\uhat}{\widehat{\uvec}}$

$\newcommand{\what}{\widehat{\wvec}}$

$\newcommand{\Sighat}{\widehat{\Sigma}}$

$\newcommand{\lt}{<}$

$\newcommand{\gt}{>}$

$\newcommand{\amp}{&}$

$\definecolor{fillinmathshade}{gray}{0.9}$

Read Section 10.1 of the Stephen Boyd and Lieven Vandenberghe Applied Linear algebra book which covers Matrix Multiplication. Here is a quick review:

Two matrices $A$ and $B$ can be multiplied together if and only if their “inner dimensions” are the same, i.e. $A$ is $n \times d$ and $B$ is $d \times m$ (note that the columns of $A$ and the rows of $B$ are both $d$ ). Multiplication of these two matrices results in a third matrix $C$ with the dimension of $n \times m$ . Note that $C$ has the same first dimension as $A$ and the same second dimension as $B$ . i.e $n \times m$ .

The ( $i$ , $j$ ) element in $C$ is the dot product of the $i$ th row of $A$ and the $j$ th column of $B$ .

The $i$ th row of $A$ is:

$[ a_{i1}, a_{i2}, \dots , a_{id} ], \nonumber$

and the $j$ th column of $B$ is:

$\begin{split} \left[ \begin{matrix} b_{1j}\\ b_{2j}\\ \vdots \\ b_{dj} \end{matrix} \right] \end{split} \nonumber$

So, the dot product of these two vectors is:

$c_{ij} = a_{i1}b_{1j} + a_{i2}b_{2j} + \dots + a_{id}b_{dj} \nonumber$

Consider the simple $2 \times 2$ example below:

$\begin{split} \left[ \begin{matrix} a & b\\ c & d \end{matrix} \right] \left[ \begin{matrix} w & x\\ y & z \end{matrix} \right] = \left[ \begin{matrix} aw+by & ax+bz\\ cw + dy & cx + dz \end{matrix} \right] \end{split} \nonumber$

Let’s do an example using numpy and show the results using sympy:

If you have already signed in, please refresh the page.

import numpy as np
import sympy as sym
sym.init_printing(use_unicode=True) # Trick to make matrixes look nice in jupyter

import numpy as np
import sympy as sym
sym.init_printing(use_unicode=True) # Trick to make matrixes look nice in jupyter

If you have already signed in, please refresh the page.

A = np.matrix([[1,1], [2,2]])
sym.Matrix(A)
# 'Run' this cell to see the output

A = np.matrix([[1,1], [2,2]])
sym.Matrix(A)
# 'Run' this cell to see the output

If you have already signed in, please refresh the page.

B = np.matrix([[3,4], [3,4]])
sym.Matrix(B)
# 'Run' this cell to see the output

B = np.matrix([[3,4], [3,4]])
sym.Matrix(B)
# 'Run' this cell to see the output

If you have already signed in, please refresh the page.

sym.Matrix(A*B)
# 'Run' this cell to see the output

sym.Matrix(A*B)
# 'Run' this cell to see the output

Do This

Given two matrices; $A$ and $B$ , show that order matters when doing a matrix multiply. That is, in general, $AB \neq BA$ . Show this with an example using two $3 \times 3$ matrices and numpy.

If you have already signed in, please refresh the page.

# Put your code here.

# Put your code here.

Now consider the following set of linear equations:

$3x_1-3x_2+9x_3 =~24 \nonumber$

$2x_1-2x_2+7x_3 =~17 \nonumber$

$-x_1+2x_2-4x_3 = -11 \nonumber$

We typically write this in the following form:

$\begin{split} \left[ \begin{matrix} 3 & -3 & 9\\ 2 & -2 & 7 \\ -1 & 2 & -4 \end{matrix} \right] \left[ \begin{matrix} x_1 \\ x_2 \\ x_3 \end{matrix} \right] = \left[ \begin{matrix} 24\\ 17 \\ -11 \end{matrix} \right] \end{split} \nonumber$

Notice how doing the matrix multiplication results back into the original system of equations. If we rename the three matrices from above to $A$ , $x$ , and $b$ (note $x$ and $b$ are lowercase because they are column vectors) then we get the main equation for this class, which is:

$Ax=b \nonumber$

Note the information about the equation doesn’t change when you change formats. For example, the equation format, the augmented format and the $Ax=b$ format contain the same information. However, we use the different formats for different applications. Consider the numpy.linalg.solve function which assumes the format $Ax=b$

If you have already signed in, please refresh the page.

A = np.matrix([[3, -3,9], [2, -2, 7], [-1, 2, -4]])
sym.Matrix(A)
# 'Run' this cell to see the output

A = np.matrix([[3, -3,9], [2, -2, 7], [-1, 2, -4]])
sym.Matrix(A)
# 'Run' this cell to see the output

If you have already signed in, please refresh the page.

b = np.matrix([[24], [17], [-11]])
sym.Matrix(b)
# 'Run' this cell to see the output

b = np.matrix([[24], [17], [-11]])
sym.Matrix(b)
# 'Run' this cell to see the output

If you have already signed in, please refresh the page.

#Calculate answer to x using numpy
x = np.linalg.solve(A,b)
sym.Matrix(x)
# 'Run' this cell to see the output

#Calculate answer to x using numpy
x = np.linalg.solve(A,b)
sym.Matrix(x)
# 'Run' this cell to see the output

Question

What is the size of the matrix resulting from multiplying a $10 \times 40$ matrix with a $40 \times 3$ matrix?

Do This

Question

Support Center

How can we help?