# 3 Ways to Understand Matrix Multiplication

--

Boost your intuition for matrix multiplication from the ground up

When I first learned about matrix multiplication I was surprised by how hard it was for me to develop intuition about the operation. The usual definition of matrix multiplication hides a lot of interesting facts that are easier to recognize when you look from different points of view. In this post I will describe matrix multiplication from three points of view: columns, rows, and combinations of both. I will also discuss some simple facts to help test your intuition. After reading you will hopefully gain a deeper understanding of matrix multiplication, rows, and columns. This post was inspired by a linear algebra course taught by the great **Gilbert Strang (MIT)**.

This post lists three ways to interpret matrix multiplication. For each of these interpretations we will discuss the following.

**Interpretation:**What does this mean?**Why does this work?:**How does this interpretation arise from the definition of matrix multiplication?**Check your intuition:**A list of facts that you can use to check your intuition for the interpretation we are considering.

I occasionally mention the concepts of **linear combination**, **linear dependence, linear independence, dot product**. If you would like to quickly refresh your memory on these topics then see my article 3 basic concepts in linear algebra.

**Example.** Say we have three barflies: Lars, Fatima, and Georgia. On a night out Lars bought 2 beers 1 cocktail, Fatima** **bought 1 beer 2 cocktails, and Georgia had 4 beers and no cocktails. Beers cost $7 and cocktails cost $10. We can model their spending for the night with matrix multiplication.

How were the numbers on the right computed?

Our goal is to understand the properties of matrix multiplication with more generality so throughout this post we will consider the product of a **3×3** matrix ** A** and a

**3×2**matrix

**. The result will be a 3×2 matrix**

*B***.**

*C*# The definition of matrix multiplication

Jacques Philippe Marie Binet …recognized as the first to derive the rule for multiplying matrices in 1812. — Oliver Knill

The usual way to define matrix multiplication is as a summation or, more compactly, a dot product of rows of A and columns of B. The dot product of row 1 of A and column 1 of B will give the first entry of C.

In general the ij-th entry of **C** is the i-th row of **A** dotted with the j-th column of **B**.

**Example. **Find the third row and second column of the product **C**.

The answer is (1)(1)+(2)(2) +(3)(1) = 8. Try using the definition to find the rest of entries of **C**.

**Interpretation: **An entry of ** C **is the dot product of a row of

**and a column of**

*A***. Zero entries in**

*B***correspond to a row of**

*C***and a column of**

*A***that are orthogonal (at right angles to each other).**

*B***Check your intuition:** From this point of view several facts become clearer*.*

- The number of columns of
must equal the number of rows of*A*. Otherwise the sums in the definition won’t be defined.*B* - The product
will be a matrix with the same number of columns as*AB***A**and the same number of rows as*B.* - A zero entry in
means that the correspond row of*C***A**and column of**B**are orthogonal. Orthogonal vectors are**linearly independent**. But not all pairs of linearly independent vectors are orthogonal.

# #1 The column point of view

The first thing to notice about ** AB = C** is that the columns of the matrix

**C**are related to the columns of the matrix

**A**in an important way.

**Interpretation **Each of the column of ** C** is the matrix

**multiplied by a column of**

*A***The effect of this is that**

*B.***the columns of**

**are linear combinations of the columns of**

*C***with weights given by columns of**

*A*

*B.***Why does this work? **To see why the columns of ** C** are linear combinations of the columns of

**let’s take a close look at how we calculate the first column of**

*A*

*C.***Check your intuition:** From this point of view several facts become clearer*.*

- A matrix multiplied by a vector,
, is simply a linear combination of the columns of a by the entries of x. So the columns of*Ax*are linearly independent if and only if equation*A*has only the zero solution.*Ax = 0* - We can view the columns of
as the results of applying a linear transformation, defined by*C*, to columns of*B*.*A* - Suppose the columns of
are linearly independent. Then, if*A*has a column of zeros,*C*must also*B*have a column of zeros. - If the columns of
are linearly dependent and the columns of*C*are linearly independent, then the columns of*B*are dependent. This follows from that fact that if*A*is a non-trivial solution of*x*then*Cx = 0*is a non-trivial solution*Bx*of*Ax = 0.* - If the equation
has no solution then*Ax = b*has no solution. After all, the columns of*ABx = Cx = b*are just combinations of columns of*C*.*A* - The span of the columns of
are contained in the span of the columns of*C*. Therefore,*A*.*rank(AB) ≤ rank(A)* - If
is invertible with inverse*B*then the columns of*B’*and*A*have the same span. We can prove this from the previous fact,*AB*, combined with the fact that*rank(AB) ≤ rank(A)**rank(A) = rank(AI) = rank(ABB’) ≤ rank(AB).*

So matrix multiplication from the point of view of columns. Now let’s move on to rows?

# #2 The row point of view

**Interpretation **The rows of ** C** are rows of

**multiplied by the matrix**

*A***. Therefore, the rows of**

*B***are linear combinations of the rows of**

*C***with weights given by rows of**

*B*

*A.***Why does this work? **To see why the rows of ** C** are linear combinations of the rows of

**let’s take a close look at how we calculate the first row of**

*B***using the definition of matrix multiplication.**

*C***Check your intuition: **Once again let’s list some facts about rows that lead from this interpretation of matrix multiplication.

- For
, if the rows of*AB = C*are linearly independent then so are the rows of*C***B**. Warning: the converse is not necessarily true. - If
has a row of zeros then*A*has a row of zeros.*AB* - The the span of the rows of
contains the span the rows of*B*.*C* - If
is an invertible*E*matrix and*n×n*is any*B*matrix. Then*n×m*has the same row space as*EB*. In particular, elementary row operations preserve the row space.*E*

We can use the row and column interpretations the help sketch a proof of an interesting result about the dimension of the row space and column space of an ** m×n** matrix. The dimension of the span of the columns of a matrix is called its

**. The dimension of the span of the rows is called the**

*rank***.**

*rowrank***Claim: **The ** rank** and

**of an**

*rowrank*

*m×n**matrix*

*C**are equal.*

There are many ** m×r **matrices

**and**

*A***matrices**

*r×n***such that**

*B***Choose**

*C = AB.***and**

*A***such that**

*B***is minimum.**

*r***The**

**columns**

*r***of**

**span the column**

*A***space of**

**The**

*C.***rows of**

*r***span the row space of**

*B***Since we chose r to be the smallest such number, the**

*C.*

*rank(C) = rowrank(C) = r.***Claim: **If** A **and** B **are square matrices and** AB = I **then** BA = I. **Therefore **B **is the inverse of** A.**

We have ** AB = I**. Therefore the columns of

**are linearly independent. Therefore the equation**

*A***has only the trivial solution. multiply the first equation on the right by**

*Ax = 0***to get**

*A***. Then**

*ABA = A***. Therefore**

*ABA-A = A(BA-I)=0***.**

*BA = I*# #3 Columns and rows

Our last interpretation gives us a way to decompose a product of two matrices into a sum of matrices.

**Interpretation **The matrix** C **is the sum of matrices that are columns of **A **multiplied by rows of** B. **The matrices that make up the sum have columns that are scalar multiples of a column of** A.**

**Why does this work?** To see why this is true consider what happens when you write the matrix A as the sum of matrices and compute ** AB** by distributing

**.**

*B***Check your intuition:**

- Each of the matrices in the summand have 1-dimensional column spaces.
- You can interchange two columns of
and get the same product*A*as long as you interchange the corresponding rows of*AB*.*B*

**Conclusion**

We talked about three different ways to understand matrix multiplication.

- A matrix multiplied by columns
- A rows multiplied by matrices
- And columns multiplied by rows

We used these different interpretations to review some basic facts about matrix multiplication, independence, and span.