Multiplying two matrices

One number at a time: row times column

Take

The top-left number of AB comes from row 1 of A and column 1 of B. Multiply matching parts and add, a dot product (lesson 1, slide 2):

Do the same for every pair. Row i and column j make the number in row i, column j:

One column at a time: A times each column of B

Now look at a whole column of AB. Column 1 of AB is just A times column 1 of B, a matrix times a vector, exactly as in lesson 1. By columns:

That is column 1 of AB. Column 2 is A·68 = 2250. So every column of AB is a combination of the columns of A.

One row at a time

There is a mirror image of this. Row 1 of AB is row 1 of A times B: take 1 of B's first row plus 2 of B's second row:

All three views add up the same products. Use row times column to compute by hand. Use the column view to understand what AB is.

The same with 3 × 3 matrices

Nothing new happens with bigger matrices. Each dot product just has three products to add instead of two. Watch all nine numbers of AB fill in:

Not every pair can be multiplied

A row of A must have as many numbers as a column of B, or the dot product doesn't work. So the number of columns of A must equal the number of rows of B. If they don't match, AB simply does not exist:

So with A = 123456, the product AA does not exist, even though A times itself sounds natural. Only square matrices (same number of rows and columns) can always be multiplied by themselves. And sometimes AB exists but BA doesn't: a 2 × 3 times a 3 × 4 works, but 3 × 4 times 2 × 3 does not.

A vector is a matrix with one column, so lesson 1's Ax is the case p = 1.

Order matters

Multiply the same two matrices the other way round:

Different. With numbers, 3·5 = 5·3. With matrices, usually AB ≠ BA, so always keep the order. Other rules do carry over: (AB)C = A(BC), so the brackets can move, and A(B + C) = AB + AC (add matrices entry by entry).

A bigger example: 2 × 3 times 3 × 2

Each dot product now has 3 products to add. A neural network layer is one big matrix product like this, with thousands of rows and columns.

Where this is used

Slide 5 already used it: each elimination step is a matrix, and its steps E₃₂E₃₁E₂₁A are a chain of these products. The next slide turns that chain around into A = LU. Graphics, robotics and machine learning chain matrices the same way.

Next: Elimination turned A into the triangle U. Can we get A back from U? Undoing the steps, as a product of matrices, gives the factorization A = LU.