Linear Algebra, Live
Strang §1.3 · Matrices

Multiplying a matrix by a vector

A matrix A is a table of numbers, in rows and columns. To multiply it by a vector x = x₁x₂, there are two ways, and they always give the same answer. By rows: dot each row of A with x (slide 2), and each dot product gives one number of the answer. By columns: take x₁ of column 1, plus x₂ of column 2, a linear combination like on slide 1. Rows are the quick way to compute. Columns are the way to understand what Ax can be.

Lesson 1 · slide 3Matrix times vector
a₁₁a₁₂a₂₁a₂₂ x₁x₂ = x₁ a₁₁a₂₁ + x₂ a₁₂a₂₂
Key ideas
  1. Row way: entry i of Ax is row i of A dotted with x.
  2. Column way: Ax = x₁·(column 1) + x₂·(column 2), a combination of the columns.
  3. Both ways add up the same products, so they always agree.
  4. x needs one number per column of A. Ax has one number per row of A.

A note on writing: aij is the entry in row i, column j, so a₁₂ is in row 1, column 2. That is what the small numbers in the banner above mean.

One example, two ways

Call the matrix A, and the two numbers in x x₁ and x₂. Take

A = 2−111    x = 31

Way 1, by rows. Each row of A is dotted with x. Row 1 gives the top number of the answer, row 2 the bottom number:

row 1: 2−1 · 31 = 2·3 + (−1)·1 = 6 − 1 = 5
row 2: 11 · 31 = 1·3 + 1·1 = 3 + 1 = 4
Ax = 54
By rows: the highlighted row meets x, and their dot product fills one number of the answer.

Way 2, by columns. x = 31 says: take 3 of column 1 and 1 of column 2, and add:

Ax = 3·21 + 1·−11 = 63 + −11 = 54
By columns: each number of x scales its column, and the scaled columns add up to Ax.

Same answer, 54. It has to be: both ways add the same four products, just in a different order. On the plane, the column way is a walk: 3 steps of 21, then 1 step of −11.

The column way as a walk: 3 of column 1, then 1 of column 2, ends at Ax = 54.

An everyday picture: a shopping basket

An egg has about 6 g of protein and 1 g of carbohydrate. A bowl of rice has about 4 g and 45 g. Each food is a column:

A = 64145    x = (2 eggs, 1 bowl)

The basket holds 2·61 + 1·445 = 1647: 16 g of protein, 47 g of carbohydrate. That total is Ax. Adding whole foods, one column at a time, is the column way.

The basket by rows

Or add one nutrient at a time. Protein: 6·2 + 4·1 = 16. Carbohydrate: 1·2 + 45·1 = 47. Each is a dot product of a row of A with x.

Output Ax
16 g protein47 g carbs
←
A: one operation per row
protein = 6·eggs + 4·bowlscarbs = 1·eggs + 45·bowls
←
Input x
2 eggs1 bowl
The input x goes into A. Each row of A is one operation on the input, and makes one number of the output.

Both ways add the same products, so they always agree. Rows are quicker by hand. Columns show what Ax can be.

Watch the sizes

A vector with n entries is an n by 1 matrix. x needs one entry per column of A, and Ax has one entry per row. So a 2 by 3 matrix (2 rows, 3 columns) takes in 3 numbers and gives back 2.

(2 × 3) · (3 × 1) → (2 × 1)    the inner numbers must match, the outer numbers give the size of the answer

A bigger example: 4 nutrients, 3 foods

Add an apple, and track fat and fiber too. Now A has 4 rows (nutrients) and 3 columns (foods):

A = 64014525500014    x = (2 eggs, 1 bowl, 1 apple)
Ax = 2·6150 + 1·44501 + 1·02504 = 1672105

16 g protein, 72 g carbs, 10 g fat, 5 g fiber. Same idea, bigger table: 3 numbers go in, 4 come out, because (4 × 3) · (3 × 1) → (4 × 1). A matrix can have any number of rows and columns.

Next: So one x gives one output Ax. Next: try every x and collect all the outputs. That set tells us which targets b (vectors we want Ax to equal) we can hit.

Examples

Example 1 · Both ways

A = 3125, x = 12.

Show solutionHide solution
By columns: 1·32 + 2·15 = 512
By rows: 3·1 + 1·22·1 + 5·2 = 512

Example 2 · A wide matrix

A = 102311 has 3 columns, so x needs 3 entries. Take x = 211. (2 × 3) · (3 × 1) → (2 × 1)

Show solutionHide solution
Ax = 2·13 + 1·01 + 1·21 = 48

3 numbers in, 2 numbers out: one per row of A.

Example 3 · Picking out a column

For any 2 × 2 matrix A: what is A times x = 10? And times 01?

Show solutionHide solution
A10 = 1·(column 1) + 0·(column 2) = column 1
A01 = column 2

So the columns of A are exactly what A does to 10 and 01. We use this again in lesson 6, where a matrix moves the whole plane.

Example 4

Which x makes Ax equal to column 2 of A? Which x gives column 1 + column 2?

Show solutionHide solution

x = 01 picks out column 2. x = 11 gives column 1 + column 2.

Example 5

A is 3 by 2. How many entries must x have, and how many does Ax have?

Show solutionHide solution

x needs 2 entries, one per column. Ax has 3 entries, one per row.

Exercise Choose x so that Ax = (0, 3). Hint: x₁(1, −1) + x₂(1, 2) = (0, 3) means x₁ + x₂ = 0 and −x₁ + 2x₂ = 3. Edit the code and press Run.
slide_3.py Loading Python…
1
Output · Ctrl+Enter runs
Press Run to execute this slide's code. Ctrl+Enter works too.

Enjoying lesson 1? Get the full course: all 8 lessons through the SVD and PCA, your progress saved lesson by lesson, every solution and 5 extra problems per lesson. $20 once, yours for life.

Questions and comments

Stuck on something, or found a mistake? Ask here and I'll answer. You sign in with GitHub.

The full course

From Ax = b to PCA, by running it

Lessons 1 to 3 are free. Finish all eight and you will be able to:

You get: all 8 lessons with Python labs · every worked example and solution · 5 extra problems per lesson · your progress tracked · answers to your questions · every future update.

$20 once · lifetime access
Continue to checkout →

Already a member? Log in to the course