Linear Algebra, Live
Strang §2.3 · Elimination using matrices

Elimination matrices: each row step is a matrix

Slides 2 to 4 did elimination by hand, one row step at a time. Now we write each step as a matrix, so a computer can store the steps and reuse them. Think of an instruction card: “take 2 times row 1 away from row 2, leave the other rows alone”. An elimination matrix is that card written in numbers. It starts from the identity matrix I (1s on the diagonal, 0s elsewhere), the card that says “keep every row as it is”. Put −ℓ in row 2, column 1, and you get E₂₁. Here E₂₁A means one simple thing: each row of E₂₁ says how to combine the rows of A. Slide 6 shows how to multiply any two matrices.

Lesson 2 · slide 5Elimination matrices
E₂₁ = 100−ℓ10001    E₂₁A = A with row 2 − ℓ·(row 1)
Key ideas
  1. An elimination matrix E is the identity with one extra entry, −ℓ, below the diagonal (if ℓ is negative, this entry is positive).
  2. Multiplying EA performs that row step on A.
  3. A row of EA is a combination of the rows of A.
  4. To undo E, change −ℓ to +ℓ. This is the key to A = LU on slide 7.

Why write a step as a matrix?

As matrices, the whole list of steps becomes one equation that a computer can store and reuse.

An instruction card in numbers

E₂₁ is the card "subtract 2 times row 1 from row 2, leave the rest alone", written as numbers. Multiplying by it follows the card.

E₂₁ = 100−210001

Why it works: rows combine rows

This is a new rule, the mirror image of lesson 1's column rule. There, Ax combined the columns of A. Here, row i of EA is a combination of the rows of A, and row i of E gives the weights. (Slide 6 shows this matches the usual row-times-column rule.)

  • Row 1 of E₂₁ is 100: keep row 1.
  • Row 2 is −210: row 2 minus 2 times row 1.
  • Row 3 is 001: keep row 3.

A row times a matrix uses the row's numbers as weights on the matrix's rows. Row 2, with A from slide 3:

−210 A = −2·211 + 1·4−60 + 0·−272 = 0−8−2

That is the new row 2 from slide 3. The same rule explains why I changes nothing: row i of I has a single 1, in spot i, so it picks out row i of A. So IA = A.

Output E₂₁A
 2  1  1 0 −8 −2−2  7  2
←
E₂₁: one instruction per row
row 1 = row 1row 2 = row 2 − 2·row 1row 3 = row 3
←
Input A
 2  1  1 4 −6  0−2  7  2
A goes in. E₂₁ holds one instruction per row: keep row 1, replace row 2 by row 2 − 2·row 1, keep row 3. Out comes E₂₁A.

The whole process as one product

E₃₂ E₃₁ E₂₁ A = U

Read it right to left: E₂₁ is closest to A, so it acts first. Then E₃₁ acts on that result, then E₃₂. Each step uses only the rows-combine-rows rule; we never need to multiply the E's with each other here.

Each step puts one more E on the left of A and changes one row. After three steps the matrix is the triangle U from slide 3.

Undoing a step

To undo E₂₁, add 2 times row 1 back. We call that matrix E₂₁⁻¹. For now, ⁻¹ just means “the undo step”; lesson 3 covers inverses. It is the identity with +2 in the same spot. Every elimination matrix is undone by flipping the sign of its extra entry.

Next: So far E·A only needed the rows-combine-rows rule. Slide 6 shows how to multiply any two matrices. Then slide 7 uses the undo steps to factor A = LU.

Examples

Example 1 · A 2 by 2 step

A = 3164. The multiplier is ℓ = 6 ÷ 3 = 2.

Show solutionHide solution
E₂₁ = 10−21    E₂₁A = 3102
Row 2 of E₂₁ is −21: −2·31 + 1·64 = 02. Row 1 10 keeps row 1.

Example 2 · The third step, E₃₂

After the first two steps, B = 2110−8−2083. Clear the 8 with ℓ₃₂ = 8 ÷ (−8) = −1.

Show solutionHide solution
E₃₂ = 100010011   (−ℓ₃₂ = +1 in row 3, column 2)
Row 3 of E₃₂ is 011: row 2 + row 3 of B = 0−8−2 + 083 = 001
E₃₂B = 2110−8−2001 = U

Example 3 · Undoing a step

E₂₁ = 100−210001 subtracts 2·row 1 from row 2.

Show solutionHide solution
E₂₁⁻¹ = 100210001 adds it back
On A from slide 3, row 2: 4−60 → 4−60 − 2·211 = 0−8−2 → 0−8−2 + 2·211 = 4−60

Do E₂₁, then E₂₁⁻¹, and A is back where it started. Only the sign of the multiplier changes.

Example 4

Write E₂₁ for A = 2451 and compute E₂₁A.

Show solutionHide solution

ℓ = 5 ÷ 2 = 2.5, so E₂₁ = 10−2.51 and E₂₁A = 240−9. Row 2 of E₂₁A: −2.5·24 + 1·51 = −5−10 + 51 = 0−9.

Example 5

Use the rows-combine-rows rule to check that 10−31 times 1031 gives I.

Show solutionHide solution

Row 1 of the left matrix, 10, keeps row 1 of the right matrix: 10. Row 2, −31, says −3·(row 1) + 1·(row 2) = −3·10 + 31 = 01. The product is 1001.

Exercise Build E₃₁ so that E₃₁ E₂₁ A has zeros below the first pivot. Edit the code and press Run.
slide_5.py Loading Python…
1
Output · Ctrl+Enter runs
Press Run to execute this slide's code. Ctrl+Enter works too.

Enjoying lesson 2? Get the full course: all 8 lessons through the SVD and PCA, your progress saved lesson by lesson, every solution and 5 extra problems per lesson. $20 once, yours for life.

Questions and comments

Stuck on something, or found a mistake? Ask here and I'll answer. You sign in with GitHub.

The full course

From Ax = b to PCA, by running it

Lessons 1 to 3 are free. Finish all eight and you will be able to:

You get: all 8 lessons with Python labs · every worked example and solution · 5 extra problems per lesson · your progress tracked · answers to your questions · every future update.

$20 once · lifetime access
Continue to checkout →

Already a member? Log in to the course