Linear Algebra, Live
Strang §2.2 · The idea of elimination

Elimination: remove one unknown at a time

Slide 1 found the answer to x − 2y = 1 and 3x + 2y = 11 by looking where two lines cross. That is fine for two lines, but nobody can look at a hundred. Elimination is a recipe that does the work step by step. You already know it from the café. A coffee and a croissant cost €5, and a coffee and two croissants cost €7. The second order has one more croissant and costs €2 more, so a croissant is €2 and the coffee is €3. You subtracted one order from the other and the coffee disappeared. Elimination does exactly that: subtract a multiple of one equation from another so that one unknown disappears, then solve what is left.

Lesson 2 · slide 2Elimination
x − 2y = 1    3x + 2y = 11   →   8y = 8
Key ideas
  1. The pivot is the number we divide by: here the 1 in front of x in equation 1 (the number in front of an unknown is its coefficient). We use it to remove x from the equations below. Later pivots sit on the diagonal, from top-left to bottom-right (slide 3).
  2. The multiplier is ℓ = (entry to remove) ÷ (pivot). In code we write ℓ as l.
  3. Subtracting ℓ times one equation from another keeps the same solution.
  4. When one unknown is left in the last equation, we solve it and work back up: back substitution.

The café puzzle as equations

Call the price of a coffee x and the price of a croissant y. The two orders say:

x + y = 5    x + 2y = 7
(x + 2y) − (x + y) = 7 − 5  →  y = 2
Back to the first order: x + 2 = 5, so x = 3

One subtraction, and x was gone. Every system is solved this way. We only need a rule for how much to subtract.

The same idea, step by step

Take Strang's first system from slide 1: x − 2y = 1 and 3x + 2y = 11.

  • Equation 1 starts with 1x. That 1 is the pivot. It clears the x below.
  • Equation 2 has 3x, so we need 3 copies of equation 1. That 3 is the multiplier, ℓ = 3 ÷ 1 = 3.
  • Subtract 3 times equation 1 from equation 2. The x terms cancel.
(3x + 2y) − 3(x − 2y) = 11 − 3·1
8y = 8,  so y = 1
Back up to equation 1: x − 2·1 = 1,  so x = 3

Putting y = 1 back into equation 1 is called back substitution.

What it looks like

Draw the lines before and after the step. Line 1 stays where it is. Line 2 turns until it is horizontal: 8y = 8 is the line y = 1. The crossing point 31 never moves, because the step did not change the solution.

Equation 2 turns into 8y = 8, a horizontal line. Equation 1 and the crossing point, x = 3 and y = 1, stay where they are.

Why the answer does not change

Any xy that solves both equations also solves "equation 2 minus 3 times equation 1". And adding 3 times equation 1 back undoes the step. So the new system has exactly the same solutions, just easier to read.

Only the numbers matter

The letters x and y only hold the numbers in place. So we can drop them and keep a table. Each equation becomes one row: the number in front of x, the number in front of y, a bar, and the right side. This table is called the augmented matrix [A | b]. The step "row 2 − 3·row 1" acts on the whole row, the right side too.

[A | b] = 1−2|132|11  →  1−2|108|8
The circle marks the pivot. Row 2 minus 3 times row 1 puts a 0 under it, and row 2 now says 8y = 8.

A bigger example: 3 equations

Slide 1's bigger system works the same way. First use equation 1 to remove x from equations 2 and 3. Then use the new equation 2 to remove y from equation 3.

x + y + z = 6    2x + y − z = 1    x + 2y + z = 8
equation 2 − 2·equation 1:  −y − 3z = −11
equation 3 − 1·equation 1:  y = 2
(Lucky: this already gives y. In general this row would still contain z, so we finish the recipe.)
new equation 3 + new equation 2 (ℓ = −1, so subtracting −1 times means adding):  −3z = −9, so z = 3
Back up: −y − 9 = −11, so y = 2 (as we saw). Then x + 2 + 3 = 6, so x = 1

The answer x = 1, y = 2, z = 3 matches slide 1. Circuit simulators and bridge-design programs follow the same steps with thousands of equations. The computer just does the arithmetic.

Next: With three or more equations, the order of the steps matters. Slide 3 clears one column at a time, names the pivots, and climbs back up with back substitution.

Examples

Example 1 · Subtract twice the first equation

Solve 2x + y = 5 and 4x − y = 1.

Show solutionHide solution
Pivot 2, multiplier ℓ = 4 ÷ 2 = 2
(4x − y) − 2(2x + y) = 1 − 2·5  →  −3y = −9  →  y = 3
Back up: 2x + 3 = 5, so x = 1
Check: 4·1 − 3 = 1 ✓

Example 2 · A pivot of 1

Solve x + 3y = 7 and 2x + y = 4.

Show solutionHide solution
ℓ = 2 ÷ 1 = 2.   (2x + y) − 2(x + 3y) = 4 − 14  →  −5y = −10  →  y = 2
x + 6 = 7, so x = 1

A pivot of 1 is the friendliest case: the multiplier is just the entry below it.

Example 3 · A fractional multiplier

Solve 3x + 2y = 7 and x + 4y = 9.

Show solutionHide solution
ℓ = 1 ÷ 3 = 1/3
(x + 4y) − (1/3)(3x + 2y) = 9 − 7/3  →  (10/3) y = 20/3  →  y = 2
3x + 4 = 7, so x = 1

Fractions are fine. The rule never changes: multiplier = entry to remove ÷ pivot.

Example 4

Solve 5x + 2y = 9 and 10x + 3y = 16.

Show solutionHide solution

ℓ = 10 ÷ 5 = 2: (10x + 3y) − 2(5x + 2y) = 16 − 18 gives −y = −2, so y = 2. Then 5x + 4 = 9, x = 1.

Example 5

The first equation is 4x + y = 6, the second 6x + 5y = 16. Find ℓ and solve.

Show solutionHide solution

ℓ = 6 ÷ 4 = 1.5. (6x + 5y) − 1.5(4x + y) = 16 − 9 gives 3.5y = 7, so y = 2. Then 4x + 2 = 6, x = 1.

Exercise Choose the multiplier l so that the new second row starts with 0. Edit the code and press Run.
slide_2.py Loading Python…
1
Output · Ctrl+Enter runs
Press Run to execute this slide's code. Ctrl+Enter works too.

Enjoying lesson 2? Get the full course: all 8 lessons through the SVD and PCA, your progress saved lesson by lesson, every solution and 5 extra problems per lesson. $20 once, yours for life.

Questions and comments

Stuck on something, or found a mistake? Ask here and I'll answer. You sign in with GitHub.

The full course

From Ax = b to PCA, by running it

Lessons 1 to 3 are free. Finish all eight and you will be able to:

You get: all 8 lessons with Python labs · every worked example and solution · 5 extra problems per lesson · your progress tracked · answers to your questions · every future update.

$20 once · lifetime access
Continue to checkout →

Already a member? Log in to the course