Gram-Schmidt: making vectors orthonormal
Slide 6 showed how easy everything gets with orthonormal vectors. But real vectors rarely come that way, so we need a recipe that makes them. Picture two sticks pushed into the ground, the second one leaning toward the first. Keep the first stick as it is. From the second, take away the part that leans along the first: its shadow on the first stick, from slide 2. What is left stands at a right angle to the first stick. Do the same with each next vector, removing its shadows on all the earlier ones, then divide every vector by its length. That recipe is Gram-Schmidt. Writing the old vectors in terms of the new ones gives A = QR, where R holds the dot products qᵢᵀ(column j) and its diagonal holds the lengths (see below).
Lesson 5 · slide 7Gram-Schmidt- The first vector a is kept as it is.
- From b, subtract its projection onto a. The remainder u₂ is perpendicular to a.
- From a third vector c, subtract its projections onto both a and u₂.
- Finally divide each by its length: q₁ = a/‖a‖, q₂ = u₂/‖u₂‖, and so on.
- With a, b, c as the columns of A, each column is a combination of the q's: A = QR, with R upper triangular.
The idea
From now on a, b, c name three vectors, the columns of A (not the numbers in a 2 by 2 matrix). Gram-Schmidt turns independent vectors a, b, c into perpendicular ones, one at a time. From each new vector, remove its projection onto the ones before it (slide 2).
The steps
u₂ is the error e from slide 2: the part of b off the line through a.
A worked example
A second pair, with less friendly numbers:
Three vectors and more
For c, subtract its projection onto each earlier direction:
Each fraction is a number. Since aᵀu₂ = 0, the two subtractions don't get in each other's way. A useful trick: stretching a vector doesn't change its direction, so we may multiply any u by a number to clear fractions before going on. The picture below does this for a = 110, b = 101, c = 011, with the vectors as the columns:
Done without the stretching (one of the examples below), the same three vectors give the same directions.
A bigger example: the columns of a line fit
Fitting a line to readings at t = 0, 1, 2, 3 (slide 5) uses the columns a = 1111 and b = 0123.
Gram-Schmidt just subtracted the average time, 1.5. The new times add to zero, which is the trick from the accelerometer example on slide 5. With a thousand readings it is the same: subtract the average time.
Keep reading with the full course
Finish lesson 5 and you will be able to: fit the best line to noisy sensor data.
Still locked here:
- The rest of this explanation
- 4 more examples with step-by-step solutions
- Five extra members-only problems with worked solutions
$20 once, lifetime access. Lessons 1 to 3 stay free.
Examples
Example 1 · Two vectors in the plane
a = 10, b = 34.
Show solutionHide solution
The result is the standard basis, and R = 1304.
Questions and comments
Stuck on something, or found a mistake? Ask here and I'll answer. You sign in with GitHub.