Orthonormal vectors and orthogonal matrices Q
Slide 5 saw the equations split apart when the two columns were perpendicular. Now we make that happen on purpose. Think of a city laid out as a perfect square grid of streets, one block apart. To say where you are, you just count blocks east and blocks north, with no calculation. A city with slanted streets makes you work it out. Vectors that are all perpendicular to each other and each have length 1 are called orthonormal: they are the perfect grid. Put them as the columns of a matrix Q and QᵀQ = I. Then coordinates are plain dot products, a square Q is inverted just by transposing it, and least squares needs no inverse at all.
Lesson 5 · slide 6Orthonormal bases- q₁, ..., qₙ are orthonormal when qᵢᵀqⱼ = 0 for i ≠ j and qᵢᵀqᵢ = 1.
- With those vectors as columns, QᵀQ = I.
- A square Q with QᵀQ = I is an orthogonal matrix. Its inverse is its transpose.
- Q keeps lengths and angles (lesson 6 shows that in the plane these Q are turns and mirror flips).
- Coordinates in an orthonormal basis are dot products: x = (q₁ᵀx)q₁ + ... + (qₙᵀx)qₙ.
A perfect grid
The word orthonormal packs two conditions together: ortho, perpendicular to each other, and normal, each of length 1. The usual axes 10 and 01 are orthonormal, and so are other perpendicular pairs of length 1, like q₁ = 0.60.8 and q₂ = −0.80.6:
Any perpendicular pair can be made orthonormal: divide each vector by its length. 34 and −43 are perpendicular with length 5; dividing by 5 gives exactly q₁ and q₂.
QᵀQ = I
Put q₁ and q₂ in the columns of Q. Entry (i, j) of QᵀQ is row i of Qᵀ times column j of Q, which is qᵢ · qⱼ. Perpendicular columns give zeros off the diagonal. Length 1 gives ones on it:
A square Q with QᵀQ = I is called an orthogonal matrix. For it, QᵀQ = I says Q⁻¹ = Qᵀ: inverting is just transposing. (The name is odd: the columns are orthonormal, not just orthogonal.)
Lengths stay the same
‖Qx‖² = (Qx)ᵀ(Qx) = xᵀQᵀQx = xᵀx = ‖x‖². Multiplying by Q never stretches a vector:
Angles stay the same too, since dot products do: (Qx) · (Qy) = xᵀQᵀQy = xᵀy, worked out in the exercises below. Lesson 6 shows what such a Q looks like in the plane: a turn or a mirror flip.
Coordinates are dot products
To write x = c₁q₁ + c₂q₂, dot both sides with q₁. The other term drops out, because q₁ · q₂ = 0, and q₁ · q₁ = 1. So c₁ = q₁ᵀx. No elimination needed.
Projections get simple
With orthonormal columns, AᵀA becomes QᵀQ = I. The normal equations turn into x̂ = Qᵀb, and the projection matrix becomes P = QQᵀ. With one column q₁ = 0.60.8:
That is the shadow in the picture above. Careful: if Q has fewer columns than rows, QQᵀ is not I. It is the projection onto the column space of Q.
Keep reading with the full course
Finish lesson 5 and you will be able to: fit the best line to noisy sensor data.
Still locked here:
- The rest of this explanation
- 4 more examples with step-by-step solutions
- Five extra members-only problems with worked solutions
$20 once, lifetime access. Lessons 1 to 3 stay free.
Examples
Example 1 · The matrix Q(θ) is orthogonal
Q(θ) = cos θ−sin θsin θcos θ. Q(θ) turns the plane by the angle θ; lesson 6 builds it. Is it orthogonal?
Show solutionHide solution
So its inverse turns back by θ.
Questions and comments
Stuck on something, or found a mistake? Ask here and I'll answer. You sign in with GitHub.