Linear Algebra, Live
Chapter 4 · Lesson 5 · Strang §4.1 to §4.4 · Free preview

Orthogonality and least squares

By the end of this lesson you can
  • Fit the best line to noisy sensor data
  • Project any vector onto a line or a plane
  • Turn any basis into an orthonormal one with Gram-Schmidt

Perpendicular subspaces, projections, the best answer when Ax = b has no solution, fitting lines to data, and orthonormal bases built by Gram-Schmidt. 8 short slides, then worked examples and problems. Each slide has an everyday picture, a worked example and Python you can run right on the page.

Free preview: most of every slide is yours to read, with its Python lab. Members get the full text, every example with its solution, and five extra problems per lesson.

Start the lesson →

Slides

Free previewLesson 5OrthogonalityPerpendicular vectors and perpendicular subspacesOpen slide →01 · Strang §4.1 · Orthogonality of the four subspaces

Orthogonal Vectors and Subspaces Explained Visually

Perpendicular vectors have dot product zero. The row space is perpendicular to the null space. Examples and live Python. Strang §4.1.

Free previewLesson 5ProjectionProjecting a vector onto a lineOpen slide →02 · Strang §4.2 · Projections

Projection Onto a Line: Formula and Picture

Project b onto the line through a: p = (aᵀb / aᵀa) a, the closest point, with the error perpendicular to a. Examples and Python. Strang §4.2.

Free previewLesson 5Projection matrixThe projection matrix POpen slide →03 · Strang §4.2 · Projection matrices

Projection Matrix P = A(AᵀA)⁻¹Aᵀ Explained

The projection onto a line is Pb with P = aaᵀ/aᵀa. Onto the column space of A, P = A(AᵀA)⁻¹Aᵀ. Why P² = P, with examples and Python. Strang §4.2.

Free previewLesson 5Least squaresLeast squares: the best answer when Ax = b has no solutionOpen slide →04 · Strang §4.3 · Least squares approximations

Least Squares Method and the Normal Equations AᵀAx = Aᵀb

When Ax = b has no solution, least squares finds the best x̂ by solving AᵀAx̂ = Aᵀb. Why it works, worked examples and Python. Strang §4.3.

Free previewLesson 5Line fittingFitting a straight line to dataOpen slide →05 · Strang §4.3 · Fitting a straight line

Least Squares Regression Line: Fitting a Line to Data

Fit the best straight line C + Dt to noisy sensor data. Drag C and D to shrink the squared error, then solve the normal equations. Strang §4.3.

Free previewLesson 5Orthonormal basesOrthonormal vectors and orthogonal matrices QOpen slide →06 · Strang §4.4 · Orthonormal bases

Orthonormal Basis and Orthogonal Matrices (QᵀQ = I)

Orthonormal vectors are perpendicular with length 1. In Q they give QᵀQ = I and Q⁻¹ = Qᵀ. Examples and Python. Strang §4.4.

Free previewLesson 5Gram-SchmidtGram-Schmidt: making vectors orthonormalOpen slide →07 · Strang §4.4 · The Gram-Schmidt process

Gram-Schmidt Orthogonalization Process Step by Step (A = QR)

Gram-Schmidt turns independent vectors into an orthonormal basis: subtract projections, divide by length, and get A = QR. Strang §4.4.

Free previewLesson 5RecapWhat you can now do.Open slide →08 · Recap · Reading

Lesson 5 Recap: Orthogonality, Projections and Least Squares

Chapter 4 on one page: orthogonal subspaces, projections, least squares and the normal equations, line fitting, orthonormal bases, Gram-Schmidt.

The full course

From Ax = b to PCA, by running it

Lessons 1 to 3 are free. Finish all eight and you will be able to:

You get: all 8 lessons with Python labs · every worked example and solution · 5 extra problems per lesson · your progress tracked · answers to your questions · every future update.

$20 once · lifetime access
Continue to checkout →

Already a member? Log in to the course