Linear Algebra, Live
Chapter 7 · Lesson 8 · Strang §7.1 to §7.4 · Free preview

The SVD and PCA

By the end of this lesson you can
  • Compress an image and keep only what matters
  • Find the main direction in your data with PCA
  • Take any matrix apart as turn, stretch, turn

Every matrix taken apart: turn, stretch, turn. Singular values, the most important pieces of a matrix, image compression, principal component analysis and the pseudoinverse. 7 short slides, then worked examples and problems. Each slide has an everyday picture, a worked example and Python you can run right on the page.

Free preview: most of every slide is yours to read, with its Python lab. Members get the full text, every example with its solution, and five extra problems per lesson.

Start the lesson →

Slides

Free previewLesson 8Turn, stretch, turnThe SVD: every matrix is a turn, a stretch and a turnOpen slide →01 · Strang §7.4 · The geometry of the SVD

Singular Value Decomposition (SVD) Explained Visually

The SVD says every matrix turns, stretches along perpendicular axes, then turns again: A = UΣVᵀ. The unit circle becomes an ellipse. Strang §7.4.

Free previewLesson 8Singular valuesFinding singular values from AᵀAOpen slide →02 · Strang §7.2 · Bases and matrices in the SVD

How to Find Singular Values: Eigenvalues of AᵀA

Singular values are square roots of the eigenvalues of AᵀA; its eigenvectors are the v's, and u = Av/σ. Worked examples and Python. Strang §7.2.

Free previewLesson 8Rank-one piecesSplitting a matrix into rank-one piecesOpen slide →03 · Strang §7.2 · The pieces of the SVD

SVD as a Sum of Rank-One Matrices: A = σ₁u₁v₁ᵀ + σ₂u₂v₂ᵀ

The SVD writes any matrix as a sum of simple rank-one pieces σuvᵀ, ordered from most to least important. Worked examples and Python. Strang §7.2.

Free previewLesson 8Image compressionCompressing an image with the SVDOpen slide →04 · Strang §7.1 · Image processing by linear algebra

Image Compression with the SVD (Low-Rank Approximation)

A grey image is a matrix. Keep only its largest singular values and the picture survives with far fewer numbers. Interactive Python example. Strang §7.1.

Free previewLesson 8PCAPCA: finding the main direction of dataOpen slide →05 · Strang §7.3 · Principal component analysis

Principal Component Analysis (PCA) with the SVD, Visually

PCA finds the direction where data spreads most: center the data, take the first direction from the SVD. Interactive example with noisy robot positions. Strang §7.3.

Free previewLesson 8PseudoinverseThe pseudoinverse and the condition numberOpen slide →06 · Strang §7.4 · The pseudoinverse

Pseudoinverse and Condition Number from the SVD

The pseudoinverse A⁺ = VΣ⁺Uᵀ gives the best least-squares answer for any matrix, and σ₁/σₙ measures sensitivity. Python examples. Strang §7.4.

Free previewLesson 8RecapWhat you can now do.Open slide →07 · Recap · Reading

Lesson 8 Recap: The SVD and PCA

Chapter 7 on one page: A = UΣVᵀ as turn-stretch-turn, singular values from AᵀA, rank-one pieces, image compression, PCA and the pseudoinverse.

The full course

From Ax = b to PCA, by running it

Lessons 1 to 3 are free. Finish all eight and you will be able to:

You get: all 8 lessons with Python labs · every worked example and solution · 5 extra problems per lesson · your progress tracked · answers to your questions · every future update.

$20 once · lifetime access
Continue to checkout →

Already a member? Log in to the course