Recap · Reading
What you can now do.
Six ideas from Strang chapter 7, all built on one fact: every matrix is a turn, a stretch and a turn. From that you can find the stretches, split a matrix into pieces, compress a picture, find the main direction of data, and undo a matrix as well as it can be undone.
Lesson 8 · slide 7RecapA = UΣVᵀ = σ₁u₁v₁ᵀ + … + σᵣuᵣvᵣᵀ
Key ideas
- Every matrix is turn, stretch, turn: A = UΣVᵀ.
- σᵢ = √(eigenvalues of AᵀA); the vᵢ are its eigenvectors, uᵢ = Avᵢ/σᵢ.
- The first k pieces give the best rank-k approximation: compression and PCA.
- A⁺ = VΣ⁺Uᵀ undoes what can be undone; σ₁/σₙ measures sensitivity.
- §7.4 The SVD. Rolling dough: turn, stretch along two perpendicular directions, turn again. A = UΣVᵀ, and the unit circle becomes an ellipse with half-axes σ₁ and σ₂.
- §7.2 Finding σ. AᵀA is the tape measure. Its eigenvectors are the input directions v, the square roots of its eigenvalues are the σ's, and u = Av/σ.
- §7.2 Rank-one pieces. A song made of tracks: A = σ₁u₁v₁ᵀ + σ₂u₂v₂ᵀ + …, loudest first. Each piece is a column times a row.
- §7.1 Image compression. A painter blocks in the big shapes first. Keep k pieces of a photo and store k(m + n + 1) numbers instead of mn.
- §7.3 PCA. GPS dots along a straight street. Subtract the average, take the SVD, and v₁ is the main direction. σ₁² / (σ₁² + σ₂² + …) is the share of the spread it explains.
- §7.4 Pseudoinverse and condition number. A shadow can't give back the lost depth. A⁺ = VΣ⁺Uᵀ undoes every stretch it can and picks the shortest best answer. σ₁/σₙ warns how much small errors can grow.
Keep reading with the full course
Finish lesson 8 and you will be able to: compress an image and keep only what matters.
Still locked here:
- The rest of this explanation
- 2 more examples with step-by-step solutions
- Five extra members-only problems with worked solutions
$20 once, lifetime access. Lessons 1 to 3 stay free.
Examples
Example 1
Find the singular values of 0340.
Show solutionHide solution
AᵀA = [16 0; 0 9]: σ₁ = 4, σ₂ = 3. (The axes are swapped and stretched by 4 and 3.)
Questions and comments
Stuck on something, or found a mistake? Ask here and I'll answer. You sign in with GitHub.