The transpose: rows become columns, and symmetric matrices

Turn the table

Entry (i, j) is the number in row i, column j. The transpose moves it to row j, column i: entry (i, j) of Aᵀ is entry (j, i) of A. A 3 by 2 matrix becomes 2 by 3.

A vector is a column, an n by 1 matrix. Its transpose is a row, 1 by n: 123ᵀ = 123.

The dot product as a row times a column

Lesson 1's dot product fits right in. Write x as a column and xᵀ as a row. Then xᵀy, a 1 by n row times an n by 1 column, is a 1 by 1 matrix: the dot product x · y.

The order flips again

For a product, (AB)ᵀ = BᵀAᵀ, the same reversal as slide 3. Why? Compare one entry of each side:

With the numbers below: entry (1, 2) of AB is row 1 of A times column 2 of B, 11·01 = 1. Entry (2, 1) of BᵀAᵀ is row 2 of Bᵀ, 01, times column 1 of Aᵀ, 11: also 1.

Inverting then transposing gives the same as transposing then inverting: (A⁻¹)ᵀ = (Aᵀ)⁻¹. Why? Transpose both sides of A⁻¹A = I. The product rule (AB)ᵀ = BᵀAᵀ turns the left side into Aᵀ(A⁻¹)ᵀ, and Iᵀ = I. So Aᵀ(A⁻¹)ᵀ = I, which says (A⁻¹)ᵀ undoes Aᵀ. With slide 2's A:

Symmetric matrices

Think of a table of distances between cities. Paris to Lyon is the same as Lyon to Paris. Flip the table across its diagonal and nothing changes. A matrix like that is symmetric: S = Sᵀ (here S stands for symmetric, not the S from slides 1 and 3), so entry (i, j) equals entry (j, i).

AᵀA is always symmetric

Row i of Aᵀ is column i of A. So entry (i, j) of AᵀA is column i of A dotted with column j of A. Swap i and j and you get the same dot product, so entry (j, i) is the same number.

A bigger example: AAᵀ is 3 × 3

Multiply the other way round and you get a 3 by 3 matrix. Entry (i, j) of AAᵀ is row i of A dotted with row j of A. Swapping i and j gives the same dot product, so AAᵀ is symmetric too:

This is what the lab shows first. With a thousand sensor readings, A is 1000 by 2, so AᵀA is only 2 by 2, and symmetric. That is where lesson 5 starts.

Where this is used

AᵀA shows up whenever we fit a model to many noisy measurements, like a straight line through sensor readings, or tuning a motion sensor in a phone so its readings match reality. And for R from slide 1, which sends pq to −qp, the transpose is the inverse, so undoing R costs nothing to compute (lesson 5 explains which matrices do this). Check it: Rᵀ = 01−10, and RᵀR = 0·0 + 1·10·(−1) + 1·0−1·0 + 0·1(−1)(−1) + 0·0 = I ✓

Next: One more family of matrices hides inside elimination: the ones that only reorder rows. Slide 6 builds them, and their transpose turns out to be their inverse too.