Feedback | SoME

Map of SVD

Audience:

Tags: linear-algebrasingular-value-decompositionsvdsymmetric-matrixpositive-semi-definite-matrixorthogonal-transformationdiagonalizationeigenvalueeigenvectorsingular-valuespectral-theoremlyapunov-exponents

I came up with a pedagogical/visual tool for singular value decomposition during the spring semester of 2024, while teaching an introductory course in linear algebra. I call this tool the "map of SVD". During a review before the finals I had helped my students make their own maps of SVD, but there was no public and recorded description of the idea. In the video I go over the map, make one from scratch that the audience can follow along, and finally mention some of the algebra details underneath. While from an arts-and-crafts point of view the video is suitable for a broad audience, in my estimation to really understand and get the most out of the video and the map one would need to be at least somewhat familiar with the more abstract ideas typically developed in an introductory linear algebra class at the undergraduate level. These ideas are approximately (a few reincarnations of) the Fundamental Theorem of Linear Algebra (aka rank-nullity theorem), eigenvalues, eigenvectors, orthogonal transformations, diagonalization and spectral theorem for symmetric matrices. Initially I had also thought of including more advanced reincarnations of the same idea; for instance consider trying to applying the spectral theorem to a sequence of matrices. Although this is very natural from a dynamical point of view, and leads to Lyapunov exponents and works of Furstenberg-Kesten, Oseledets, Karlsson-Margulis etc., ultimately I decided to keep things simple. Perhaps in the future I'll expand on the map of SVD in these directions.

Analytics

4 Overall score*

152 Rank

5 Votes

1 Comment

Comments

Even though I have background in linear algebra and I’m familiar with concepts like row space, column space, how row rank = column rank, how eigenvalues work, what singular values are, etc., I still found this hard to follow unfortunately.

Some suggestions I’d make for the map at 4:17

Clearly define the meanings of red lines, blue lines, and green lines. These were mentioned in passing verbally and I had a hard time keeping track.
Clearly mention that $A$ is rectangular, not necessarily a square, matrix
I didn’t understand the middle row. It looks like you’re repeatedly applying $A$ , $A^T$ , $A$ , $A^T$ , etc to some vector space but why does it go in a cycle like that? If you have a vector $v$ , it is almost certainly not the case that $v = A^TAv = A^TAA^TAv = A^TAA^TAA^TAv = \cdots$ , but the pictures look like they’re going in a cycle.
For the 1st row, maybe I sort of understand it. I’m looking at the bottom left corner and I see $AA^T = UEU^T$ . So if I stretch my imagination and assume that I can go backwards on the $U$ arrow (I don’t even know if this is allowed), and knowing that $U$ is an orthogonal matrix, so $U^{-1} = U^T$ , then by following the arrows:

row 2 column 1 -> row 2 column 2 -> row 2 column 3: $AA^T$
row 2 column 1 -> row 1 column 1 -> row 1 column 3 -> row 2 column 3 $UEU^T$
this makes sense since $AA^T = UEU^T$

I think this is may be useful teaching tool to memorize the SVD formula ONLY if the meaning of the picture is well explained. But there were too many things that were left out and I had to think a lot on my own to fill in the gaps.