📎 Singular Value Decomposition

SVD decomposes a matrix into

where and , both orthonormal, contain left and right singular vectors respectively and diagonal contains singular values.

1. The singular values are roots of the nonzero eigenvalues of both and .
2. The left singular vectors in are eigenvectors of .
3. The right singular vectors in are eigenvectors of .

The singular values in satisfy

for left and right singular vector and , respectively.

Geometrically, we can interpret this decomposition as a rotation, scaling, and another rotation.

Matrix Approximation §

We can approximate by keeping the largest singular values and their associated vectors. Specifically,

This is the best approximation of , minimizing distortion .

Furthermore,

where the subscript denotes the spectral 📌 Norm.