Eigenvalues and eigenvectors describe the properties of certain 🗺️ Linear Mappings. For square matrix , is an eigenvalue if it satisfies

for some , and is an eigenvector. The set of eigenvalues is called the Spectrum of , .

We can find eigenvalues by rearranging the equation above, . We want to find non-trivial , so must be singular with determinant . Thus, we'll solve

for all . This polynomial is called the characteristic polynomial.

Note that the same eigenvalue can repeat multiple times in the characteristic polynomial. The algebraic multiplicity of is the number of times it appears as a root in the characteristic polynomial.

Additionally, observe that if we simply let , we see from the (negated) characteristic polynomial that the determinant of is the product of its eigenvalues,

We can also find the trace of as the sum of eigenvalues,

Eigenspace#

The set of all eigenvectors associated with eigenvalue spans the eigenspace . This is the solution space of

The geometric multiplicity of is the dimension of . This value is always between and the algebraic multiplicity, inclusive.

Diagonalization#

If we have linearly independent eigenvectors, we can diagonalize the matrix. For matrix with eigenvalues and corresponding eigenvectors , diagonalization is

This represents in a new basis , where simply scales along each basis vector. This transformation is called a similarity transformation.

Note that if has distinct eigenvalues, its eigenvectors will be linearly independent. Otherwise, repeated eigenvalues may still have linearly independent eigenvectors, but it's not guaranteed.

Spectral Theorem#

The Spectral theorem states that if is symmetric, , there exists an orthonormal basis consisting of eigenvectors of , and each eigenvalue is real.

Spectral Mapping Theorem#

The Spectral Mapping theorem states that for square and an arbitrary function , we have

In other words, if has eigenvalues , has eigenvalues . Additionally, the eigenvector associated with is then associated with .

Cayley-Hamilton Theorem#

The Cayley-Hamilton theorem states that for with characteristic polynomial

we have that

A common consequence of this theorem is that (and any higher power) can be expressed as a linear combination of .

Power Method#

The Power method is an algorithm for finding eigenvalues of square matrix .

Any vector can be written as a summation over scaled eigenvectors, . Then, we have the following observation.

Every time we multiply by , the eigenvector corresponding the largest eigenvalue gets bigger; multiplying by multiple times, we approach this eigenvector.

After multiplying multiple times by , we find this eigenvector , project it off, and continue to find .