📌 Norm

Norm of , denoted , represents the “size” of the vector with three properties.

1. Positive definite: for and for .
2. Absolutely homogeneous: .
3. Triangle inequality: .

The -norm is defined as

Special Norms §

For extreme values of , there are special norms.

1. For , . In other words, this is the number of non-zero elements in .
2. For , . This is the maximum magnitude value in .

Info

Note that is a pseudo-norm since it violates the second property defined above. , and instead, for .

Matrix Norms §

For a matrix, we commonly use the Frobenius norm, a function of the elements in the matrix or the singular values of the matrix:

Another norm is the spectral norm, defined as

Intuitively, this measures how long any vector can at most become when multiplied by .