Visualising Eigenvectors

I. The Core Intuition: the “Fixed Directions” of a Transformation

Most vectors change direction after a linear transformation. But some vectors stay on the same straight line after the transformation — they are merely stretched or compressed.

Such a vector is called an eigenvector, and the factor of stretching or compression is called the eigenvalue:

Mv=λvM\mathbf{v} = \lambda \mathbf{v}
  • v\mathbf{v} = the eigenvector (the direction is unchanged)
  • λ\lambda = the eigenvalue (the factor of stretching or compression, a scalar)
        M
v ──────────► still on this straight line (merely stretched or compressed)
  direction unchanged

II. Interactive Demo: Finding the Eigenvectors

How to operate and what to observe:

  • “Scale [[2,0],[0,3]]”: the x-axis is scaled by 2 and the y-axis by 3 → the two eigenvectors lie along the coordinate axes
  • “Shear [[1,1],[0,1]]”: there is only one eigenvector direction (along the x-axis, with λ=1\lambda=1)
  • “Symmetric [[1,0.5],[0.5,1]]”: the eigenvectors lie along the diagonal
  • “Rotate 90°”: there are no real eigenvectors — because every vector is turned away, and no vector keeps its direction

The key point: the purple line = the eigen-directions; the orange MvMv is collinear with the purple v\mathbf{v}, but its length is λ\lambda times as long.

III. How to Solve for the Eigenvalues

Step 1: Write the characteristic equation

Starting from Mv=λvM\mathbf{v} = \lambda\mathbf{v}:

(M−λI)v=0(M - \lambda I)\mathbf{v} = \mathbf{0}

For this equation to have a nonzero solution v\mathbf{v}, you must have:

det⁡(M−λI)=0\det(M - \lambda I) = 0

This is the characteristic equation.

Step 2: Solve for λ

For M=(abcd)M = \begin{pmatrix} a & b \\ c & d \end{pmatrix}:

det⁡(a−λbcd−λ)=(a−λ)(d−λ)−bc=0\det\begin{pmatrix} a - \lambda & b \\ c & d - \lambda \end{pmatrix} = (a-\lambda)(d-\lambda) - bc = 0

Expanding:

λ2−(a+d)λ+(ad−bc)=0\lambda^2 - (a+d)\lambda + (ad - bc) = 0

Using the quadratic formula:

λ=(a+d)±(a+d)2−4(ad−bc)2\lambda = \frac{(a+d) \pm \sqrt{(a+d)^2 - 4(ad-bc)}}{2}

The discriminant Δ=(a+d)2−4(ad−bc)\Delta = (a+d)^2 - 4(ad-bc):

  • Δ>0\Delta > 0: two distinct real eigenvalues
  • Δ=0\Delta = 0: one repeated root
  • Δ<0\Delta < 0: no real eigenvalues (as for a rotation matrix)

Step 3: Substitute back to find the eigenvectors

Substitute each λ\lambda back into (M−λI)v=0(M - \lambda I)\mathbf{v} = \mathbf{0} and solve for the eigenvector direction.

Example: M = [[1, 1], [0, 1]] (a shear)
characteristic equation: (1−λ)² = 0 → λ = 1 (a repeated root)
substitute: (M − I)v = [[0, 1], [0, 0]]v = 0
gives v = (x, 0), that is, along the x-axis → the eigenvector (1, 0)
Geometrically: a shear leaves vectors in the x direction fixed ✓

IV. Three Points about Eigenvalues

  1. What does λ=0\lambda = 0 mean?

    • Mv=0v=0M\mathbf{v} = 0\mathbf{v} = \mathbf{0}, that is, the eigenvector lies in the null space
    • λ=0\lambda = 0 exists ⟺ the matrix is singular (det = 0) — which matches the earlier conclusion about determinants
  2. The eigenvector is not unique

    • there are infinitely many eigenvectors for the same eigenvalue (the whole straight line)
    • a unit vector is usually taken as the representative
  3. The trace and the determinant of the eigenvalues

    • the sum of the eigenvalues = the trace tr(M)=a+d\text{tr}(M) = a + d
    • the product of the eigenvalues = the determinant det⁡(M)=ad−bc\det(M) = ad - bc
    • these make a good quick way to check your work

V. Diagonalisation: Making the Matrix Simple

If the matrix has nn linearly independent eigenvectors, put them in order as a basis; in that basis the matrix becomes a diagonal matrix:

M=BΛB−1,Λ=(λ10⋯0λ2⋯⋮⋮⋱)M = B \Lambda B^{-1}, \quad \Lambda = \begin{pmatrix} \lambda_1 & 0 & \cdots \\ 0 & \lambda_2 & \cdots \\ \vdots & \vdots & \ddots \end{pmatrix}

where the columns of BB are the eigenvectors and the diagonal of Λ\Lambda holds the eigenvalues.

The benefits:

  • computation with a diagonal matrix is extremely simple
  • Mk=BΛkB−1M^k = B\Lambda^k B^{-1} (taking a power only requires powering the diagonal entries)
  • many problems in scientific computation, differential equations and graph theory depend on diagonalisation

VI. Why Eigenvalues Matter in Machine Learning

ApplicationThe relation to eigenvalues
Principal component analysis (PCA)the eigenvectors of the covariance matrix = the principal directions of the data
PageRankthe ranking of web pages = the principal eigenvector of the adjacency matrix
Dynamical systemsthe eigenvalues decide whether the system is stable (∣λ∣<1\lvert\lambda\rvert < 1 converges)
Image compressioneigen-decomposition / SVD compresses images
Quantum mechanicsobservables = the eigenvalues of an operator

The PCA example: the eigenvectors of the covariance matrix of a data set point in the directions where “the variance of the data is greatest”; sorting them by eigenvalue and taking the first few gives the principal components — this is the mathematical essence of dimension reduction.

Summary

Symbols Used in This Article

SymbolTypePronunciation/ExplanationMeaning in This Article
λ\lambdaGreek letterLambdaThe eigenvalue: the factor by which the transformation stretches an eigen-direction
v\mathbf{v}Mathematical symbolbold vThe eigenvector: a nonzero vector that stays on the same straight line, with its direction unchanged, after the transformation
Mv=λvM\mathbf{v} = \lambda\mathbf{v}Mathematical symbolM v equals lambda vThe eigenvalue equation: along an eigen-direction there is only scaling, no rotation
IIMathematical symbolidentity matrixThe identity matrix
(M−λI)v=0(M - \lambda I)\mathbf{v} = \mathbf{0}Mathematical symbol(M minus lambda I) v equals zeroThe matrix form of the characteristic equation, requiring a nonzero solution v\mathbf{v}
det⁡(M−λI)=0\det(M - \lambda I) = 0Mathematical symboldeterminant of (M minus lambda I) equals zeroThe characteristic equation, from which all the eigenvalues are found
Δ\DeltaGreek letterDeltaThe discriminant Δ=(a+d)2−4(ad−bc)\Delta = (a+d)^2 - 4(ad-bc), which decides how many real roots there are
tr(M)\text{tr}(M)Mathematical symboltrace of MThe trace of the matrix MM, equal to the sum of the eigenvalues
Λ\LambdaGreek lettercapital LambdaThe diagonal matrix formed from the eigenvalues
M=BΛB−1M = B\Lambda B^{-1}Mathematical symbolM equals B Lambda B inverseThe diagonalisation: the columns of BB are the eigenvectors and the diagonal entries of Λ\Lambda are the eigenvalues

Found an issue with this content?

Tell us what could be improved.