Visualising Eigenvectors
Most vectors change direction after a linear transformation. But some vectors stay on the same straight line after the transformation — they are merely stretched or compressed.
Such a vector is called an eigenvector, and the factor of stretching or compression is called the eigenvalue:
- = the eigenvector (the direction is unchanged)
- = the eigenvalue (the factor of stretching or compression, a scalar)
M
v ──────────► still on this straight line (merely stretched or compressed)
direction unchanged
Geometric Intuition
The eigenvectors are the “skeleton” of the transformation — they reveal along which directions the transformation merely scales and does not rotate. Once you have found the eigenvectors, you have grasped the essence of the transformation.
How to operate and what to observe:
- “Scale [[2,0],[0,3]]”: the x-axis is scaled by 2 and the y-axis by 3 → the two eigenvectors lie along the coordinate axes
- “Shear [[1,1],[0,1]]”: there is only one eigenvector direction (along the x-axis, with )
- “Symmetric [[1,0.5],[0.5,1]]”: the eigenvectors lie along the diagonal
- “Rotate 90°”: there are no real eigenvectors — because every vector is turned away, and no vector keeps its direction
The key point: the purple line = the eigen-directions; the orange is collinear with the purple , but its length is times as long.
Step 1: Write the characteristic equation
Starting from :
For this equation to have a nonzero solution , you must have:
This is the characteristic equation.
Step 2: Solve for λ
For :
Expanding:
Using the quadratic formula:
The discriminant :
- : two distinct real eigenvalues
- : one repeated root
- : no real eigenvalues (as for a rotation matrix)
Step 3: Substitute back to find the eigenvectors
Substitute each back into and solve for the eigenvector direction.
Example: M = [[1, 1], [0, 1]] (a shear)
characteristic equation: (1−λ)² = 0 → λ = 1 (a repeated root)
substitute: (M − I)v = [[0, 1], [0, 0]]v = 0
gives v = (x, 0), that is, along the x-axis → the eigenvector (1, 0)
Geometrically: a shear leaves vectors in the x direction fixed ✓
-
What does mean?
- , that is, the eigenvector lies in the null space
- exists ⟺ the matrix is singular (det = 0) — which matches the earlier conclusion about determinants
-
The eigenvector is not unique
- there are infinitely many eigenvectors for the same eigenvalue (the whole straight line)
- a unit vector is usually taken as the representative
-
The trace and the determinant of the eigenvalues
- the sum of the eigenvalues = the trace
- the product of the eigenvalues = the determinant
- these make a good quick way to check your work
If the matrix has linearly independent eigenvectors, put them in order as a basis; in that basis the matrix becomes a diagonal matrix:
where the columns of are the eigenvectors and the diagonal of holds the eigenvalues.
The benefits:
- computation with a diagonal matrix is extremely simple
- (taking a power only requires powering the diagonal entries)
- many problems in scientific computation, differential equations and graph theory depend on diagonalisation
Core Conclusions
- Eigenvector: a vector whose direction is unchanged by the transformation; eigenvalue: the factor of stretching or compression
- How to solve: → solve for λ → substitute back for v
- A shear matrix has a repeated eigenvalue; a rotation matrix has no real eigenvalues
- λ=0 ⟺ a singular matrix; the product of the eigenvalues = the determinant
- Diagonalisation: using the eigenvectors as a basis turns the matrix into its simplest form
| Application | The relation to eigenvalues |
|---|---|
| Principal component analysis (PCA) | the eigenvectors of the covariance matrix = the principal directions of the data |
| PageRank | the ranking of web pages = the principal eigenvector of the adjacency matrix |
| Dynamical systems | the eigenvalues decide whether the system is stable ( converges) |
| Image compression | eigen-decomposition / SVD compresses images |
| Quantum mechanics | observables = the eigenvalues of an operator |
The PCA example: the eigenvectors of the covariance matrix of a data set point in the directions where “the variance of the data is greatest”; sorting them by eigenvalue and taking the first few gives the principal components — this is the mathematical essence of dimension reduction.
Symbols Used in This Article
| Symbol | Type | Pronunciation/Explanation | Meaning in This Article |
|---|---|---|---|
| Greek letter | Lambda | The eigenvalue: the factor by which the transformation stretches an eigen-direction | |
| Mathematical symbol | bold v | The eigenvector: a nonzero vector that stays on the same straight line, with its direction unchanged, after the transformation | |
| Mathematical symbol | M v equals lambda v | The eigenvalue equation: along an eigen-direction there is only scaling, no rotation | |
| Mathematical symbol | identity matrix | The identity matrix | |
| Mathematical symbol | (M minus lambda I) v equals zero | The matrix form of the characteristic equation, requiring a nonzero solution | |
| Mathematical symbol | determinant of (M minus lambda I) equals zero | The characteristic equation, from which all the eigenvalues are found | |
| Greek letter | Delta | The discriminant , which decides how many real roots there are | |
| Mathematical symbol | trace of M | The trace of the matrix , equal to the sum of the eigenvalues | |
| Greek letter | capital Lambda | The diagonal matrix formed from the eigenvalues | |
| Mathematical symbol | M equals B Lambda B inverse | The diagonalisation: the columns of are the eigenvectors and the diagonal entries of are the eigenvalues |
