Abstract Vector Spaces
Until now we have treated vectors as “arrows in space”. But that view has its limits — the truly powerful thing about linear algebra is that a “vector” can be any object satisfying certain operational rules.
Think about it: from beginning to end we have used only two operations — addition and scalar multiplication. As soon as an object can be “added”, can be “multiplied by a number”, and satisfies those few properties, it can be used as a “vector” with every tool of linear algebra!
1. Functions are vectors too
Functions can be added and multiplied by a number:
So functions form a vector space! The “linear combination” of functions is .
This brings about a remarkable fact: the derivative operator of calculus is a “linear transformation”!
Differentiation satisfies the two properties of a linear transformation — so differentiation is a “linear transformation on the space of functions”, and its “eigenvectors” satisfy , whose solution is .
A Remarkable Conclusion
The exponential function is an “eigenvector” of the differentiation operator! This is precisely the mathematical essence of solving the differential equation . The language of linear algebra unifies the theory of differential equations.
2. Other vector spaces
| “Vector” object | Addition | Scalar multiplication | Examples |
|---|---|---|---|
| Ordered arrays | adding components | multiplying each component by c | |
| Matrices | adding corresponding entries | multiplying each entry by c | all matrices |
| Polynomials | adding corresponding coefficients | multiplying each coefficient by c | |
| Continuous functions | adding pointwise | multiplying pointwise by c |
The key: they all satisfy the same rule of “addition + scalar multiplication + eight axioms”, so every tool of linear algebra (bases, linear transformations, eigenvalues, …) applies to all of them.
A set is a vector space if it satisfies:
- Closed under addition:
- Closed under scalar multiplication:
- It satisfies the associativity, commutativity, zero element and inverse element of addition
- It satisfies the associativity, distributivity and identity element of scalar multiplication
As long as these hold, it is a vector space — whatever the “vectors” inside it happen to look like.
The polynomial space
All polynomials of degree ≤ 2 form a three-dimensional vector space, with a basis
Any quadratic polynomial is a linear combination of them:
The dimension = 3 (the number of coefficients).
The matrix space
All matrices form a four-dimensional vector space, with a basis
Any matrix is a linear combination of them, and the dimension = 4.
Core Conclusion
A “vector” = any object that can be added, can be scaled by a scalar, and satisfies the axioms. Functions, matrices and polynomials can all be vectors. Linear algebra is not about “arrows” — it is about linear structure. Once the structure is established, every tool (linear combinations, bases, transformations, eigenvalues) can be reused.
- Unity: differential equations, Fourier series and machine learning can all be described in the language of linear algebra
- Reuse of tools: what you learn in one vector space transfers to all of them
- The power of abstraction: setting aside appearances to grasp the essence, namely “linear structure”
The Fourier analysis view: the Fourier transform = “changing the basis” of a function to the sine/cosine basis (the frequency basis) — this is exactly the idea of a change of basis at work. The sine functions are the “eigenfunctions” of the translation operator.
Symbols Used in This Article
| Symbol | Type | Pronunciation/Explanation | Meaning in This Article |
|---|---|---|---|
| Mathematical symbol | the space V | A vector space: a set satisfying the axioms | |
| Mathematical symbol | v plus w | Closure under addition: | |
| Mathematical symbol | c times v | Closure under scalar multiplication: | |
| Mathematical symbol | derivative operator | The differentiation operator, a linear transformation on the space of functions | |
| Greek letter | Lambda | The eigenvalue, satisfying | |
| Mathematical symbol | e to the lambda x | The eigenfunction of the differentiation operator, giving | |
| Mathematical symbol | one, x, x squared | A basis of the space of quadratic polynomials, of dimension 3 |
