Abstract Vector Spaces

I. Breaking the Limits of the “Arrow”

Until now we have treated vectors as “arrows in space”. But that view has its limits — the truly powerful thing about linear algebra is that a “vector” can be any object satisfying certain operational rules.

Think about it: from beginning to end we have used only two operations — addition and scalar multiplication. As soon as an object can be “added”, can be “multiplied by a number”, and satisfies those few properties, it can be used as a “vector” with every tool of linear algebra!

II. Generalising Vectors: More Things That “Masquerade as Arrows”

1. Functions are vectors too

Functions can be added and multiplied by a number:

(f+g)(x)=f(x)+g(x),(cf)(x)=c f(x)(f + g)(x) = f(x) + g(x), \quad (cf)(x) = c\,f(x)

So functions form a vector space! The “linear combination” of functions is c1f1+c2f2+⋯c_1f_1 + c_2f_2 + \cdots.

This brings about a remarkable fact: the derivative operator of calculus is a “linear transformation”!

ddx(f+g)=dfdx+dgdx,ddx(cf)=cdfdx\frac{d}{dx}(f + g) = \frac{df}{dx} + \frac{dg}{dx}, \quad \frac{d}{dx}(cf) = c\frac{df}{dx}

Differentiation satisfies the two properties of a linear transformation — so differentiation is a “linear transformation on the space of functions”, and its “eigenvectors” satisfy dfdx=λf\frac{df}{dx} = \lambda f, whose solution is f(x)=Ceλxf(x) = Ce^{\lambda x}.

2. Other vector spaces

“Vector” objectAdditionScalar multiplicationExamples
Ordered arraysadding componentsmultiplying each component by c(x1,…,xn)(x_1, \dots, x_n)
Matricesadding corresponding entriesmultiplying each entry by call 2×22\times2 matrices
Polynomialsadding corresponding coefficientsmultiplying each coefficient by ca0+a1x+a2x2a_0 + a_1x + a_2x^2
Continuous functionsadding pointwisemultiplying pointwise by cf(x)f(x)

The key: they all satisfy the same rule of “addition + scalar multiplication + eight axioms”, so every tool of linear algebra (bases, linear transformations, eigenvalues, …) applies to all of them.

III. The Axiomatic Definition of a Vector Space

A set VV is a vector space if it satisfies:

  1. Closed under addition: v+w∈V\mathbf{v} + \mathbf{w} \in V
  2. Closed under scalar multiplication: cv∈Vc\mathbf{v} \in V
  3. It satisfies the associativity, commutativity, zero element and inverse element of addition
  4. It satisfies the associativity, distributivity and identity element of scalar multiplication

As long as these hold, it is a vector space — whatever the “vectors” inside it happen to look like.

IV. The Polynomial Space and the Matrix Space

The polynomial space

All polynomials of degree ≤ 2 form a three-dimensional vector space, with a basis

1,x,x21, \quad x, \quad x^2

Any quadratic polynomial is a linear combination of them:

a0+a1x+a2x2=a0⋅1+a1⋅x+a2⋅x2a_0 + a_1x + a_2x^2 = a_0\cdot 1 + a_1\cdot x + a_2\cdot x^2

The dimension = 3 (the number of coefficients).

The matrix space

All 2×22\times2 matrices form a four-dimensional vector space, with a basis

(1000),(0100),(0010),(0001)\begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix}, \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix}, \begin{pmatrix} 0 & 0 \\ 1 & 0 \end{pmatrix}, \begin{pmatrix} 0 & 0 \\ 0 & 1 \end{pmatrix}

Any 2×22\times2 matrix is a linear combination of them, and the dimension = 4.

V. Why This Matters

  1. Unity: differential equations, Fourier series and machine learning can all be described in the language of linear algebra
  2. Reuse of tools: what you learn in one vector space transfers to all of them
  3. The power of abstraction: setting aside appearances to grasp the essence, namely “linear structure”

The Fourier analysis view: the Fourier transform = “changing the basis” of a function to the sine/cosine basis (the frequency basis) — this is exactly the idea of a change of basis at work. The sine functions are the “eigenfunctions” of the translation operator.

Summary

Symbols Used in This Article

SymbolTypePronunciation/ExplanationMeaning in This Article
VVMathematical symbolthe space VA vector space: a set satisfying the axioms
v+w\mathbf{v} + \mathbf{w}Mathematical symbolv plus wClosure under addition: v+w∈V\mathbf{v}+\mathbf{w}\in V
cvc\mathbf{v}Mathematical symbolc times vClosure under scalar multiplication: cv∈Vc\mathbf{v}\in V
ddx\dfrac{d}{dx}Mathematical symbolderivative operatorThe differentiation operator, a linear transformation on the space of functions
λ\lambdaGreek letterLambdaThe eigenvalue, satisfying dfdx=λf\dfrac{df}{dx}=\lambda f
eλxe^{\lambda x}Mathematical symbole to the lambda xThe eigenfunction of the differentiation operator, giving f(x)=Ceλxf(x)=Ce^{\lambda x}
1,x,x21, x, x^2Mathematical symbolone, x, x squaredA basis of the space of quadratic polynomials, of dimension 3

Found an issue with this content?

Tell us what could be improved.