Linear Algebra: Subspaces

Mastering the concept of a subspace is the critical step that transforms linear algebra from a collection of matrix operations into a powerful language for describing structure. For engineers, subspaces are not abstract curiosities; they are the mathematical bedrock for analyzing solution spaces in control systems, compressing data via principal component analysis, and understanding the fundamental geometry of forces, signals, and transformations.

What is a Subspace?

A subspace is a subset of a vector space that is itself a vector space, using the same operations of addition and scalar multiplication. This definition means that every subspace inherits properties like associativity and commutativity from the larger space. The real power lies in a simple test: to prove a subset $H$ of a vector space $V$ is a subspace, you only need to verify three conditions.

The Subspace Test: A subset $H$ of a vector space $V$ is a subspace of $V$ if:

1. The zero vector of $V$ is in $H$.
2. $H$ is closed under addition: If $\mathbf{u}$ and $\mathbf{v}$ are in $H$, then $\mathbf{u}+\mathbf{v}$ is in $H$.
3. $H$ is closed under scalar multiplication: If $\mathbf{u}$ is in $H$ and $c$ is any scalar, then $c\mathbf{u}$ is in $H$.

Consider the set of all vectors in ${\mathbb{R}}^3$ of the form $(a,b,0)$. Is this a subspace? The zero vector $(0,0,0)$ is present. Adding two such vectors, $(a_1,b_1,0)+(a_2,b_2,0)=(a_1+a_2,b_1+b_2,0)$, yields another vector with a zero in the third entry. Scalar multiplication, $c(a,b,0)=(ca,cb,0)$, also preserves this form. All three conditions hold, so it is a subspace—visually, it is the $xy$-plane in 3D space. In contrast, the set of vectors of the form $(a,b,1)$ fails the first test because the zero vector is not included.

Fundamental Subspaces of a Matrix

Given an $m\times n$ matrix $A$, there are four naturally associated subspaces. For engineering, the column space and null space are paramount.

The column space, denoted $\mathrm{Col}A$, is the set of all linear combinations of the columns of $A$. If $A=[{\mathbf{a}}_1\cdots {\mathbf{a}}_n]$, then $\mathrm{Col}A=\mathrm{Span}\{{\mathbf{a}}_1,\dots ,{\mathbf{a}}_n\}$. This subspace of ${\mathbb{R}}^m$ consists of all vectors $\mathbf{b}$ for which the equation $A\mathbf{x}=\mathbf{b}$ is consistent. In a physical system $A\mathbf{x}=\mathbf{b}$, the column space represents all possible output vectors $\mathbf{b}$ the system can produce.

The null space, denoted $\mathrm{Nul}A$, is the set of all solutions to the homogeneous equation $A\mathbf{x}=0$. This subspace of ${\mathbb{R}}^n$ contains all input vectors $\mathbf{x}$ that produce a zero output. If the null space contains more than just the zero vector, the system $A\mathbf{x}=\mathbf{b}$ has either no solution or infinitely many. For a matrix $A=\left[\begin{array}{cc}1& 2\\ 3& 6\end{array}\right]$, solving $A\mathbf{x}=0$ yields solutions of the form $x_1=-2x_2$. Therefore, $\mathrm{Nul}A=\mathrm{Span}\left\{\left[\begin{array}{c}-2\\ 1\end{array}\right]\right\}$, a line in ${\mathbb{R}}^2$.

The row space is the set of all linear combinations of the rows of $A$, which is equivalent to $\mathrm{Col}{A}^T$. Its dimension (the rank) is crucially equal to the dimension of the column space.

Solution Spaces as Subspaces

The solution set of a homogeneous system of linear equations, $A\mathbf{x}=0$, is always a subspace—specifically, it is the null space of $A$. This is why we find a basis for the null space to describe all solutions. In contrast, the solution set of a nonhomogeneous system, $A\mathbf{x}=\mathbf{b}$ (with $\mathbf{b}\ne 0$), is never a subspace because it does not contain the zero vector. Geometrically, if the null space is a plane through the origin, the solution set to $A\mathbf{x}=\mathbf{b}$ is a parallel plane not passing through the origin.

Combining Subspaces: Intersection and Sum

New subspaces can be constructed from existing ones. Given two subspaces $H$ and $K$ of a vector space $V$:

• The intersection $H\cap K$ (all vectors in both $H$ and $K$) is always a subspace.
• The sum $H+K$, defined as the set of all vectors that can be written as $\mathbf{h}+\mathbf{k}$ for $\mathbf{h}\in H$ and $\mathbf{k}\in K$, is also always a subspace. It is essentially the subspace spanned by the union of the bases of $H$ and $K$.

A key result is the dimension formula for subspaces: $$\dim (H+K)=\dim (H)+\dim (K)-\dim (H\cap K).$$ When $H\cap K=\{0\}$, the sum is called a direct sum, written $H\oplus K$, and every vector in the sum is written uniquely as $\mathbf{h}+\mathbf{k}$. In ${\mathbb{R}}^3$, the direct sum of the $xy$-plane and the $z$-axis gives you all of ${\mathbb{R}}^3$.

Characterizing All Subspaces of R² and R³

A complete classification of subspaces in low dimensions provides powerful geometric intuition.

• In ${\mathbb{R}}^2$, the only subspaces are:

1. The trivial subspace $\{0\}$ (dimension 0).
2. Lines through the origin (dimension 1). Any subspace spanned by a single non-zero vector.
3. All of ${\mathbb{R}}^2$ (dimension 2).

• In ${\mathbb{R}}^3$, the subspaces are:

1. The trivial subspace $\{0\}$ (dimension 0).
2. Lines through the origin (dimension 1).
3. Planes through the origin (dimension 2). These are precisely the sets of all vectors orthogonal to a given non-zero normal vector $\mathbf{n}$.
4. All of ${\mathbb{R}}^3$ (dimension 3).

This classification reveals a fundamental principle: every non-trivial subspace is a span of a set of vectors and can be described as all vectors orthogonal to a set of other vectors (a concept fully developed when studying orthogonal complements).

Common Pitfalls

1. Assuming any subset defined by a condition is a subspace. The set of vectors in ${\mathbb{R}}^2$ where $x\ge 0$ is not a subspace. While it contains the zero vector, it fails scalar multiplication: multiply $(1,0)$ by $c=-1$ to get $(-1,0)$, which is not in the set. Always apply the three-part test.
2. Confusing the column space with the column vectors themselves. The column space is the infinite set of all linear combinations of the columns. It is a geometric object (a line, plane, etc.), not just a list of vectors.
3. Forgetting that the null space is about inputs, not outputs. The null space of an $m\times n$ matrix $A$ lives in ${\mathbb{R}}^n$ (the domain of the transformation $\mathbf{x}\mapsto A\mathbf{x}$), while the column space lives in ${\mathbb{R}}^m$ (the codomain). Keeping the dimensions straight is essential.
4. Treating the solution set of $A\mathbf{x}=\mathbf{b}$ as a subspace when $\mathbf{b}\ne 0$. This is a critical error. Always check for the presence of the zero vector first. These translated sets are called "affine spaces," but they are not vector subspaces.

Summary

• A subspace is a vector space within another, verifiable by checking it contains the zero vector and is closed under addition and scalar multiplication.
• The column space $\mathrm{Col}A$ consists of all outputs $A\mathbf{x}$ and determines what equations $A\mathbf{x}=\mathbf{b}$ are solvable. The null space $\mathrm{Nul}A$ consists of all inputs that produce a zero output and reveals the "degrees of freedom" in a system's solution.
• The solution set to a homogeneous linear system is always a subspace (the null space), while the solution set to a nonhomogeneous system is never a subspace.
• The intersection and sum of subspaces are powerful tools for building and decomposing vector spaces, governed by the dimension formula.
• Geometrically, the only subspaces of ${\mathbb{R}}^2$ are the origin, lines through the origin, and ${\mathbb{R}}^2$ itself. In ${\mathbb{R}}^3$, they are the origin, lines through the origin, planes through the origin, and ${\mathbb{R}}^3$ itself.