Linear Algebra: Null Space and Column Space

Understanding the null space and column space of a matrix is not just an algebraic exercise; it's essential for solving systems of equations, analyzing transformations, and grasping the fundamental structure of data in engineering fields from control systems to machine learning. These subspaces reveal the hidden geometry within a matrix, telling you what gets erased and what can be reached.

The Null Space: What Gets Annihilated

The null space (or kernel) of an $m\times n$ matrix $A$ is the set of all vectors $\vec{x}$ in ${\mathbb{R}}^n$ such that when multiplied by $A$, they yield the zero vector. Formally, it is defined as:

$$\text{Null}(A)=\{\vec{x}\in {\mathbb{R}}^n\mid A\vec{x}=\vec{0}\}$$

Think of matrix $A$ as a function or a transformation. The null space contains all the input vectors that are completely "crushed" to zero by this transformation. To compute it, you solve the homogeneous system $A\vec{x}=\vec{0}$. The standard method is to reduce $A$ to its Reduced Row Echelon Form (RREF). The solutions will be expressed in terms of free variables, and the vectors that multiply these free variables form a basis for the null space.

Example: Find the null space of $A=\left[\begin{array}{ccc}1& 2& 3\\ 2& 4& 6\end{array}\right]$.

1. Set up $A\vec{x}=\vec{0}$: $\left[\begin{array}{ccc}1& 2& 3\\ 2& 4& 6\end{array}\right]\left[\begin{array}{c}{x}_1\\ x_2\\ x_3\end{array}\right]=\left[\begin{array}{c}0\\ 0\end{array}\right]$.
2. Row reduce to RREF. The second row is redundant, yielding: $\left[\begin{array}{ccc}1& 2& 3\\ 0& 0& 0\end{array}\right]$.
3. The equation is $x_1+2x_2+3x_3=0$. Let $x_2=r$ (free) and $x_3=s$ (free).
4. Then $x_1=-2r-3s$.
5. The solution vector is $\vec{x}=r\left[\begin{array}{c}-2\\ 1\\ 0\end{array}\right]+s\left[\begin{array}{c}-3\\ 0\\ 1\end{array}\right]$.

Thus, $\text{Null}(A)=\text{Span}\left\{\left[\begin{array}{c}-2\\ 1\\ 0\end{array}\right],\left[\begin{array}{c}-3\\ 0\\ 1\end{array}\right]\right\}$.

The Column Space: What Can Be Reached

The column space (or range) of an $m\times n$ matrix $A$ is the set of all possible linear combinations of its column vectors. It is a subspace of ${\mathbb{R}}^m$. Formally, if $A=[{\vec{a}}_1{\vec{a}}_2\dots {\vec{a}}_n]$, then:

$$\text{Col}(A)=\text{Span}\{{\vec{a}}_1,{\vec{a}}_2,\dots ,{\vec{a}}_n\}$$

This space answers a critical question: "For a given $A$, what vectors $\vec{b}$ in ${\mathbb{R}}^m$ can be expressed as $A\vec{x}$?" In other words, the column space consists of all possible outputs $\vec{b}$ for which the system $A\vec{x}=\vec{b}$ is consistent. A practical basis for the column space is found by taking the pivot columns from the original matrix $A$, not its RREF. The RREF is used only to identify which columns are pivot columns.

Example: Find a basis for the column space of the same matrix $A=\left[\begin{array}{ccc}1& 2& 3\\ 2& 4& 6\end{array}\right]$.

1. Row reduce to RREF: $\left[\begin{array}{ccc}1& 2& 3\\ 0& 0& 0\end{array}\right]$.
2. Identify the pivot column: only column 1 contains a pivot.
3. Therefore, a basis for $\text{Col}(A)$ is the first column of the original $A$: $\left\{\left[\begin{array}{c}1\\ 2\end{array}\right]\right\}$.

Notice the second and third columns are just scalar multiples of the first, so they don't add new independent directions to the span.

Relating the Spaces to System Solutions

The null space and column space directly explain the solution behavior of the linear system $A\vec{x}=\vec{b}$.

• Existence of a Solution: A solution exists if and only if $\vec{b}$ is in the column space of $A$.
• Uniqueness of a Solution: If a solution exists, it will be unique only if the null space contains only the zero vector. If the null space contains other vectors, then any particular solution can have an infinite amount of solutions created by adding any vector from the null space. The general solution is: $\vec{x}={\vec{x}}_p+{\vec{x}}_n$, where ${\vec{x}}_p$ is any particular solution and ${\vec{x}}_n$ is *any$vectorin$\text{Null}(A)$.

This relationship is the cornerstone of understanding why some systems are solvable and why others have infinitely many solutions.

Dimension, Rank, and the Rank-Nullity Theorem

The power of these concepts is fully realized when we quantify the "size" of these subspaces.

• The dimension of the column space is called the rank of the matrix, denoted $\text{rank}(A)$. It is the number of pivot columns, or the number of linearly independent columns in $A$.
• The dimension of the null space is called the nullity of the matrix, denoted $\text{nullity}(A)$. It is the number of free variables in the solution to $A\vec{x}=\vec{0}$.

The Rank-Nullity Theorem (or the Fundamental Theorem of Linear Algebra, Part 1) provides a profound and verifiable link between these dimensions. For an $m\times n$ matrix $A$:

$$\text{rank}(A)+\text{nullity}(A)=n$$

The number $n$ is the number of columns, which is the dimension of the input space (${\mathbb{R}}^n$). This theorem states that the dimension of what you can reach (rank) plus the dimension of what gets crushed to zero (nullity) must equal the total number of input dimensions. In our example, $A$ is a $2\times 3$ matrix. We found $\text{rank}(A)=1$ (one pivot column) and $\text{nullity}(A)=2$ (two free vectors in the null space basis). Indeed, $1+2=3$, verifying the theorem.

Geometric Interpretations

Visualizing these spaces solidifies your intuition.

• For a matrix $A$ that represents a linear transformation from ${\mathbb{R}}^n$ to ${\mathbb{R}}^m$, the null space is a subspace of the domain (${\mathbb{R}}^n$). It consists of all domain vectors that are mapped to the origin in the codomain (${\mathbb{R}}^m$).
• The column space is a subspace of the codomain (${\mathbb{R}}^m$). It is the "image" or the "footprint" of the entire domain after transformation—it's where all possible outputs live.

Consider a projection matrix $P$ that projects vectors in ${\mathbb{R}}^3$ onto the $xy$-plane. The column space is the entire $xy$-plane (rank = 2). The null space is the $z$-axis, because any vector purely along the $z$-axis gets projected to the origin (nullity = 1). The Rank-Nullity Theorem holds: $2+1=3$.

Common Pitfalls

1. Using RREF Columns for the Column Space Basis: A frequent error is to take the pivot columns from the RREF of $A$ as the basis for $\text{Col}(A)$. This is incorrect. The column relationships are not preserved under row operations. You must use the corresponding columns from the original matrix $A$.

• Correction: Use RREF to identify the pivot column positions (e.g., columns 1 and 3). Then extract columns 1 and 3 from the original matrix $A$ to form your basis.

1. Confusing Domain and Codomain: It's easy to misremember where each space lives, especially when $m\ne n$. This leads to trying to draw the null space in the wrong dimension.

• Correction: Always note the matrix dimensions $m\times n$. The null space is a subspace of ${\mathbb{R}}^n$ (the input space). The column space is a subspace of ${\mathbb{R}}^m$ (the output space).

1. Assuming Rank is Always the Number of Non-Zero Rows: While the rank equals the number of pivot rows in RREF, this is only true if the matrix is in echelon form. For a raw matrix with linearly dependent rows, simply counting non-zero rows is unreliable.

• Correction: Always perform row reduction to RREF or echelon form to count the number of pivots definitively.

1. Overlooking the Zero Vector: Both the null space and column space are subspaces, which means they always contain the zero vector. Forgetting to include it in a geometric description is a conceptual omission.

• Correction: When describing these spaces, explicitly state or visualize that they pass through the origin.

Summary

• The null space of a matrix $A$ consists of all solutions to $A\vec{x}=\vec{0}$. It is computed by solving this homogeneous system, typically using RREF to find basis vectors tied to free variables.
• The column space is the span of $A$'s column vectors. A basis is formed by the original matrix columns that correspond to pivot positions identified in RREF.
• These subspaces govern linear systems: $\vec{b}$ must be in $\text{Col}(A)$ for $A\vec{x}=\vec{b}$ to be solvable, and the null space dictates whether solutions are unique or infinite.
• The dimension of the column space is the rank, and the dimension of the null space is the nullity. The Rank-Nullity Theorem, $\text{rank}(A)+\text{nullity}(A)=n$, is a fundamental law linking these quantities.
• Geometrically, the null space lies in the domain of the transformation and is mapped to zero, while the column space lies in the codomain and is the set of all possible outputs.