Linear Algebra: Basis and Dimension

Understanding basis and dimension is not merely an academic exercise; it is the cornerstone of modeling and solving real-world engineering problems. From determining the degrees of freedom in a mechanical system to compressing data in machine learning, these concepts provide the language for describing spaces and transformations efficiently. Mastering them allows you to navigate complex vector spaces with confidence, ensuring your mathematical models are both robust and computable.

From Spanning Sets to Linear Independence

A spanning set for a vector space $V$ is a collection of vectors such that every vector in $V$ can be expressed as a linear combination of vectors from that set. Formally, if $S=\{{\mathbf{v}}_1,{\mathbf{v}}_2,...,{\mathbf{v}}_k\}$ is a subset of $V$, then $S$ spans $V$ if for any vector $\mathbf{u}\in V$, there exist scalars $a_1,a_2,...,a_k$ such that $\mathbf{u}=a_1{\mathbf{v}}_1+a_2{\mathbf{v}}_2+...+a_k{\mathbf{v}}_k$. Consider the xy-plane in ${\mathbb{R}}^3$; the set $\{(1,0,0),(0,1,0)\}$ spans this plane because any vector like $(3,-2,0)$ can be written as $3(1,0,0)+(-2)(0,1,0)$.

However, a spanning set can be redundant. The concept of linear independence eliminates this redundancy. A set of vectors is linearly independent if the only solution to the equation $a_1{\mathbf{v}}_1+a_2{\mathbf{v}}_2+...+a_k{\mathbf{v}}_k=0$ is $a_1=a_2=...=a_k=0$. If another solution exists, the vectors are linearly dependent, meaning at least one vector is a linear combination of the others. For instance, $\{(1,0),(0,1),(1,1)\}$ spans ${\mathbb{R}}^2$ but is dependent because $(1,1)=(1,0)+(0,1)$. Independence ensures efficiency, a critical property for engineering design where redundant parameters waste computational resources.

Basis: The Gold Standard for Vector Spaces

A basis for a vector space $V$ is a set of vectors that is both linearly independent and spans $V$. This dual property makes a basis the optimal coordinate system for $V$: it provides a unique, minimal representation for every vector. If $B=\{{\mathbf{v}}_1,{\mathbf{v}}_2,...,{\mathbf{v}}_n\}$ is a basis, then every vector $\mathbf{u}\in V$ can be written uniquely as $\mathbf{u}=c_1{\mathbf{v}}_1+c_2{\mathbf{v}}_2+...+c_n{\mathbf{v}}_n$, where the scalars $c_1,c_2,...,c_n$ are called the coordinates of $\mathbf{u}$ relative to $B$.

The most familiar examples are the standard bases. For ${\mathbb{R}}^n$, the standard basis is $\{{\mathbf{e}}_1,{\mathbf{e}}_2,...,{\mathbf{e}}_n\}$ where ${\mathbf{e}}_i$ has a 1 in the $i$th position and 0 elsewhere. In the vector space of polynomials of degree at most 2, $P_2$, a standard basis is $\{1,x,x^2\}$. These bases are intuitive, but many problems require working with non-standard bases tailored to specific applications, such as eigenbases in vibration analysis or Fourier bases in signal processing.

Finding Bases for Subspaces

In engineering, you often work with subspaces—like the solution set of a homogeneous system $A\mathbf{x}=0$ or the column space of a matrix. Finding a basis for such a subspace is a fundamental skill. For the null space (solution space) of a matrix $A$, you solve $A\mathbf{x}=0$ using Gaussian elimination to get the parametric vector form; the vectors multiplying the free variables form a basis. For the column space of $A$, a basis consists of the pivot columns from the original matrix (not the reduced form).

Let's walk through a concrete example. Find a basis for the subspace $W$ of ${\mathbb{R}}^4$ spanned by the vectors: $${\mathbf{v}}_1=\left[\begin{array}{c}1\\ 2\\ 0\\ 1\end{array}\right],{\mathbf{v}}_2=\left[\begin{array}{c}-1\\ 1\\ 1\\ 0\end{array}\right],{\mathbf{v}}_3=\left[\begin{array}{c}0\\ 3\\ 1\\ 1\end{array}\right],{\mathbf{v}}_4=\left[\begin{array}{c}2\\ 1\\ -1\\ 2\end{array}\right].$$

1. Form a matrix $A$ with these vectors as columns: $A=[{\mathbf{v}}_1{\mathbf{v}}_2{\mathbf{v}}_3{\mathbf{v}}_4]$.
2. Row reduce $A$ to its reduced row echelon form (RREF):

$$\left[\begin{array}{cccc}1& 0& 1& 0\\ 0& 1& 1& 0\\ 0& 0& 0& 1\\ 0& 0& 0& 0\end{array}\right].$$

1. The pivot columns are 1, 2, and 4. Therefore, a basis for the column space (and thus for $W$) is $\{{\mathbf{v}}_1,{\mathbf{v}}_2,{\mathbf{v}}_4\}$. This set spans $W$ and is independent, while ${\mathbf{v}}_3$ is redundant as it equals ${\mathbf{v}}_1+{\mathbf{v}}_2$.

Dimension: The Intrinsic Measure of a Space

The dimension of a vector space $V$, denoted $\dim (V)$, is the number of vectors in any basis for $V$. This is well-defined due to the dimension theorem: all bases for a given vector space contain the same number of elements. Dimension is an intrinsic property; it does not depend on the specific basis chosen. For example, $\dim ({\mathbb{R}}^n)=n$, $\dim (P_2)=3$, and the xy-plane in ${\mathbb{R}}^3$ has dimension 2. This concept quantifies the "degrees of freedom" or the minimum number of coordinates needed to specify any point in the space, directly applicable to state-space models in control theory.

Key properties of dimension include:

• If $\dim (V)=n$, any linearly independent set in $V$ has at most $n$ vectors.
• Any spanning set for $V$ must have at least $n$ vectors.
• If you find a set of $n$ linearly independent vectors in an $n$-dimensional space, it automatically forms a basis.

The Rank-Nullity Theorem: Connecting Fundamental Spaces

The rank-nullity theorem is a powerful tool that relates the dimensions of the fundamental subspaces of a linear transformation. For a linear map $T:V\to W$ represented by an $m\times n$ matrix $A$, the theorem states: $$\dim (\text{Col}(A))+\dim (\text{Nul}(A))=n.$$ Here, $\text{Col}(A)$ is the column space (range of $T$), and its dimension is the rank of $A$. $\text{Nul}(A)$ is the null space (kernel of $T$), and its dimension is the nullity of $A$. The number $n$ is the number of columns in $A$, which equals the dimension of the domain ${\mathbb{R}}^n$.

Consider a matrix $A$ of size $5\times 7$ with rank 3. The rank-nullity theorem immediately tells us that the nullity is $7-3=4$. This means the solution space to $A\mathbf{x}=0$ is 4-dimensional. In engineering terms, if $A$ models a system of constraints, the rank tells you the number of independent equations, while the nullity reveals the number of free variables or inherent modes of the system.

Common Pitfalls

1. Confusing a spanning set with a basis. A spanning set need not be linearly independent. Mistaking a dependent spanning set for a basis leads to non-unique coordinates and inefficiency. Correction: Always check for linear independence after confirming a set spans the space. For a set of vectors in ${\mathbb{R}}^n$, place them as columns in a matrix and row reduce; the pivot columns form a basis for the spanned space.

1. Misapplying dimension to subspaces. Assuming that if a set has $k$ vectors, the subspace it spans has dimension $k$. This is only true if the vectors are independent. Correction: The dimension of $\text{Span}\{{\mathbf{v}}_1,...,{\mathbf{v}}_k\}$ is equal to the rank of the matrix with those vectors as rows or columns. In the earlier example with four vectors, the spanned subspace had dimension 3, not 4.

1. Incorrectly finding a basis for a column space. A common error is taking the pivot columns from the reduced row echelon form (RREF) of $A$ as the basis vectors. Correction: The basis for $\text{Col}(A)$ must consist of the columns from the original matrix $A$ that correspond to pivot positions in the RREF. The RREF columns themselves often do not even belong to the original column space.

1. Overlooking the domain in the rank-nullity theorem. Forgetting that $n$ refers to the number of columns (domain dimension) can lead to incorrect nullity calculations. Correction: Clearly identify the matrix dimensions. For an $m\times n$ matrix, the theorem is $\text{rank}(A)+\text{nullity}(A)=n$, not $m$.

Summary

• A basis is a linearly independent set that spans the entire vector space, providing a unique coordinate system for every vector.
• The dimension of a vector space is the number of vectors in any basis, a fundamental invariant that measures the space's "size" or degrees of freedom.
• To find a basis for a subspace spanned by given vectors, row reduce the matrix formed by these vectors as columns; the pivot columns from the original matrix yield the basis.
• The rank-nullity theorem, $\text{rank}(A)+\text{nullity}(A)=n$, is an essential identity linking the dimensions of the column space (range) and null space (kernel) of a linear transformation.
• Standard bases like $\{{\mathbf{e}}_1,...,{\mathbf{e}}_n\}$ for ${\mathbb{R}}^n$ offer convenient references, but problem-specific bases are often required for efficient computation and analysis in engineering contexts.
• Always verify both spanning and independence when claiming a set is a basis, and use the dimension theorem to check consistency in your work.