Linear Algebra: Change of Basis

In engineering and applied mathematics, you often have the freedom to choose your coordinate system. The physical reality—a force, a vibration mode, a digital signal—doesn't change, but its mathematical description can become simple or intractable based on your choice of perspective. Change of basis is the formal, linear algebraic framework for translating between these different descriptive languages, allowing you to select the coordinate system where your problem is easiest to solve.

Foundations: Bases, Coordinates, and Representations

A basis for a vector space $V$ is a set of linearly independent vectors that span $V$. Every vector in the space can be written uniquely as a linear combination of these basis vectors. The coefficients in that linear combination form the coordinate vector of that vector relative to the chosen basis. This is a crucial distinction: the abstract vector is the object itself, while its coordinate vector is its description in a particular language.

Let's ground this with an example. Consider the vector space ${\mathbb{R}}^2$. The standard basis is $\mathcal{E}=\{{\mathbf{e}}_1=(1,0),{\mathbf{e}}_2=(0,1)\}$. In this basis, the vector $\mathbf{v}=(3,2)$ has coordinate vector $[\mathbf{v}{]}_{\mathcal{E}}=\left[\begin{array}{c}3\\ 2\end{array}\right]$ because $\mathbf{v}=3{\mathbf{e}}_1+2{\mathbf{e}}_2$.

Now, introduce a new basis $\mathcal{B}=\{{\mathbf{b}}_1=(1,1),{\mathbf{b}}_2=(0,1)\}$. To find the coordinates of the same abstract vector $\mathbf{v}$ in this new basis, we solve $\mathbf{v}=x_1{\mathbf{b}}_1+x_2{\mathbf{b}}_2$. This gives the equations: $3=1\cdot x_1+0\cdot x_2$ $2=1\cdot x_1+1\cdot x_2$ Solving yields $x_1=3$ and $x_2=-1$. Therefore, $[\mathbf{v}{]}_{\mathcal{B}}=\left[\begin{array}{c}3\\ -1\end{array}\right]$. The same arrow in the plane now has a different numerical address.

The Change of Basis Matrix

Manually solving a system for every vector is inefficient. The change of basis matrix (or transition matrix) automates this translation. To change coordinates from basis $\mathcal{B}$ to basis $\mathcal{C}$, you construct matrix $P_{\mathcal{C}\leftarrow \mathcal{B}}$. The columns of this matrix are the coordinate vectors of the $\mathcal{B}$-basis vectors, expressed in the $\mathcal{C}$ basis.

$$P_{\mathcal{C}\leftarrow \mathcal{B}}=\left[\begin{array}{cccc}[{\mathbf{b}}_1{]}_{\mathcal{C}}& [{\mathbf{b}}_2{]}_{\mathcal{C}}& \cdots & [{\mathbf{b}}_n{]}_{\mathcal{C}}\end{array}\right]$$

This matrix acts as a translator. Given a coordinate vector $[\mathbf{v}{]}_{\mathcal{B}}$, you multiply to get the coordinates in the $\mathcal{C}$ basis:

$$[\mathbf{v}{]}_{\mathcal{C}}=P_{\mathcal{C}\leftarrow \mathcal{B}}[\mathbf{v}{]}_{\mathcal{B}}$$

Returning to our example, let's construct $P_{\mathcal{E}\leftarrow \mathcal{B}}$, the matrix to change from the new basis $\mathcal{B}$ to the standard basis $\mathcal{E}$. The columns are $[{\mathbf{b}}_1{]}_{\mathcal{E}}=(1,1)$ and $[{\mathbf{b}}_2{]}_{\mathcal{E}}=(0,1)$. Thus:

$$P_{\mathcal{E}\leftarrow \mathcal{B}}=\left[\begin{array}{cc}1& 0\\ 1& 1\end{array}\right]$$

To find $[\mathbf{v}{]}_{\mathcal{E}}$ from $[\mathbf{v}{]}_{\mathcal{B}}$, we compute: $$[\mathbf{v}{]}_{\mathcal{E}}=P_{\mathcal{E}\leftarrow \mathcal{B}}[\mathbf{v}{]}_{\mathcal{B}}=\left[\begin{array}{cc}1& 0\\ 1& 1\end{array}\right]\left[\begin{array}{c}3\\ -1\end{array}\right]=\left[\begin{array}{c}3\\ 2\end{array}\right]$$ This correctly recovers the standard coordinates.

A critical property is that change of basis matrices are invertible. The matrix to change back from $\mathcal{C}$ to $\mathcal{B}$ is the inverse: $$P_{\mathcal{B}\leftarrow \mathcal{C}}=(P_{\mathcal{C}\leftarrow \mathcal{B}}{)}^{-1}$$

Transforming Linear Maps: Similarity Transformations

Vectors are not the only objects affected by a change of basis; the matrices representing linear transformations change too. Let $T:V\to V$ be a linear transformation. If you know its matrix representation $[T{]}_{\mathcal{B}}$ relative to basis $\mathcal{B}$, you can find its representation $[T{]}_{\mathcal{C}}$ relative to basis $\mathcal{C}$ using the same change of basis matrix.

The relationship is a similarity transformation:

$$[T{]}_{\mathcal{C}}=P_{\mathcal{C}\leftarrow \mathcal{B}}[T{]}_{\mathcal{B}}(P_{\mathcal{C}\leftarrow \mathcal{B}}{)}^{-1}$$

Or, equivalently: $$[T{]}_{\mathcal{C}}=P_{\mathcal{C}\leftarrow \mathcal{B}}[T{]}_{\mathcal{B}}{P}_{\mathcal{B}\leftarrow \mathcal{C}}$$

Two matrices $A$ and $B$ are similar if $B=P^{-1}AP$ for some invertible matrix $P$. Similar matrices represent the same linear transformation, just in different bases. They share fundamental properties like the determinant, trace, rank, and eigenvalues (though their eigenvectors are related by the same change of basis).

Applications: Simplifying Matrix Representations

This is where the power of change of basis becomes clear for engineering. The goal is to find a basis $\mathcal{B}$ where the matrix of a transformation $T$ is as simple as possible—ideally, diagonal. This process is diagonalization.

If you can find a basis $\mathcal{B}$ consisting entirely of eigenvectors of $T$, then $[T{]}_{\mathcal{B}}$ is a diagonal matrix whose entries are the corresponding eigenvalues. The change of basis matrix $P$ from the eigenvector basis $\mathcal{B}$ to your starting basis $\mathcal{C}$ has the eigenvectors as its columns. The similarity transformation then becomes:

$$[T{]}_{\mathcal{C}}=P[T{]}_{\mathcal{B}}{P}^{-1}\text{or}[T{]}_{\mathcal{B}}=P^{-1}[T{]}_{\mathcal{C}}P$$

In dynamics, this translates complex coupled differential equations into independent, single-variable equations. In computer graphics, changing to an object's local coordinate basis simplifies rotation and scaling operations. In signal processing, the Fourier transform is essentially a change of basis from the time domain to the frequency domain, where filtering operations become trivial pointwise multiplication.

Common Pitfalls

1. Confusing the Direction of the Transformation: The most frequent error is misapplying the change of basis matrix. Remember the subscript convention: $P_{\mathcal{C}\leftarrow \mathcal{B}}$ converts $\mathcal{B}$-coordinates to $\mathcal{C}$-coordinates. The equation $[\mathbf{v}{]}_{\mathcal{C}}=P_{\mathcal{C}\leftarrow \mathcal{B}}[\mathbf{v}{]}_{\mathcal{B}}$ is a reliable check. If you get it backward, you will be using the inverse matrix.

1. Mixing Vector and Transformation Changes: The formula for a vector is straightforward multiplication: $[\mathbf{v}{]}_{\text{new}}=P\cdot [\mathbf{v}{]}_{\text{old}}$. The formula for a linear map involves a similarity transformation: $[T{]}_{\text{new}}=P\cdot [T{]}_{\text{old}}\cdot P^{-1}$. Applying the vector rule to a matrix is incorrect and will break the fundamental relationship $[T(\mathbf{v})]=[T]\cdot [\mathbf{v}]$.

1. Assuming All Matrices are Change of Basis Matrices: A valid change of basis matrix must be square and invertible. A non-square matrix cannot map between bases of the same space, and a singular (non-invertible) matrix would not provide a unique, reversible translation.

1. Overlooking the Abstract Vector: It's easy to become overly focused on coordinate vectors and forget they are just descriptions. When we say $[\mathbf{v}{]}_{\mathcal{B}}$ and $[\mathbf{v}{]}_{\mathcal{C}}$ are related by $P$, it is because the underlying vector $\mathbf{v}$ itself is unchanged. The change of basis alters the description, not the object being described.

Summary

• A change of basis matrix $P_{\mathcal{C}\leftarrow \mathcal{B}}$ provides a systematic translation for coordinate vectors from one basis to another: $[\mathbf{v}{]}_{\mathcal{C}}=P_{\mathcal{C}\leftarrow \mathcal{B}}[\mathbf{v}{]}_{\mathcal{B}}$.
• The matrix representing a linear transformation changes under a similarity transformation: $[T{]}_{\mathcal{C}}=P_{\mathcal{C}\leftarrow \mathcal{B}}[T{]}_{\mathcal{B}}{P}_{\mathcal{B}\leftarrow \mathcal{C}}$. Similar matrices represent the same transformation in different languages.
• The primary engineering application is simplification. By changing to a well-chosen basis (like a basis of eigenvectors), the matrix of a transformation becomes diagonal, decoupling complex systems into independent, easily solvable components.
• Always mind the direction of the transition matrix and distinguish between the rules for converting vectors and the rules for converting linear maps. The underlying geometry or physics remains invariant; only its numerical representation changes.