Linear Algebra: Diagonalization

Diagonalization is one of the most powerful techniques in linear algebra, transforming complex matrix operations into simple scalar manipulations. For engineers, this process is indispensable, providing elegant solutions to problems in structural analysis, control theory, quantum mechanics, and data science. Mastering diagonalization allows you to decouple coupled systems, compute high powers of matrices efficiently, and analyze the long-term behavior of dynamical processes.

What Does It Mean for a Matrix to Be Diagonalizable?

A square matrix $A$ is diagonalizable if it is similar to a diagonal matrix. Formally, an $n\times n$ matrix $A$ is diagonalizable if there exists an invertible matrix $P$ and a diagonal matrix $D$ such that $A=PD{P}^{-1}$. This equation is the cornerstone of the entire concept.

The columns of the matrix $P$ are the eigenvectors of $A$, and the diagonal entries of $D$ are the corresponding eigenvalues. The transformation $P^{-1}AP$ essentially changes the basis of the linear transformation represented by $A$ to a basis made entirely of its eigenvectors. In this new eigenbasis, the transformation acts purely as scaling along each axis, which is what the diagonal matrix $D$ represents. This simplification is why diagonalization is so useful; it reduces problems involving $A$ to problems involving the much simpler matrix $D$.

Criteria for Diagonalizability

Not every matrix can be diagonalized. Understanding the precise conditions is critical to avoid misapplying the technique. The central criterion involves eigenvectors and eigenvalues.

An $n\times n$ matrix $A$ is diagonalizable if and only if it has $n$ linearly independent eigenvectors. These eigenvectors form the columns of the diagonalizing matrix $P$. This condition leads to two practical tests:

1. The Eigenvalue Test: If $A$ has $n$ distinct eigenvalues, then it is automatically diagonalizable. The corresponding eigenvectors are guaranteed to be linearly independent.
2. The Dimension Test: For repeated eigenvalues, diagonalizability depends on the dimension of the corresponding eigenspace. Let $\lambda$ be an eigenvalue. Its algebraic multiplicity is the number of times it appears as a root of the characteristic polynomial. Its geometric multiplicity is the dimension of the eigenspace $E_{\lambda }=\text{Nul}(A-\lambda I)$, which is the number of linearly independent eigenvectors for that $\lambda$.

A matrix is diagonalizable if and only if, for every eigenvalue, the geometric multiplicity equals the algebraic multiplicity. In other words, you must be able to find enough linearly independent eigenvectors to fill out the matrix $P$.

For example, a matrix with characteristic polynomial $(\lambda -2{)}^2(\lambda +1)$ must have two linearly independent eigenvectors for $\lambda =2$ to be diagonalizable. If the eigenspace for $\lambda =2$ is only one-dimensional, the matrix is not diagonalizable.

Constructing the Diagonalizing Matrix $P$ and Diagonal Matrix $D$

The process of diagonalization is a systematic, step-by-step procedure. Let's walk through it with a concrete matrix.

$$A=\left[\begin{array}{ccc}4& -1& 1\\ 2& 1& 1\\ 0& 0& 2\end{array}\right]$$

Step 1: Find the eigenvalues. Solve the characteristic equation $\det (A-\lambda I)=0$. $$\det \left[\begin{array}{ccc}4-\lambda & -1& 1\\ 2& 1-\lambda & 1\\ 0& 0& 2-\lambda \end{array}\right]=(2-\lambda )[(4-\lambda )(1-\lambda )+2]=0$$ Solving gives eigenvalues ${\lambda }_1=2$ (algebraic multiplicity 2) and ${\lambda }_2=3$ (algebraic multiplicity 1).

Step 2: Find a basis for each eigenspace. For each $\lambda$, solve $(A-\lambda I)\mathbf{x}=0$.

• For $\lambda =3$: $(A-3I)\mathbf{x}=0$ yields one eigenvector, e.g., ${\mathbf{v}}_1=\left[\begin{array}{c}1\\ 1\\ 0\end{array}\right]$.
• For $\lambda =2$: $(A-2I)\mathbf{x}=0$. We find two linearly independent eigenvectors, e.g., ${\mathbf{v}}_2=\left[\begin{array}{c}1\\ 2\\ 0\end{array}\right]$ and ${\mathbf{v}}_3=\left[\begin{array}{c}-1\\ 0\\ 2\end{array}\right]$. The geometric multiplicity (2) equals the algebraic multiplicity (2), so the matrix is diagonalizable.

Step 3: Construct $P$ and $D$. Form $P$ from the eigenvectors and $D$ from the corresponding eigenvalues. $$P=[{\mathbf{v}}_1{\mathbf{v}}_2{\mathbf{v}}_3]=\left[\begin{array}{ccc}1& 1& -1\\ 1& 2& 0\\ 0& 0& 2\end{array}\right],D=\left[\begin{array}{ccc}3& 0& 0\\ 0& 2& 0\\ 0& 0& 2\end{array}\right]$$ You can verify that $A=PD{P}^{-1}$.

Computing Matrix Powers and Exponentials

This is where diagonalization pays immediate dividends. Computing $A^k$ directly for a large $k$ is computationally prohibitive. However, if $A=PD{P}^{-1}$, then: $$A^k=(PD{P}^{-1}{)}^k=P{D}^k{P}^{-1}$$ Since $D$ is diagonal, $D^k$ is simply the diagonal matrix with each diagonal entry raised to the $k$-th power. This reduces a complex matrix multiplication problem to two matrix multiplications and trivial exponentiation.

This is directly applicable to dynamical systems of the form ${\mathbf{x}}_{k+1}=A{\mathbf{x}}_k$. The state after $k$ steps is ${\mathbf{x}}_k=A^k{\mathbf{x}}_0=P{D}^k{P}^{-1}{\mathbf{x}}_0$. By analyzing the eigenvalues in $D$, you can immediately see growth/decay rates and dominant modes of the system. Similarly, the matrix exponential $e^{At}$, crucial for solving systems of linear differential equations, becomes $e^{At}=P{e}^{Dt}{P}^{-1}$, where $e^{Dt}$ is a diagonal matrix with entries $e^{{\lambda }_{i}t}$.

Applications: Markov Chains and System Stability

Diagonalization provides deep insight into Markov chains, where the state transition is governed by a stochastic matrix $A$. A key question is the long-term (steady-state) behavior. If $A$ is diagonalizable, the state vector ${\mathbf{x}}_k=A^k{\mathbf{x}}_0$ can be expressed in terms of the eigenvalues. For a regular Markov chain, the eigenvalue ${\lambda }_1=1$ will be dominant, and all other eigenvalues will satisfy $|\lambda |<1$. As $k\to \infty$, $D^k$ will converge to a matrix with a 1 in the (1,1) position and zeros elsewhere. This allows you to compute the steady-state vector directly from the eigenvector corresponding to $\lambda =1$.

In control and systems engineering, diagonalization is used to decouple a multi-input, multi-output system into independent single-input, single-output channels. The eigenvalues determine system stability: if all eigenvalues of the system matrix have negative real parts, the system is stable. Diagonalization (or its generalization for non-diagonalizable matrices) is the primary tool for performing this analysis.

When Diagonalization Is Not Possible

A classic example of a non-diagonalizable matrix is a defective matrix, which lacks a full set of linearly independent eigenvectors. $$J=\left[\begin{array}{cc}2& 1\\ 0& 2\end{array}\right]$$ It has a single eigenvalue $\lambda =2$ with algebraic multiplicity 2. Solving $(J-2I)\mathbf{x}=0$ yields only one independent eigenvector, e.g., $\left[\begin{array}{c}1\\ 0\end{array}\right]$. The geometric multiplicity (1) is less than the algebraic multiplicity (2), so $J$ is not diagonalizable. Such matrices are in Jordan form, which is the closest analog to a diagonal matrix. While not diagonal, the Jordan form is still upper triangular and facilitates computations of functions of the matrix. In engineering contexts, defective matrices often signal resonant or degenerate behavior in physical systems.

Common Pitfalls

1. Assuming distinct eigenvalues guarantee easy eigenvectors. While distinct eigenvalues guarantee diagonalizability, you must still correctly compute the eigenvectors. A simple arithmetic error in row reduction when solving $(A-\lambda I)\mathbf{x}=0$ will lead to an incorrect matrix $P$.
2. Misordering the pairings. The order of eigenvectors in $P$ must correspond to the order of eigenvalues on the diagonal of $D$. Placing eigenvector ${\mathbf{v}}_i$ in column $j$ of $P$ means its eigenvalue must be in the $(j,j)$ entry of $D$.
3. Overlooking the invertibility of $P$. The diagonalizing matrix $P$ must be invertible. If your computed eigenvectors are not linearly independent, $P$ will be singular and $P^{-1}$ will not exist. This is a clear sign you have made an error in calculation or that the matrix is not diagonalizable.
4. Applying diagonalization to non-square matrices. Diagonalization is defined only for square matrices. Related techniques like Singular Value Decomposition (SVD) are used for rectangular matrices.

Summary

• A matrix $A$ is diagonalizable if it can be expressed as $A=PD{P}^{-1}$, where $D$ is diagonal and $P$ is invertible. The columns of $P$ are eigenvectors of $A$, and $D$ holds the corresponding eigenvalues.
• The key diagonalizability criterion is that an $n\times n$ matrix must have $n$ linearly independent eigenvectors. This occurs if for every eigenvalue, its geometric multiplicity equals its algebraic multiplicity.
• The primary application of diagonalization is simplifying matrix computations, especially finding high powers $A^k=P{D}^k{P}^{-1}$ and analyzing the evolution of dynamical systems and Markov chains.
• Not all matrices are diagonalizable; defective matrices lack a full set of eigenvectors and are represented by Jordan canonical form instead.
• The procedure is methodical: find eigenvalues, find a full basis of eigenvectors, construct $P$ and $D$, and verify. Avoiding misordering and ensuring linear independence are crucial for success.