Linear Quadratic Regulator Design

The Linear Quadratic Regulator (LQR) is a cornerstone of modern optimal control theory, providing a systematic method for designing feedback controllers for linear systems. It computes optimal state feedback gains by minimizing a cost function that mathematically formalizes the engineering trade-off between achieving rapid state regulation and expending reasonable control effort. For engineers, mastering LQR is essential for designing high-performance, multivariable controllers in aerospace, robotics, and process automation, where balancing speed, accuracy, and efficiency is critical.

The Fundamental Trade-Off: State Deviation vs. Control Effort

Every control design involves inherent compromises. You typically want a system to return to its desired setpoint (state regulation) quickly and with minimal overshoot. However, aggressive control actions demand more energy, stress actuators, and can excite unmodeled dynamics. The LQR framework quantifies this trade-off through a quadratic cost function.

The objective is to find a control law $u(t)=-Kx(t)$ that minimizes the cost functional $J$ over an infinite time horizon: $$J={\int }_0^{\infty }\left(x^{T}Qx+u^{T}Ru\right)dt.$$ Here, $x$ is the state vector deviation from zero, and $u$ is the control input. The two terms inside the integral represent the competing objectives:

• $x^{T}Qx$: This term penalizes deviations of the states from zero. The matrix $Q$ is a positive semidefinite state weighting matrix. By adjusting its elements, you assign relative importance to different states. A large weight on a state variable means you care deeply about regulating it quickly.
• $u^{T}Ru$: This term penalizes control effort. The matrix $R$ is a positive definite control weighting matrix. Larger values in $R$ tell the optimizer that control energy is expensive, forcing it to use gentler inputs.

The designer's primary task is selecting appropriate $Q$ and $R$ matrices to shape the closed-loop system's response. This is the core "knob-turning" of LQR design.

Deriving the Optimal Control Law

Given a linear time-invariant system described by $\dot{x}=Ax+Bu$, we seek the gain matrix $K$ that minimizes the cost $J$. The derivation uses principles from calculus of variations and dynamic programming. The key result is that the optimal control law is a linear state feedback: $$u^{\ast }(t)=-R^{-1}{B}^{T}Px(t)=-Kx(t).$$ Therefore, the optimal feedback gain matrix is: $$K=R^{-1}{B}^{T}P.$$ The matrix $P$ is not arbitrary; it is the solution to a central equation in optimal control.

The Algebraic Riccati Equation

The symmetric, positive definite matrix $P$ is found by solving the Algebraic Riccati Equation (ARE): $$A^{T}P+PA-PB{R}^{-1}{B}^{T}P+Q=0.$$ This is a nonlinear matrix equation. For LQR problems with an infinite time horizon, we seek the unique stabilizing solution $P$ that results in a stable closed-loop system $(A-BK)$. Efficient numerical algorithms (like lqr in MATLAB or solve_continuous_are in Python) exist to compute $P$ and subsequently $K$.

The ARE's solution encapsulates the entire optimization problem. The term $PB{R}^{-1}{B}^{T}P$ reflects the "cost of feedback," and the equation balances the open-loop system dynamics ($A^{T}P+PA$), the state penalties ($Q$), and this feedback cost.

Designing the Weighting Matrices Q and R

Selecting $Q$ and $R$ is more art than science, guided by engineering intuition and iterative simulation. A standard starting point is to choose diagonal matrices. For a system with $n$ states and $m$ inputs, a common heuristic is: $$Q=\text{diag}(q_1,q_2,...,q_n),R=\text{diag}(r_1,r_2,...,r_m).$$ The ratios $q_i/r_j$ determine the relative aggressiveness of controlling state $i$ using input $j$.

A practical design procedure often involves:

1. Normalization: Scale state and input variables so that their maximum desired deviations or efforts are comparable (e.g., 1 unit). This makes initial weight selection easier.
2. Bryson's Rule: Set diagonal weights as the inverse square of the maximum acceptable value for each variable: $q_i=1/x_{i,\text{max}}^2$, $r_j=1/u_{j,\text{max}}^2$.
3. Iterative Tuning: Simulate the closed-loop system. Increase weights on states with unacceptably slow response. Increase control weights if actuator demands are too high or unrealistic. This iterative loop shapes the trade-off between fast response and control energy.

For multivariable systems, off-diagonal terms in $Q$ can be used to penalize correlations between states, but diagonal matrices are sufficient for most applications. The choice of weights directly shapes the resulting gain matrix $K$ and the closed-loop pole locations, moving them into a region of the complex plane that optimally balances performance and effort.

Common Pitfalls

1. Choosing Non-Positive-Definite R: The matrix $R$ must be positive definite. If any control input has a zero penalty ($r_i=0$), the ARE may have no finite solution, implying the optimizer would try to use infinite control effort. Always ensure $R>0$.
2. Ignoring Scaling and Units: Directly using raw state values (e.g., position in meters and angle in radians) with arbitrary weights leads to meaningless designs. A 1-meter error and a 1-radian error are vastly different. Always normalize or use a systematic method like Bryson's Rule to establish a physically meaningful baseline.
3. Overlooking Actuator Limits: LQR provides optimal gains for the unconstrained linear model. It does not account for saturation limits ($|u|\le u_{max}$). Applying the calculated $K$ to a system with saturating actuators can cause performance degradation or instability (integrator windup). Always simulate with saturation blocks and consider techniques like anti-windup compensation.
4. Assuming Robustness: While LQR possesses some good stability margins (e.g., infinite gain margin and at least 60° phase margin for single-input systems), these guarantees vanish for multivariable systems or when not all states are fed back. The optimality of LQR does not automatically imply robustness to model uncertainties. Always perform a robustness analysis (e.g., using singular value plots) on your final design.

Summary

• The Linear Quadratic Regulator (LQR) is a method for computing optimal state feedback gains by minimizing a quadratic cost function that balances state deviation ($x^{T}Qx$) and control effort ($u^{T}Ru$).
• The optimal feedback law is $u=-Kx$, where the gain $K=R^{-1}{B}^{T}P$ is derived from the solution $P$ to the Algebraic Riccati Equation (ARE).
• Design is accomplished by strategically selecting the diagonal elements of the weighting matrices $Q$ (state penalty) and $R$ (control penalty), often through an iterative, simulation-based tuning process.
• Successful application requires careful scaling of variables, ensuring $R$ is positive definite, and validating the design against actuator limits and model uncertainties.