Calculus III: Lagrange Multipliers

Lagrange multipliers are the indispensable tool for solving constrained optimization problems, where you need to find the maximum or minimum of a multivariable function subject to a side condition. In engineering design, this translates to optimizing material strength given a fixed cost, or maximizing fuel efficiency within a specific engine volume. Understanding this technique moves you from solving abstract textbook problems to tackling real-world design limitations.

The Core Idea: Constraint Forces and Gradient Parallelism

The fundamental problem is this: maximize or minimize an objective function $f(x,y,z)$ subject to a constraint $g(x,y,z)=c$, where $c$ is a constant. You cannot freely roam the entire $(x,y,z)$-space; you are confined to the constraint surface defined by $g(x,y,z)=c$.

The key insight from Joseph-Louis Lagrange is that at an optimal point on the constraint surface, the gradient of the objective function $\nabla f$ must be parallel to the gradient of the constraint function $\nabla g$. If they weren't parallel, you could "walk" along the constraint surface in a direction that would increase (or decrease) $f$, meaning you weren't at an optimum. This parallelism is written as:

$$\nabla f=\lambda \nabla g$$

The scalar $\lambda$ is the Lagrange multiplier. This single vector equation, combined with the original constraint $g(x,y,z)=c$, gives you a system of equations to solve for the critical points $(x_0,y_0,z_0,\lambda )$.

Setting Up and Solving the Lagrange System

The method standardizes a complex problem into a procedural setup. To find the extreme values of $f(\vec{x})$ subject to $g(\vec{x})=c$:

1. Form the Lagrangian function: $\mathcal{L}(x,y,z,\lambda )=f(x,y,z)-\lambda (g(x,y,z)-c)$.
2. Take the partial derivatives and set them equal to zero:

$$\frac{\partial \mathcal{L}}{\partial x}=0,\frac{\partial \mathcal{L}}{\partial y}=0,\frac{\partial \mathcal{L}}{\partial z}=0,\frac{\partial \mathcal{L}}{\partial \lambda }=0.$$ This yields the system: $\nabla f=\lambda \nabla g$ and $g(x,y,z)=c$.

Consider a classic problem: maximize the volume $V=xyz$ of a rectangular box subject to the surface area constraint $2xy+2xz+2yz=S_0$.

• Objective: $f(x,y,z)=xyz$
• Constraint: $g(x,y,z)=2xy+2xz+2yz=S_0$
• Lagrangian: $\mathcal{L}=xyz-\lambda (2xy+2xz+2yz-S_0)$

The resulting system is: $$\frac{\partial \mathcal{L}}{\partial x}=yz-\lambda (2y+2z)=0$$ $$\frac{\partial \mathcal{L}}{\partial y}=xz-\lambda (2x+2z)=0$$ $$\frac{\partial \mathcal{L}}{\partial z}=xy-\lambda (2x+2y)=0$$ $$\frac{\partial \mathcal{L}}{\partial \lambda }=-(2xy+2xz+2yz-S_0)=0$$

Solving this (often by comparing ratios of the first three equations) leads to $x=y=z$, proving the optimal box is a cube.

Extending to Multiple Constraints

Real engineering systems often have more than one limiting factor. The method generalizes elegantly. To optimize $f(\vec{x})$ subject to two constraints $g(\vec{x})=c$ and $h(\vec{x})=d$, you are now restricted to the curve of intersection of the two surfaces. At an optimum, $\nabla f$ must lie in the plane spanned by $\nabla g$ and $\nabla h$—it must be a linear combination of them. This introduces a second multiplier:

$$\nabla f=\lambda \nabla g+\mu \nabla h$$

You now solve this vector equation along with both constraint equations: $g(\vec{x})=c$ and $h(\vec{x})=d$. For example, finding the extreme temperatures $T(x,y,z)$ on a spacecraft's trajectory (a curve) defined by the intersection of two orbital surfaces is a two-constraint problem.

Second-Order Conditions and the Bordered Hessian

Solving the Lagrange system finds candidate points (critical points). It does not tell you if they are maxima, minima, or saddle points on the constrained surface. For a single constraint $g(\vec{x})=c$, you classify the critical point using the bordered Hessian matrix.

For a function of two variables $f(x,y)$ with one constraint $g(x,y)=c$, the bordered Hessian is: $$\overline{H}=\left[\begin{array}{ccc}0& g_x& g_y\\ g_x& {\mathcal{L}}_{xx}& {\mathcal{L}}_{xy}\\ g_y& {\mathcal{L}}_{yx}& {\mathcal{L}}_{yy}\end{array}\right]$$

The second-derivative test states: At a critical point found by the Lagrange method,

• If $\det (\overline{H})>0$, then $f$ has a local maximum subject to the constraint.
• If $\det (\overline{H})<0$, then $f$ has a local minimum subject to the constraint.
• If $\det (\overline{H})=0$, the test is inconclusive.

This moves your analysis from mere candidate identification to definitive classification, which is crucial for confident design decisions.

Applied Optimization in Engineering and Economics

The power of Lagrange multipliers shines in application. In engineering design, it’s used for optimal resource allocation. For instance, minimizing the material cost (a function of panel dimensions) for a cylindrical chemical tank subject to a required volume constraint. The multiplier $\lambda$ itself has a potent interpretation: it is the rate of change of the optimal value of $f$ with respect to the constraint constant $c$. If $V_{max}$ is the maximum volume for a given surface area $S_0$, then $\lambda =d{V}_{max}/d{S}_0$. It tells you how much the optimal volume would increase if you were allowed one more unit of surface area—essentially the "shadow price" of the constraint.

In economics, this is direct. Maximizing a utility function $U(x,y)$ subject to a budget constraint $p_{x}x+p_{y}y=B$ yields optimal bundle $(x^{\ast },y^{\ast })$. The Lagrange multiplier $\lambda$ represents the marginal utility of money: the increase in maximum utility achievable if the budget $B$ were increased by one unit. This quantifies the economic value of relaxing the constraint.

Common Pitfalls

1. Forgetting the Constraint is an Equation: The constraint must be written in the form $g(x,y)=c$ (or $g(x,y)-c=0$). A common error is mis-handling inequalities. The basic Lagrange multiplier method applies only to equality constraints.
2. Algebraic Missteps in Solving the System: The equations $\nabla f=\lambda \nabla g$ often lead to systems where the most efficient solve involves comparing ratios or factoring to find relationships between variables (like $x=y$), rather than prematurely trying to solve for $\lambda$. Rushing leads to lost solutions or incorrect conclusions.
3. Misinterpreting the Multiplier's Sign: In a maximization problem, $\lambda$ is typically positive if the constraint is binding and relaxing it increases the objective. In a minimization problem (like minimizing cost), a positive $\lambda$ indicates that tightening the constraint (making it stricter) would increase the minimum cost. Always contextualize the sign.
4. Neglecting to Check Boundary Points: The Lagrange method finds interior optima on the constraint curve/surface. If the domain of the variables is restricted (e.g., $x>0$, $y\ge 0$), the absolute maximum/minimum might lie at a boundary point where the gradient condition $\nabla f=\lambda \nabla g$ does not hold. You must evaluate candidates from the Lagrange method and any relevant boundary points.

Summary

• The Lagrange multiplier method solves constrained optimization by setting $\nabla f=\lambda \nabla g$, indicating that at an optimum, the objective function's gradient is parallel to the constraint's gradient.
• You solve the resulting system of equations, which includes the original constraint $g(\vec{x})=c$, to find candidate points. The scalar $\lambda$ is the Lagrange multiplier.
• The method extends to multiple constraints using multiple multipliers: $\nabla f=\lambda \nabla g+\mu \nabla h$.
• The bordered Hessian matrix provides a second-derivative test to classify a critical point as a constrained local maximum or minimum.
• In application, the multiplier $\lambda$ has a crucial interpretation as a rate of change: the change in the optimal value of $f$ per unit change in the constraint constant $c$, which is vital for sensitivity analysis in engineering and economics.