A Deeper Analysis of the Lagrange Multiplier Method
The Core Idea
At a constrained extremum point, the gradient of the objective function is parallel to the normal vector of the constraint surface.
The geometric explanation: consider the level curve of the objective function and the constraint curve . At the extremum point the level curve is tangent to the constraint curve — if it were not tangent, then moving along the constraint curve could still make the value of increase or decrease. Tangency means that the normal vectors of the two curves are parallel, that is, , or .
This is where the vanishing of the partial derivatives of the Lagrangian function comes from: .
Theorem
A theorem is a proposition that has been rigorously proved, and it is the foundation of mathematical reasoning. A theorem usually states its conditions and its conclusion, and is an important part of any mathematical system.
The sensitivity interpretation of the multiplier: suppose is the extremum of under the constraint . Then
that is, the Lagrange multiplier (after taking the negative sign) gives the rate of change of the extremal value of the objective function when the right-hand side of the constraint changes by one unit.
In economics is called the shadow price, and it expresses the marginal value of a resource.
An Example
Under the budget constraint , maximising the utility , the Lagrange multiplier expresses the marginal utility of income — the amount by which the utility increases when the income increases by 1 unit. This is the economic meaning of .
Theorem
A theorem is a proposition that has been rigorously proved, and it is the foundation of mathematical reasoning. A theorem usually states its conditions and its conclusion, and is an important part of any mathematical system.
For the constrained optimisation problem with inequality constraints:
the KKT conditions (the Karush-Kuhn-Tucker conditions) are necessary conditions for an extremum point:
- Stationarity:
- Primal feasibility:
- Dual feasibility:
- Complementary slackness:
The complementary slackness condition means: if the constraint (it is not active), then ; and if , then (the constraint is active).
The Intuition behind the KKT Conditions
- For a constraint that is not active (), it does not affect the extremum, so the corresponding multiplier .
- For an active constraint (), it restricts the direction of the optimisation, and the gradient of the objective function must be a non-negative linear combination of the gradients of the active constraints ( guarantees the correct direction).
- The KKT conditions are the generalisation of the Lagrange multiplier method for equality constraints to the case of inequality constraints, and they are the foundation of nonlinear programming.
Definition
A definition is a precise description of what a concept, a term, or an object means. Understanding definitions is the foundation of learning any field, and every concept has its own exact meaning.
The primal problem: s.t.
The Lagrangian function:
The dual function:
The dual problem:
Weak duality theorem: the optimal value of the dual problem is the optimal value of the primal problem.
Strong duality theorem: in a convex optimisation problem (satisfying a constraint qualification), the optimal value of the dual problem equals that of the primal problem.
Example 1: Computing the Shadow Price
Under the constraint , maximise . Find the optimal value and the Lagrange multiplier , and verify that .
Approach:
The shadow price is the shadow value of the constraint, so the Lagrange multiplier is exactly the derivative of the optimal value with respect to the constant of the constraint — which is the identity to be verified here.
Steps:
- and , giving
- Substituting into the constraint: , so and
- The optimal value
- , while , which verifies
Answer: and , satisfying .
Example 2: Applying the KKT Conditions
Use the KKT conditions to solve s.t. .
Approach:
The KKT conditions turn the inequality constraint into a system of equations with complementary slackness, and the complementary slackness also tells you whether the constraint is “active” or “not active”.
Steps:
- Standardising the constraint:
- The KKT conditions:
- ⟹
- If : , but , which violates primal feasibility.
- So , and complementary slackness gives , that is,
- Combining with : , so and
- The minimum
Answer: the minimum value is , attained at .
Exercise 1
Use the KKT conditions to solve s.t. , .
Approach:
Standardise all the constraints in the form and write out the four KKT conditions; complementary slackness tells you which constraints are active.
Steps:
- The constraints: and ,
- Stationarity: ,
- The solution must have and ; by symmetry the optimum has
- If the circular constraint is active () and : with we get , so and
- Checking stationarity: gives , which violates (the maximum is a different case from the minimum)
- For a maximum, the sign convention flips: use with , and then ✓
- Comparing with the boundary points and , where , the maximum is
Answer: the maximum value is , attained at .
Exercise 2
Show that the shadow price of the constraint in the problem of maximising is .
Approach:
Find the optimal value as a function of the constant of the constraint and differentiate it.
Steps:
- Under and by symmetry, the optimum is
- The optimal value
- Hence the shadow price is (the magnitude of the Lagrange multiplier)
Answer: the shadow price is .
Symbols Used in This Article
| Symbol | Type | Pronunciation/Explanation | Meaning in This Article |
|---|---|---|---|
| Lagrange multiplier | lambda | The multiplier for an equality or inequality constraint | |
| Multiplier | mu | The multiplier for an equality constraint | |
| Lagrangian | Lagrangian | The Lagrangian function | |
| Dual function | dual function | The Lagrange dual function | |
| Optimal value | optimal value | The optimal value of the constrained optimisation | |
| Right-hand side of the constraint | constraint RHS | The right-hand side of the constraint | |
| Extremum point | extremum point | The extremum point of the optimisation |
