The Lagrange Multiplier Method
Definition
A definition is a precise description of what a concept, a term, or an object means. Understanding definitions is the foundation of learning any field, and every concept has its own exact meaning.
Finding the extrema of the function (called the objective function) subject to the constraint is called a constrained extremum (or a constrained optimisation).
More generally, finding the extrema of a function of variables subject to constraints
is the general constrained extremum problem.
Unconstrained versus Constrained Extrema
- Unconstrained extremum: the independent variables vary freely within the domain, with no extra restriction.
- Constrained extremum: the independent variables must satisfy certain constraints, and the extremum points are restricted to the constraint surface.
There are two methods for solving a constrained extremum:
- The substitution method: solve the constraint for some of the variables, substitute into the objective function, and reduce it to an unconstrained extremum. This is hard to use when the constraint is complicated.
- The Lagrange multiplier method: by introducing a Lagrange multiplier, the constrained extremum is turned into solving a system of equations. This is the more general method.
Theorem
A theorem is a proposition that has been rigorously proved, and it is the foundation of mathematical reasoning. A theorem usually states its conditions and its conclusion, and is an important part of any mathematical system.
To find the possible extremum points of the objective function subject to the constraint , construct the Lagrangian function:
where is called the Lagrange multiplier. Set all the first-order partial derivatives of equal to zero:
The obtained by solving this system are the possible constrained extremum points.
A Geometric Explanation
The Geometric Meaning of the Lagrange Multiplier Method
At a constrained extremum point, the gradient of the objective function is parallel to the normal vector of the constraint curve, that is,
This means that at the extremum point the level curve of the objective function is tangent to the constraint curve — if it were not tangent, then moving along the constraint curve could still make the value of the objective function increase or decrease, and the point would not be an extremum point.
The meaning of the Lagrange multiplier : expresses the rate of change of the extremal value of the objective function when the constraint changes slightly, that is, (where is the extremum under the constraint ); for this reason is also called the shadow price.
Theorem
A theorem is a proposition that has been rigorously proved, and it is the foundation of mathematical reasoning. A theorem usually states its conditions and its conclusion, and is an important part of any mathematical system.
To find the possible extremum points of the objective function subject to the two constraints and , construct the Lagrangian function:
Set all the first-order partial derivatives equal to zero:
Solving the system gives the possible constrained extremum points.
In general, constraints call for Lagrange multipliers .
Deciding on the Extremum Points
The Lagrange multiplier method only gives the possible extremum points (stationary points); whether they really are extremum points has to be decided further:
- Practical problems: judge from the nature of the problem. If the maximum (or minimum) is certain to exist and there is only one stationary point, then that stationary point is the extremum point.
- The second-order condition: this can be done via the positive definiteness of the Hessian matrix of the Lagrangian restricted to the tangent space of the constraint (rather complicated, and generally not required in a course).
- Substitution and comparison: substitute the constraint into the objective function, reduce it to an unconstrained extremum, and apply the second-derivative test.
- Comparison in a neighbourhood: take other points on the constraint curve near the stationary point and compare the values of the function.
Example 1: A Constrained Extremum with a Single Constraint
Find the extrema of the function subject to the constraint .
Approach:
Write the constraint as , construct the Lagrangian function, set its partial derivatives with respect to the two variables to zero, and solve the system together with the constraint.
Steps:
- The constraint
- The Lagrangian function:
- , and
- From the first two equations: ; substituting into the third: and
- The stationary point , and
- Deciding: substituting gives , a downward-opening parabola, which attains its maximum value at .
Answer: the constrained local maximum is (at ), and there is no constrained minimum (as , ).
Example 2: A Constrained Extremum with Two Constraints
Find the extrema of the function subject to the constraints and .
Approach:
With two constraints you introduce two multipliers; the partial derivatives of the Lagrangian with respect to the three variables, together with the two constraint equations, give five equations to solve.
Steps:
- The constraints and
- From :
- From : ; if then
- Substituting into : , that is,
- Substituting and and simplifying gives the concrete solutions
- The stationary points: , with ; and and , with
- Comparing: the maximum is and the minimum is
Answer: the constrained local maximum is (at ), and the constrained local minimum is (at ).
Example 3: The Extreme Distances
Find the longest and the shortest distance from the origin to the curve .
Approach:
Minimising the distance is equivalent to minimising the square of the distance, which avoids the square root and makes the derivatives cleaner.
Steps:
- The objective function: the square of the distance (minimising the distance is equivalent to minimising its square)
- The constraints: and
- , and
- From the first two: , so or
- If : from we get and
- :
- :
- If : from we get , from we get , and from and we get , that is, or , with
- Comparing: the shortest distance , and the longest distance
Answer: the shortest distance is , and the longest distance is .
Exercise 1
Find the minimum of subject to the constraint .
Approach:
The objective function is exactly the square of the distance from the origin to the line, so the minimum is the square of that distance.
Steps:
- and
- and ; substituting into the constraint: , so
- and
- The minimum
Answer: the minimum is (this is the square of the distance from the origin to the line ).
Exercise 2
Find the largest volume of a rectangular box whose surface area is .
Approach:
When the surface area is fixed, the volume is largest; by symmetry one expects the optimum to be a cube, and the Lagrange multiplier method confirms it.
Steps:
- Let the length, width and height be ; the objective is and the constraint is
- , and
- From the first two: , which simplifies to ; similarly
- Hence , and substituting into the constraint: and
- The maximum volume
Answer: the maximum volume is (when the box is a cube).
Symbols Used in This Article
| Symbol | Type | Pronunciation/Explanation | Meaning in This Article |
|---|---|---|---|
| Lagrangian | Lagrangian | The Lagrangian function | |
| Lagrange multiplier | lambda, mu | The Lagrange multipliers | |
| Constraint function | constraint functions | The constraint functions | |
| Objective function | objective function | The function whose extremum is sought | |
| Gradient | gradient | The gradient of the constraint function |
