2.7: Constrained Optimization - Lagrange Multipliers
( \newcommand{\kernel}{\mathrm{null}\,}\)
In Sections 2.5 and 2.6 we were concerned with finding maxima and minima of functions without any constraints on the variables (other than being in the domain of the function). What would we do if there were constraints on the variables? The following example illustrates a simple case of this type of problem.
Example 2.24
For a rectangle whose perimeter is 20 m, find the dimensions that will maximize the area.
Solution
The area A of a rectangle with width x and height y is A=xy. The perimeter P of the rectangle is then given by the formula P=2x+2y. Since we are given that the perimeter P=20, this problem can be stated as:
Maximize : f(x,y)=xygiven : 2x+2y=20
The reader is probably familiar with a simple method, using single-variable calculus, for solving this problem. Since we must have 2x+2y=20, then we can solve for, say, y in terms of x using that equation. This gives y=10−x, which we then substitute into f to get f(x,y)=xy=x(10−x)=10x−x2. This is now a function of x alone, so we now just have to maximize the function f(x)=10x−x2 on the interval [0,10]. Since f′(x)=10−2x=0⇒x=5 and f′′(5)=−2<0, then the Second Derivative Test tells us that x=5 is a local maximum for f, and hence x=5 must be the global maximum on the interval [0,10] (since f=0 at the endpoints of the interval). So since y=10−x=5, then the maximum area occurs for a rectangle whose width and height both are 5 m.
Notice in the above example that the ease of the solution depended on being able to solve for one variable in terms of the other in the equation 2x+2y=20. But what if that were not possible (which is often the case)? In this section we will use a general method, called the Lagrange multiplier method, for solving constrained optimization problems:
Maximize (or minimize) : f(x,y)(or f(x,y,z))given : g(x,y)=c(or g(x,y,z)=c) for some constant c
The equation g(x,y)=c is called the constraint equation, and we say that x and y are constrained by g(x,y)=c. Points (x,y) which are maxima or minima of f(x,y) with the condition that they satisfy the constraint equation g(x,y)=c are called constrained maximum or constrained minimum points, respectively. Similar definitions hold for functions of three variables.
The Lagrange multiplier method for solving such problems can now be stated:
Theorem 2.7: The Lagrange Multiplier Method
Let f(x,y) and g(x,y) be smooth functions, and suppose that c is a scalar constant such that ∇g(x,y)≠0 for all (x,y) that satisfy the equation g(x,y)=c. Then to solve the constrained optimization problem
Maximize (or minimize) : f(x,y)given : g(x,y)=c,
find the points (x,y) that solve the equation ∇f(x,y)=λ∇g(x,y) for some constant λ (the number λ is called the Lagrange multiplier). If there is a constrained maximum or minimum, then it must be such a point.
A rigorous proof of the above theorem requires use of the Implicit Function Theorem, which is beyond the scope of this text. Note that the theorem only gives a necessary condition for a point to be a constrained maximum or minimum. Whether a point (x,y) that satisfies ∇f(x,y)=λ∇g(x,y) for some λ actually is a constrained maximum or minimum can sometimes be determined by the nature of the problem itself. For instance, in Example 2.24 it was clear that there had to be a global maximum.
So how can you tell when a point that satisfies the condition in Theorem 2.7 really is a constrained maximum or minimum? The answer is that it depends on the constraint function g(x,y), together with any implicit constraints. It can be shown that if the constraint equation g(x,y)=c (plus any hidden constraints) describes a bounded set B in R2, then the constrained maximum or minimum of f(x,y) will occur either at a point (x,y) satisfying ∇f(x,y)=λ∇g(x,y) or at a “boundary” point of the set B.
In Example 2.24 the constraint equation 2x+2y=20 describes a line in R2, which by itself is not bounded. However, there are “hidden” constraints, due to the nature of the problem, namely 0≤x,y≤10, which cause that line to be restricted to a line segment in R2 (including the endpoints of that line segment), which is bounded.
Example 2.25
For a rectangle whose perimeter is 20 m, use the Lagrange multiplier method to find the dimensions that will maximize the area.
Solution
As we saw in Example 2.24, with x and y representing the width and height, respectively, of the rectangle, this problem can be stated as:
Maximize : f(x,y)=xygiven : g(x,y)=2x+2y=20
Then solving the equation ∇f(x,y)=λ∇g(x,y) for some λ means solving the equations ∂f∂x=λ∂g∂x and ∂f∂y=λ∂g∂y, namely:
y=2λ,x=2λ
The general idea is to solve for λ in both equations, then set those expressions equal (since they both equal λ) to solve for x and y. Doing this we get
y2=λ=x2⇒x=y,
so now substitute either of the expressions for x or y into the constraint equation to solve for x and y:
20=g(x,y)=2x+2y=2x+2x=4x⇒x=5⇒y=5
There must be a maximum area, since the minimum area is 0 and f(5,5)=25>0, so the point (5,5) that we found (called a constrained critical point) must be the constrained maximum.
∴ The maximum area occurs for a rectangle whose width and height both are 5 m.
Example 2.26
Find the points on the circle x2+y2=80 which are closest to and farthest from the point (1,2).
Solution
The distance d from any point (x,y) to the point (1,2) is
d=√(x−1)2+(y−2)2,
and minimizing the distance is equivalent to minimizing the square of the distance. Thus the problem can be stated as:
Maximize (and minimize) : f(x,y)=(x−1)2+(y−2)2given : g(x,y)=x2+y2=80
Solving ∇f(x,y)=λ∇g(x,y) means solving the following equations:
2(x−1)=2λx,2(y−2)=2λy
Note that x≠0 since otherwise we would get −2 = 0 in the first equation. Similarly, y≠0. So we can solve both equations for λ as follows:
x−1x=λ=y−2y⇒xy−y=xy−2x⇒y=2x
Substituting this into g(x,y)=x2+y2=80 yields 5x2=80, so x=±4. So the two constrained critical points are (4,8) and (−4,−8). Since f(4,8)=45 and f(−4,−8)=125, and since there must be points on the circle closest to and farthest from (1,2), then it must be the case that (4,8) is the point on the circle closest to (1,2) and (−4,−8) is the farthest from (1,2) (see Figure 2.7.1).

Notice that since the constraint equation x2+y2=80 describes a circle, which is a bounded set in R2, then we were guaranteed that the constrained critical points we found were indeed the constrained maximum and minimum.
The Lagrange multiplier method can be extended to functions of three variables.
Example 2.27
Maximize (and minimize) : f(x,y,z)=x+zgiven : g(x,y,z)=x2+y2+z2=1
Solution
Solve the equation ∇f(x,y,z)=λ∇g(x,y,z):
1=2λx0=2λy1=2λz
The first equation implies λ≠0 (otherwise we would have 1 = 0), so we can divide by λ in the second equation to get y=0 and we can divide by λ in the first and third equations to get x=12λ=z. Substituting these expressions into the constraint equation g(x,y,z)=x2+y2+z2=1 yields the constrained critical points (1√2,0,1√2) and (−1√2,0,−1√2). Since f(1√2,0,1√2)>f(−1√2,0,−1√2), and since the constraint equation x2+y2+z2=1 describes a sphere (which is bounded) in R3, then (1√2,0,1√2) is the constrained maximum point and (−1√2,0,−1√2) is the constrained minimum point.
So far we have not attached any significance to the value of the Lagrange multiplier λ. We needed λ only to find the constrained critical points, but made no use of its value. It turns out that λ gives an approximation of the change in the value of the function f(x,y) that we wish to maximize or minimize, when the constant c in the constraint equation g(x,y)=c is changed by 1.
For example, in Example 2.25 we showed that the constrained optimization problem
Maximize : f(x,y)=xygiven : g(x,y)=2x+2y=20
had the solution (x,y)=(5,5), and that λ=x2=y2. Thus, λ=2.5. In a similar fashion we could show that the constrained optimization problem
Maximize : f(x,y)=xygiven : g(x,y)=2x+2y=21
has the solution (x,y)=(5.25,5.25). So we see that the value of f(x,y) at the constrained maximum increased from f(5,5)=25 to f(5.25,5.25)=27.5625, i.e. it increased by 2.5625 when we increased the value of c in the constraint equation g(x,y)=c from c=20 to c=21. Notice that λ=2.5 is close to 2.5625, that is,
λ≈∇f=f(new max. pt)−f(old max. pt)
Finally, note that solving the equation ∇f(x,y)=λ∇g(x,y) means having to solve a system of two (possibly nonlinear) equations in three unknowns, which as we have seen before, may not be possible to do. And the 3-variable case can get even more complicated. All of this somewhat restricts the usefulness of Lagrange’s method to relatively simple functions. Luckily there are many numerical methods for solving constrained optimization problems, though we will not discuss them here.