1: Introduction to Lagrange Multipliers

To avoid repetition, it is to be understood throughout that $f$ and $g_{1}$, $g_{2}$,…, $g_{m}$ are continuously differentiable on an open set $D$ in $\mathbb{R}^{n}$.

Suppose that $m<n$ and

$$\label{eq:1} g_{1}(\mathbf{X}) = g_2(\mathbf{X}) = \cdots = g_{m}(\mathbf{X})=0$$

on a nonempty subset $D_{1}$ of $D$. If  $\mathbf{X}_{0} \in D_{1}$ and there is a neighborhood $N$ of $\mathbf{X}_{0}$ such that

$$\label{eq:2} f(\mathbf{X}) \le f(\mathbf{X}_{0})$$

for every $\mathbf{X}$ in $N \cap D_{1}$, then $\mathbf{X}_{0}$ is a local maximum point of $f$ subject to the constraints Equation $\ref{eq:1}$. However, we will usually say “subject to” rather than “subject to the constraint(s).”

If Equation $\ref{eq:2}$ is replaced by

$$\label{eq:3} f(\mathbf{X}) \ge f(\mathbf{X}_{0}),$$

then “maximum” is replaced by “minimum.” A local maximum or minimum of $f$ subject to Equation $\ref{eq:1}$ is also called a local extreme point of $f$ subject to Equation $\ref{eq:1}$. More briefly, we also speak of constrained local maximum, minimum, or extreme points. If Equation $\ref{eq:2}$ or Equation $\ref{eq:3}$ holds for all $\mathbf{X}$ in $D_{1}$, we omit “local.”

Recall that ${\bf X}_{0}=(x_{10}, x_{20},\dots,x_{n0})$ is a critical point of a differentiable function $L=L(x_{1},x_{2},\dots,x_{n})$ if

$$L_{x_{i}}(x_{10},x_{20},\dots,x_{n0})=0,\quad 1\le i\le n. \nonumber$$

Therefore, every local extreme point of $L$ is a critical point of $L$; however, a critical point of $L$ is not necessarily a local extreme point of $L$.

Suppose that the system Equation $\ref{eq:1}$ of simultaneous equations can be solved for $x_{1}$, …, $x_{m}$ in terms of the $x_{m+1}$, …, $x_{n}$; thus,

$$\label{eq:4} x_{j}=h_{j}(x_{m+1},\dots,x_{n}),\quad 1\le j\le m.$$

Then a constrained extreme value of $f$ is an unconstrained extreme value of

$$\label{eq:5} f(h_{1}(x_{m+1},\dots,x_{n}),\dots,h_{m}(x_{m+1},\dots,x_{n}),x_{m+1},\dots,x_{n}).$$

However, it may be difficult or impossible to find explicit formulas for $h_{1}$, $h_{2}$, …, $h_{m}$, and, even if it is possible, the composite function Equation $\ref{eq:5}$ is almost always complicated. Fortunately, there is a better way to to find constrained extrema, which also requires the solvability assumption, but does not require an explicit formula as indicated in Equation $\ref{eq:4}$. It is based on the following theorem. Since the proof is complicated, we consider two special cases first.

Theorem $\PageIndex{1}$

Nov 2, 2019, 10:24 PM

[theorem:1]

Suppose that $n>m.$ If  ${\bf X}_{0}$ is a local extreme point of $f$ subject to

$$g_{1}({\bf X})=g_{2}({\bf X})=\cdots =g_{m}({\bf X})=0 \nonumber$$

and

$$\label{eq:6} \left|\begin{array}{ccccccc} \displaystyle{\frac{\partial{g_{1}(\mathbf{X}_{0})}}{\partial{x_{r_{1}}}}} & \displaystyle{\frac{\partial{g_{1}(\mathbf{X}_{0})}}{\partial{x_{r_{2}}}}}& &\cdots & \displaystyle{\frac{\partial{g_{1}(\mathbf{X}_{0})}}{\partial{x_{r_{m}}}}} \\\\ \displaystyle{\frac{\partial{g_{2}(\mathbf{X}_{0})}}{\partial{x_{r_{1}}}}} & \displaystyle{\frac{\partial{g_{2}(\mathbf{X}_{0})}}{\partial{x_{r_{2}}}}}& &\cdots & \displaystyle{\frac{\partial{g_{m}(\mathbf{X}_{0})}}{\partial{x_{r_{m}}}}} & \\ \vdots & \vdots &&\ddots&\vdots\\ \displaystyle{\frac{\partial{g_{m}(\mathbf{X}_{0})}}{\partial{x_{r_{1}}}}} & \displaystyle{\frac{\partial{g_{m}(\mathbf{X}_{0})}}{\partial{x_{r_{2}}}}}& &\cdots & \displaystyle{\frac{\partial{g_{m}(\mathbf{X}_{0})}}{\partial{x_{r_{m}}}}} & \end{array}\right|\ne0$$

for at least one choice of $r_{1}<r_{2}<\dots <r_{m}$ in $\{1,2,\dots,n\},$ then there are constants $\lambda_{1},$ $\lambda_{2},$ …$,$ $\lambda_{m}$ such that ${\bf X}_{0}$ is a critical point of

$$f-\lambda_{1}g_{1}-\lambda_{2}g_{2}-\cdots-\lambda_{m} g_{m}; \nonumber$$

that is$,$

$$\frac{\partial{f({\bf X}_{0})}}{\partial x_{i}} -\lambda_{1}\frac{\partial{g_{1}({\bf X}_{0})}}{\partial x_{i}} -\lambda_{2}\frac{\partial{g_{2}({\bf X}_{0})}}{\partial x_{i}}-\cdots -\lambda_{m}\frac{\partial{g_{m}({\bf X}_{0})}}{\partial x_{i}}=0, \nonumber$$

$1\le i\le n$.

The following implementation of this theorem is the method of Lagrange multipliers.

method of Lagrange multipliers

1. Find the critical points of $$f-\lambda_{1}g_{1}-\lambda_{2}g_{2}-\cdots-\lambda_{m} g_{m}, \nonumber$$ treating $\lambda_{1}$, $\lambda_{2}$, …$\lambda_{m}$ as unspecified constants.
2. Find $\lambda_{1}$, $\lambda_{2}$, …, $\lambda_{m}$ so that the critical points obtained in (a) satisfy the constraints.
3. Determine which of the critical points are constrained extreme points of $f$. This can usually be done by physical or intuitive arguments.
4. If $a$ and $b_{1}$, $b_{2}$, …, $b_{m}$ are nonzero constants and $c$ is an arbitrary constant, then the local extreme points of $f$ subject to $g_{1}=g_{2}= \cdots =g_{m}=0$ are the same as the local extreme points of $af-c$ subject to $b_{1}g_{1}=b_{2}g_{2}=\cdots=b_{m}g_{m}=0$. Therefore, we can replace $f-\lambda_{1} g_{1}-\lambda_{2}g_{2}- \cdots-\lambda_{m} g_{m}$ by $af-\lambda_{1}b_{1}g_{1}-\lambda_{2}b_{2}g_{2}- \cdots- \lambda_{m}b_{m}g_{m}-c$ to simplify computations. (Usually, the “$-c$” indicates dropping additive constants.) We will denote the final form by $L$ (for Lagrangian).