View source for Extreme value theorem

{{About|the calculus concept|the statistical concept|Fisher–Tippett–Gnedenko theorem}}
{{short description|Continuous real function on a closed interval has a maximum and a minimum}}
{{More footnotes|date=June 2012}}
[[Image:Extreme Value Theorem.svg|thumb|300px|A continuous function <math>f(x)</math> on the closed interval <math>[a, b]</math> showing the absolute max (red) and the absolute min (blue).]]
In [[calculus]], the '''extreme value theorem''' states that if a real-valued [[Function (mathematics)|function]] <math>f</math> is [[Continuous function|continuous]] on the [[Bounded interval#Classification of intervals|closed]] and [[Bounded set|bounded]] interval <math>[a,b]</math>, then <math>f</math> must attain a [[maximum]] and a [[minimum]], each at least once. 
That is, there exist numbers <math>c</math> and <math>d</math> in <math>[a,b]</math> such that:
<math display="block">f(c) \leq f(x) \leq f(d)\quad \forall x\in [a,b].</math>

The extreme value theorem is more specific than the related  '''boundedness theorem''', which states merely that a continuous function <math>f</math> on the closed interval <math>[a,b]</math> is [[Bounded function|bounded]] on that interval; that is, there exist real numbers <math>m</math> and <math>M</math> such that:
<math display="block">m \le f(x) \le M\quad \forall x \in [a, b].</math>
  
This does not say that <math>M</math> and <math>m</math> are necessarily the maximum and minimum values of <math>f</math> on the interval <math>[a,b],</math> which is what the extreme value theorem stipulates must also be the case.

The extreme value theorem is used to prove [[Rolle's theorem]].  In a formulation due to [[Karl Weierstrass]], this theorem states that a continuous function from a non-empty [[compact space]] to a [[subset]] of the [[real number]]s attains a maximum and a minimum.

==History==
The extreme value theorem was originally proven by [[Bernard Bolzano]] in the 1830s in a work ''Function Theory'' but the work remained unpublished until 1930. Bolzano's proof consisted of showing that a continuous function on a closed interval was bounded, and then showing that the function attained a maximum and a minimum value.  Both proofs involved what is known today as the [[Bolzano&ndash;Weierstrass theorem]].<ref>{{cite journal |first1=Paul |last1=Rusnock |first2=Angus |last2=Kerr-Lawson |title=Bolzano and Uniform Continuity |journal=Historia Mathematica |volume=32 |issue=3 |year=2005 |pages=303–311 |doi=10.1016/j.hm.2004.11.003 |doi-access= }}</ref>

==Functions to which the theorem does not apply==

The following examples show why the function domain must be closed and bounded in order for the theorem to apply.  Each fails to attain a maximum on the given interval.

# <math>f(x)=x </math> defined over <math>[0, \infty)</math> is not bounded from above.
# <math>f(x)= \frac{x}{1+x} </math> defined over <math>[0, \infty)</math> is bounded from below but does not attain its least upper bound <math>1</math>.
# <math>f(x)= \frac{1}{x}</math> defined over <math>(0,1]</math> is not bounded from above.
# <math>f(x) = 1-x</math> defined over <math>(0,1]</math> is bounded but never attains its least upper bound <math>1</math>.

Defining <math>f(0)=0</math> in the last two examples shows that both theorems require continuity on <math>[a,b]</math>.

==Generalization to metric and topological spaces==

When moving from the real line <math>\mathbb{R}</math> to [[metric spaces]] and general [[topological spaces]], the appropriate generalization of a closed bounded interval is a [[compact space|compact set]]. A set <math>K</math> is said to be compact if it has the following property: from every collection of [[open set]]s <math>U_\alpha</math> such that  <math display="inline">\bigcup U_\alpha \supset K</math>, a finite subcollection <math>U_{\alpha_1},\ldots,U_{\alpha_n}</math>can be chosen such that <math display="inline">\bigcup_{i=1}^n U_{\alpha_i} \supset K</math>. This is usually stated in short as "every open cover of <math>K</math> has a finite subcover". The [[Heine–Borel theorem]] asserts that a subset of the real line is compact if and only if it is both closed and bounded.  Correspondingly, a metric space has the [[Heine–Borel property]] if every closed and bounded set is also   compact.

The concept of a continuous function can likewise be generalized.  Given topological spaces <math>V,\ W</math>, a function <math>f:V\to W</math> is said to be continuous if for every open set <math>U\subset W</math>, <math>f^{-1}(U)\subset V</math> is also open.  Given these definitions, continuous functions can be shown to preserve compactness:<ref name=":0">{{Cite book|url=https://archive.org/details/1979RudinW|title=Principles of Mathematical Analysis|last=Rudin|first=Walter|publisher=McGraw Hill|year=1976|isbn=0-07-054235-X|location=New York|pages=89–90}}</ref>

{{Math theorem|If <math>V,\ W</math> are topological spaces, <math>f:V\to W</math> is a continuous function, and <math>K\subset V</math> is compact, then <math>f(K)\subset W</math> is also compact.}}

In particular, if <math>W = \mathbb{R}</math>, then this theorem implies that <math>f(K)</math> is closed and bounded for any compact set <math>K</math>, which in turn implies that <math>f</math> attains its [[Infimum and supremum|supremum]] and [[Infimum and supremum|infimum]] on any (nonempty) compact set <math>K</math>.  Thus, we have the following generalization of the extreme value theorem:<ref name=":0" />

{{Math theorem|If <math>K</math> is a nonempty compact set and <math>f:K\to \mathbb{R}</math> is a continuous function, then <math>f</math> is bounded and there exist <math>p,q\in K</math> such that <math>f(p)=\sup_{x\in K} f(x)</math> and <math>f(q) = \inf_{x\in K} f(x)</math>.
}}

Slightly more generally, this is also true for an upper semicontinuous function. (see [[compact space#Functions and compact spaces]]).

==Proving the theorems==

We look at the proof for the [[upper bound]] and the maximum of <math>f</math>. By applying these results to the function <math>-f</math>, the existence of the lower bound and the result for the minimum of <math>f</math> follows. Also note that everything in the proof is done within the context of the [[real numbers]].

We first prove the boundedness theorem, which is a step in the proof of the extreme value theorem. The basic steps involved in the proof of the extreme value theorem are:

# Prove the boundedness theorem.
# Find a sequence so that its [[Image (mathematics)|image]] converges to the [[supremum]] of <math>f</math>.
# Show that there exists a [[subsequence]] that converges to a point in the [[domain of a function|domain]].
# Use continuity to show that the image of the subsequence converges to the supremum.

===Proof of the boundedness theorem===
{{Math theorem
|name=Boundedness Theorem
|If <math>f(x)</math> is continuous on <math>[a,b],</math> then it is bounded on <math>[a,b].</math>}}
{{Math proof|
Suppose the function <math>f</math>  is not bounded above on the interval <math>[a,b]</math>.  Then, for every natural number <math>n</math>, there exists an <math>x_n \in [a,b]</math> such that <math>f(x_n)>n</math>. This defines a [[sequence]] <math>(x_n)_{n \in \mathbb{N}}</math>. Because <math>[a,b]</math> is bounded, the [[Bolzano–Weierstrass theorem]] implies that there exists a convergent subsequence <math>(x_{n_k})_{k \in \mathbb{N}}</math> of <math>({x_n})</math>. Denote its limit by <math>x</math>. As <math>[a,b]</math> is closed, it contains <math>x</math>.  Because <math>f</math> is continuous at <math>x</math>, we know that <math>f(x_{{n}_{k}})</math> converges to the real number <math>f(x)</math> (as <math>f</math> is [[sequentially continuous]] at <math>x</math>). But <math>f(x_{{n}_{k}}) > n_k \geq k </math> for every <math>k</math>, which implies that <math>f(x_{{n}_{k}})</math> diverges to [[Extended real number line|<math>+ \infty </math>]], a contradiction. Therefore, <math>f</math> is bounded above on <math>[a,b]</math>.&nbsp;[[Q.E.D.|∎]]}}
{{Math proof
|title=Alternative proof
|Consider the set <math>B</math> of points <math>p</math> in <math>[a,b]</math> such that <math>f(x)</math> is bounded on <math>[a,p]</math>.  We note that <math>a</math> is one such point, for <math>f(x)</math> is bounded on <math>[a,a]</math> by the value <math>f(a)</math>.  If <math>e > a</math> is another point, then all points between <math>a</math> and <math>e</math> also belong to <math>B</math>.  In other words <math>B</math> is an interval closed at its left end by <math>a</math>.

Now <math>f</math> is continuous on the right at <math>a</math>, hence there exists <math>\delta>0</math> such that <math>|f(x) - f(a)| < 1</math> for all <math>x</math> in <math>[a,a+\delta]</math>. Thus <math>f</math> is bounded by <math>f(a) - 1</math> and <math>f(a)+1</math> on the interval <math>[a,a+\delta]</math> so that all these points belong to <math>B</math>.

So far, we know that <math>B</math> is an interval of non-zero length, closed at its left end by <math>a</math>.

Next, <math>B</math> is bounded above by <math>b</math>.  Hence the set <math>B</math> has a supremum in <math>[a,b]</math> ; let us call it <math>s</math>. From the non-zero length of <math>B</math> we can deduce that <math>s > a</math>.

Suppose <math>s<b</math>.  Now <math>f</math> is continuous at <math>s</math>, hence there exists <math>\delta>0</math> such that <math>|f(x) - f(s)| < 1</math> for all <math>x</math> in <math>[s-\delta,s+\delta]</math> so that <math>f</math> is bounded on this interval.  But it follows from the supremacy of <math>s</math> that there exists a point belonging to <math>B</math>, <math>e</math> say, which is greater than <math>s-\delta/2</math>.  Thus <math>f</math> is bounded on <math>[a,e]</math> which overlaps <math>[s-\delta,s+\delta]</math> so that <math>f</math> is bounded on <math>[a,s+\delta]</math>.  This however contradicts the supremacy of <math>s</math>.

We must therefore have <math>s=b</math>.  Now <math>f</math> is continuous on the left at <math>s</math>, hence there exists <math>\delta>0</math> such that <math>|f(x) - f(s)| < 1</math> for all <math>x</math> in <math>[s-\delta,s]</math> so that <math>f</math> is bounded on this interval.  But it follows from the supremacy of <math>s</math> that there exists a point belonging to <math>B</math>, <math>e</math> say, which is greater than <math>s-\delta/2</math>.  Thus <math>f</math> is bounded on <math>[a,e]</math> which overlaps <math>[s-\delta,s]</math> so that <math>f</math> is bounded on <math>[a,s]</math>.&nbsp;&nbsp; 
[[Q.E.D.|∎]]}}  

===Proofs of the extreme value theorem===

{{Math proof
|title=Proof of the Extreme Value Theorem
|proof=By the boundedness theorem, ''f'' is bounded from above, hence, by the  [[Dedekind-complete]]ness of the real numbers, the least upper bound (supremum) ''M'' of ''f'' exists. It is necessary to find a point ''d'' in [''a'', ''b''] such that ''M'' = ''f''(''d''). Let ''n'' be a natural number. As ''M'' is the ''least'' upper bound, ''M'' – 1/''n'' is not an upper bound for ''f''. Therefore, there exists ''d<sub>n</sub>'' in [''a'', ''b''] so that ''M'' – 1/''n'' < ''f''(''d<sub>n</sub>''). This defines a sequence {''d<sub>n</sub>''}. Since ''M'' is an upper bound for ''f'', we have ''M'' – 1/''n'' < ''f''(''d<sub>n</sub>'') ≤ ''M'' for all ''n''.  Therefore, the sequence {''f''(''d<sub>n</sub>'')} converges to ''M''.

The [[Bolzano–Weierstrass theorem]] tells us that there exists a subsequence {<math>d_{n_k}</math>}, which converges to some ''d'' and, as [''a'', ''b''] is closed, ''d'' is in [''a'', ''b'']. Since ''f'' is continuous at ''d'', the sequence {''f''(<math>d_{n_k}</math>)} converges to ''f''(''d''). But {''f''(''d<sub>n<sub>k</sub></sub>'')} is a subsequence of {''f''(''d<sub>n</sub>'')} that converges to ''M'', so ''M'' = ''f''(''d''). Therefore, ''f'' attains its supremum ''M'' at ''d''.&nbsp;
[[Q.E.D.|∎]]}}

{{Math proof
|title=Alternative Proof of the Extreme Value Theorem
|proof=The set {{math|1= {''y'' &isin; '''R''' : ''y'' = ''f''(''x'') for some ''x'' &isin; [''a'',''b'']}<nowiki/>}} is a bounded set. Hence, its [[least upper bound]] exists by [[least upper bound property]] of the real numbers. Let {{math|1=''M'' = sup(''f''(''x''))}}&nbsp;on&nbsp;{{closed-closed|''a'', ''b''}}.  If there is no point ''x'' on [''a'',&nbsp;''b''] so that ''f''(''x'')&nbsp;=&nbsp;''M'', then
{{math|''f''(''x'') < ''M''}} on [''a'',&nbsp;''b''].  Therefore, {{math|1/(''M'' &minus; ''f''(''x''))}} is continuous on [''a'', ''b''].

However, to every positive number ''&epsilon;'', there is always some ''x'' in [''a'',&nbsp;''b''] such that {{math|''M'' &minus; ''f''(''x'') < ''&epsilon;''}} because ''M'' is the least upper bound. Hence, {{math|1/(''M'' &minus; ''f''(''x'')) > 1/''&epsilon;''}}, which means that {{math|1/(''M'' &minus; ''f''(''x''))}} is not bounded.  Since every continuous function on [''a'', ''b''] is bounded, this contradicts the conclusion that {{math|1/(''M'' &minus; ''f''(''x''))}} was continuous on [''a'',&nbsp;''b''].  Therefore, there must be a point ''x'' in [''a'',&nbsp;''b''] such that ''f''(''x'')&nbsp;=&nbsp;''M''.
[[Q.E.D.|∎]]}}

===Proof using the hyperreals===
{{Math proof
|name=Proof of Extreme Value Theorem
|proof=In the setting of [[non-standard calculus]], let ''N''&thinsp; be an infinite [[hyperinteger]].  The interval [0,&nbsp;1] has a natural hyperreal extension.  Consider its partition into ''N'' subintervals of equal [[infinitesimal]] length 1/''N'', with partition points ''x<sub>i</sub>''&nbsp;= ''i''&nbsp;/''N'' as ''i'' "runs" from 0 to ''N''.  The function ''&fnof;''&thinsp; is also naturally extended to a function ''&fnof;''* defined on the hyperreals between 0 and 1.  Note that in the standard setting (when ''N''&thinsp; is finite), a point with the maximal value of ''&fnof;'' can always be chosen among the ''N''+1 points ''x<sub>i</sub>'', by induction.  Hence, by the [[transfer principle]], there is a hyperinteger ''i''<sub>0</sub> such that 0&nbsp;≤ ''i''<sub>0</sub>&nbsp;≤ ''N'' and <math>f^*(x_{i_0})\geq f^*(x_i)</math>&thinsp; for all ''i''&nbsp;=&nbsp;0,&nbsp;...,&nbsp;''N''.  Consider the real point
<math display="block">c = \mathbf{st}(x_{i_0})</math>
where '''st''' is the [[standard part function]].  An arbitrary real point ''x'' lies in a suitable sub-interval of the partition, namely <math>x\in [x_i,x_{i+1}]</math>, so that&thinsp; '''st'''(''x<sub>i</sub>'')&nbsp;= ''x''.  Applying '''st''' to the inequality <math>f^*(x_{i_0})\geq f^*(x_i)</math>, we obtain <math>\mathbf{st}(f^*(x_{i_0}))\geq \mathbf{st}(f^*(x_i))</math>.  By continuity of ''&fnof;''&thinsp; we have
:<math>\mathbf{st}(f^*(x_{i_0}))= f(\mathbf{st}(x_{i_0}))=f(c)</math>.
Hence ''&fnof;''(''c'')&nbsp;≥ ''&fnof;''(''x''), for all real ''x'', proving ''c'' to be a maximum of ''&fnof;''.<ref>{{cite book |last=Keisler |first=H. Jerome |title=Elementary Calculus : An Infinitesimal Approach |publisher=Prindle, Weber & Schmidt |location=Boston |year=1986 |isbn=0-87150-911-3 |url=https://www.math.wisc.edu/~keisler/chapter_3e.pdf#page=60 |page=164 }}</ref>
[[Q.E.D.|∎]]
}}

===Proof from first principles===

'''Statement'''  &nbsp;&nbsp;&nbsp;&nbsp; If <math>f(x)</math> is continuous on <math>[a,b]</math> then it attains its supremum on <math>[a,b]</math>

{{Math proof
|name=Proof of Extreme Value Theorem
|proof=By the Boundedness Theorem, <math>f(x)</math> is bounded above on <math>[a,b]</math> and by the completeness property of the real numbers has a supremum in <math>[a, b]</math>.  Let us call it <math>M</math>, or <math>M[a,b]</math>.  It is clear that the restriction of <math>f</math> to the subinterval <math>[a,x]</math> where <math>x\le b</math> has a supremum <math>M[a, x]</math> which is less than or equal to <math>M</math>, and that <math>M[a,x]</math> increases from <math>f(a)</math> to <math>M</math> as <math>x</math> increases from <math>a</math> to <math>b</math>.

If <math>f(a)=M</math> then we are done.  Suppose therefore that <math>f(a)<M</math> and let <math>d=M-f(a)</math>.   Consider the set <math>L</math> of points <math>x</math> in <math>[a,b]</math> such that <math>M[a,x]< M</math>.

Clearly  <math>a\in L</math> ; moreover if <math>e>a</math> is another point in <math>L</math> then all points between <math>a</math> and <math>e</math> also belong to <math>L</math> because <math>M[a,x]</math> is monotonic increasing.  Hence <math>L</math> is a non-empty interval, closed at its left end by <math>a</math>.

Now <math>f</math> is continuous on the right at <math>a</math>, hence there exists <math>\delta>0</math> such that <math>|f(x)-f(a)| < d/2</math> for all <math>x</math> in <math>[a,a+\delta]</math>. Thus <math>f</math> is less than <math>M-d/2</math> on the interval <math>[a,a+\delta]</math> so that all these points belong to <math>L</math>.

Next, <math>L</math> is bounded above by <math>b</math> and has therefore a supremum in <math>[a,b]</math>:  let us call it <math>s</math>.   We see from the above that <math>s > a</math>.  We will show that <math>s</math> is the point we are seeking i.e. the point where <math>f</math> attains its supremum, or in other words <math>f(s)=M</math>.

Suppose the contrary viz. <math>f(s)<M</math>.  Let <math>d=M-f(s)</math> and consider the following two cases:

# <u><math>s<b</math></u>.&nbsp;&nbsp; As <math>f</math> is continuous  at <math>s</math>, there exists <math>\delta>0</math> such that <math>|f(x)-f(s)| < d/2</math> for all <math>x</math> in <math>[s-\delta,s+\delta]</math>.    This means that <math>f</math> is less than <math>M-d/2</math> on the interval <math>[s-\delta,s+\delta]</math>.   But it follows from the supremacy of <math>s</math> that there exists a point, <math>e</math> say, belonging to <math>L</math> which is greater than <math>s-\delta</math>.  By the definition of <math>L</math>, <math>M[a,e]< M</math>.  Let  <math>d_1=M-M[a,e]</math> then for all <math>x</math> in <math>[a,e]</math>, <math>f(x)\le M-d_1</math>.  Taking <math>d_2</math> to be the minimum of <math>d/2</math> and <math>d_1</math>, we have <math>f(x)\le M-d_2</math> for all <math>x</math> in <math>[a,s+\delta]</math>. {{pb}} Hence <math>M[a,s+\delta]<M</math> so that <math>s+\delta \in L</math>.  This however contradicts the supremacy of <math>s</math> and completes the proof.
# <u><math>s=b</math></u>.&nbsp;&nbsp;  As <math>f</math> is continuous on the left at <math>s</math>, there exists <math>\delta>0</math> such that <math>|f(x)-f(s)| < d/2</math> for all <math>x</math> in <math>[s-\delta,s]</math>.    This means that <math>f</math> is less than <math>M-d/2</math> on the interval <math>[s-\delta,s]</math>.   But it follows from the supremacy of <math>s</math> that there exists a point, <math>e</math> say, belonging to <math>L</math> which is greater than <math>s-\delta</math>.  By the definition of <math>L</math>, <math>M[a,e]< M</math>.  Let  <math>d_1=M-M[a,e]</math> then for all <math>x</math> in <math>[a,e]</math>, <math>f(x)\le M-d_1</math>.  Taking <math>d_2</math> to be the minimum of <math>d/2</math> and <math>d_1</math>, we have <math>f(x)\le M-d_2</math> for all <math>x</math> in <math>[a,b]</math>.  This contradicts the supremacy of <math>M</math> and completes the proof. [[Q.E.D.|∎]]
}}

==Extension to semi-continuous functions==

If the continuity of the function ''f'' is weakened to [[semi-continuity]],
then the corresponding half of the boundedness theorem and the extreme value theorem hold and the values –∞ or +∞, respectively, from the [[extended real number line]] can be allowed as possible values.{{clarify|reason=It is not clear what is meant by " –∞ or +∞, respectively...can be allowed as possible values." What is the "respectively" in respect ''to''?|date=August 2024}}

A function <math>f : [a, b] \to [-\infty, \infty)</math> is said to be ''upper semi-continuous'' if <math display="block">\limsup_{y\to x} f(y) \le f(x) \quad \forall x \in [a, b].</math> 

{{Math theorem
|If a function {{math|''f'' : [''a'', ''b''] → {{closed-open|–∞, ∞}}}} is upper semi-continuous, then ''f'' is bounded above and attains its supremum.}}
{{Math proof
|proof=If <math>f(x) = - \infty</math> for all ''x'' in [''a'',''b''], then the supremum is also <math>-\infty</math> and the theorem is true. In all other cases, the proof is a slight modification of the proofs given above. In the proof of the boundedness theorem, the  upper semi-continuity of ''f'' at ''x'' only implies that the [[limit superior]] of the subsequence {''f''(''x<sub>n<sub>k</sub></sub>'')} is bounded above by ''f''(''x'') < ∞, but that is enough to obtain the contradiction. In the proof of the extreme value theorem, upper semi-continuity of ''f'' at ''d'' implies that the limit superior of the subsequence {''f''(''d<sub>n<sub>k</sub></sub>'')} is bounded above by ''f''(''d''), but this suffices to conclude that ''f''(''d'') = ''M''.&nbsp;[[Q.E.D.|∎]]
}}


Applying this result to &minus;''f'' proves a similar result for the infimums of lower semicontinuous functions. 
A function <math>f : [a, b] \to [-\infty, \infty)</math> is said to be ''lower semi-continuous'' if <math display="block">\liminf_{y\to x} f(y) \geq f(x)\quad \forall x \in [a, b].</math> 

{{Math theorem
|If a function {{math|''f'' : [''a'', ''b''] → {{open-closed|–∞, ∞}}}} is lower semi-continuous, then ''f'' is bounded below and attains its [[infimum]].
}}

A real-valued function is upper as well as lower semi-continuous, if and only if it is continuous in the usual sense. Hence these two theorems imply the boundedness theorem and the extreme value theorem.

==References==
{{Reflist}}

==Further reading==
* {{cite book |first=Robert A. |last=Adams |title=Calculus : A Complete Course |location=Reading |publisher=Addison-Wesley |year=1995 |isbn=0-201-82823-5 |pages=706–707 }}
* {{cite book |first1=M. H. |last1=Protter |author-link1=Murray H. Protter |first2=C. B. |last2=Morrey |author-link2=Charles B. Morrey Jr. |title=A First Course in Real Analysis |location=New York |publisher=Springer |year=1977 |isbn=0-387-90215-5 |pages=71–73 |chapter=The Boundedness and Extreme–Value Theorems |chapter-url=https://books.google.com/books?id=NgX3BwAAQBAJ&pg=PA71 }}

==External links==
* [http://www.cut-the-knot.org/fta/fta_note.shtml A Proof for extreme value theorem] at [[cut-the-knot]]
* [http://demonstrations.wolfram.com/ExtremeValueTheorem/ Extreme Value Theorem] by  Jacqueline Wandzura with additional contributions by Stephen Wandzura, the [[Wolfram Demonstrations Project]].
* {{MathWorld |title=Extreme Value Theorem |urlname=ExtremeValueTheorem}}
* [[Mizar system]] proof: http://mizar.org/version/current/html/weierstr.html#T15

{{Calculus topics}}

[[Category:Articles containing proofs]]
[[Category:Theorems in calculus]]
[[Category:Theorems in real analysis]]