Linear Supporting Form and Derivative

As is usual, if E as a vector space (over R) we will denote by E^∗ = Hom(E, R) its algebraic dual space. We will indifferently use both notations p(v) or hp, vi to denote the value of v ∈ E under the linear form p ∈ E^∗.

Definition 1.2.1 (Supporting Linear Form). We say that the linear form p ∈ E^∗ is a supporting linear form at x₀ ∈ U for the function f : U → R, defined on U ⊂ E, if we have

∀x ∈ U, f (x) − f (x0) ≥ p(x − x₀) = hp, x − x₀i.

We will denote by SLF_x(f ) the set of supporting linear form at x for f , and by SLF(f ) the graph

SLF(f ) = ∪_x∈U{x} × SLFx(f ) ⊂ U × E^∗.

In the literature, the linear form p is also called subderiva-tive of f at x₀ or even sometimes subgradient. We prefer to call it supporting linear form to avoid confusion with the notion of subdifferential that we will introduce in another chapter.

Example 1.2.2. a) If f : R → R, t 7→ |t| then SLF₀(f ) = [−1, 1], for t > 0, SLF_t(f ) = {1}, and for t < 0, SLF_t(f ) = {−1}.

b) If g : R → R, t 7→ t³ then SLFt(g) = ∅, for everyt ∈ R.

The following Proposition is obvious.

Proposition 1.2.3. The set SLF_x(f ) is a convex subset of E^∗. Moreover, if we endow E^∗with the topology of simple convergence on E (”weak topology”) then SLF_x(f ) is also closed.

Here is the relation between supporting linear form and deriva-tive.

Proposition 1.2.4. Let f : U → R be a function defined on an open subset U of the normed space E.

a) If f is differentiable at some given x ∈ U then SLF_x(f ) ⊂ {Df (x)}, i.e. it is either empty or equal to the singleton {Df (x)}.

b) If E = Rⁿ, and all partial derivatives ∂f /∂x_i(x), i = 1, . . . , n, at some given x ∈ U , then SLF_x(f ) is either empty or reduced to the single linear form (a₁, . . . , a_n) 7→Pn

i=1a_i∂f /∂x_i(x).

Proof. a) If SLF_x(f ) 6= ∅, let p be a supporting linear form of f at x. If v ∈ E is fixed, for all ǫ > 0 small we have x + ǫv ∈ U and thus f (x + ǫv) − f (x) ≥ ǫp(v). Dividing by ǫ and taking the limit as ǫ goes to 0 in this last inequality, we find Df (x)(v) ≥ p(v). For linear forms this implies equality, because a linear form which is

≥ 0 everywhere has to be 0.

b) We denote by (e₁, . . . , e_n) the canonical base in Rⁿ. Let us consider a point x = (x₁, . . . , x_n) ∈ Rⁿ where all partial deriva-tives exist. This implies that the function of one variable h 7→

f (x₁, . . . , x_i−1, h, x_i+1, . . . , x_n) is differentiable at x_i, hence by part a), if p ∈ SLF_x(f ), we have p(e_i) = ∂f /∂x_i(x). Since this is true for every i = 1, . . . , n, therefore the map p must be (a₁, . . . , a_n) 7→

P_n

i=1a_i∂f /∂x_i(x).

We have not imposed any continuity in the definition of a sup-porting linear form for a function f . This is indeed the case under very mild conditions on f , as we will see presently.

Proposition 1.2.5. Let U is an open subset of the topological vector space E, and let f : U → R be a function. Suppose that f is bounded from above on a neighborhood of x₀ ∈ U , then any supporting linear form of f at x₀ is continuous.

Proof. Let V be a neighborhood of 0 such that V = −V , and f is defined and bounded from above by K < +∞ on x₀+ V . Since V

9 is symmetrical around 0, for each v ∈ V , we have

p(v) ≤ f (x0+ v) − f (x0) ≤ 2K

−p(v) = p(−v) ≤ f (x0− v) − f (x0) ≤ 2K,

hence the linear form p is thus bounded on a nonempty open sub-set, it is therefore continuous.

As is customary, if E is a topological vector space, we will denote by E^′ ⊂ E^∗ the topological dual space of E, namely E^′ is the subset formed by the continuous linear forms.Of course E^′ = E^∗ if E is finite-dimensional. If E is a normed space, with norm k·k, then E^′ is also a normed space for the usual norm

kpk = sup{p(v) | v ∈ E, kvk ≤ 1}.

In the case of continuous map, we can improve Proposition 1.2.3.

Proposition 1.2.6. Suppose that f : U → R is a continuous function defined on the topological vector space E. If we endow E^′ with the topology of simple convergence on E (”weak topology”), then the graph SLF(f ) is a closed subset of U × E^′.

The proof of this Proposition is obvious.

Exercise 1.2.7. Let f : U → R be a locally bounded function defined on the open subset U of the normed space E. (Recall that locally bounded means that each point in U has a neighborhood on which the absolute value of f is bounded)

a) Show that for every x ∈ U , we can find a constant K, and a neighborhood V such that for every y ∈ V and every p ∈ SLF_y(f ) we have kpk ≤ K. [Indication: see the proof of Theorem 1.4.1]

b) If E is finite dimensional, and f is continuous, show the following continuity property: for every x ∈ U , and every neigh-borhood W of SLF_x(f ) in E^′ = E^∗, we can find a neighborhood V of x such that for every y ∈ V we have SLF_y(f ) ⊂ W .

As we will see now the notion of linear supporting form is tailored for convex functions.

Proposition 1.2.8. If the function f : U → R, defined on the convex subset U of the vector space E, admits a supporting linear form at every x ∈ U , then f is convex.

Proof. Let us suppose that x₀ = y + (1 − t)z with y, z ∈ U and t ∈ [0, 1]. If p is a supporting linear form at x₀, we have

f (y) − f (x₀) ≥ p(y − x₀) and f (z) − f (x₀) ≥ p(z − x₀), hence

tf (y) + (1 − t)f (z) − f (x₀) ≥ p(t(y − x₀) + (1 − t)(z − x₀))

= p(ty + (1 − t)z − x₀) = 0.

The following theorem is essentially equivalent to the Hahn-Banach Theorem.

Theorem 1.2.9. Let U be a convex open subset of the locally convex topological vector space E. If f : U → R is continuous and convex, then we can find a supporting linear form for f at each x ∈ U .

Proof. As f is continuous and convex, the set O = {(x, t) | x ∈ U , f (x) < t}

is open, non-empty, and convex in E × R. Since (x₀, f (x₀)) is not in O, by the Hahn-Banach Theorem, see [RV73, Theorem C, page 84] or [Rud91, Theorem, 3.4, page 59], there exists a continuous and non identically zero linear form α : E × R → R and such that

∀(x, t) ∈ O, α(x, t) > α(x0, f (x₀)).

We can write α(x, t) = p₀(x) + k₀t, with p₀: E → R a continuous linear form and k₀ ∈ R. Since α(x0, t) > α(x₀, f (x₀)) for all t > f (x₀), we see that k₀ > 0. If we define ˜p₀ = k⁻¹₀ p₀, we get

˜

p₀(x)+t ≥ ˜p₀(x₀)+f (x₀), for all t > f (x), therefore f (x)−f (x₀) ≥ (−˜p₀)(x − x₀). The linear form −˜p₀ is the supporting linear form we are looking for.

The following Proposition is a straightforward consequence of Theorem 1.2.9 and Proposition 1.2.4

11 Proposition 1.2.10. Let f : U → R be a continuous convex function defined on an open convex subset U of the normed space E. If f is differentiable at x₀ then the derivative Df (x₀) is the only supporting linear form of f at x₀. In particular, we have

∀x ∈ U, f (x) − f (x0) ≥ Df (x₀)(x − x₀).

Corollary 1.2.11. Let f : U → R be a continuous convex func-tion defined on an open convex subset U of a normed space. If f is differentiable at x₀, then x₀ is a global minimum if and only if Df (x₀) = 0.

Proof. Of course, if the derivative exists at minimum it must be 0, this is true even if f is not convex. The converse, which uses convexity, follows from the inequality f (y) − f (x₀) ≥ Df (x₀)(y − x₀) = 0 given by Proposition 1.2.10 above.

Corollary 1.2.12. If U ⊂ Rⁿ is open and convex and f : U → R is a convex function, then, for almost all x, the function f admits a unique supporting linear form at x.

Proof. This is a consequence of Proposition 1.2.10 above and Rade-macher’s Theorem 1.1.10.

Exercise 1.2.13. Let U be an open and convex subset of Rⁿ. Suppose that f : U → R is convex and continuous. Show that if f admits a unique supporting linear formp₀ at x₀ then Df (x₀) exists, and is equal to p₀. [Indication: For each x ∈ U \ 0, choose px∈ SLFx(f ), and prove that

p₍x − x₀) ≤ f (x) − f (x₀) ≤ p_x(x − x₀).

Conclude using exercise 1.2.7.