Noether Theorem in Stochastic Optimal Control Problems via Contact Symmetries

Article

Noether Theorem in Stochastic Optimal Control Problems via Contact Symmetries

Francesco C. De Vecchi^1,† , Elisa Mastrogiacomo^2,† , Mattia Turra^1,† and Stefania Ugolini^3,*^,†






Citation: De Vecchi, F.C.;

Mastrogiacomo, E.; Turra, M.; Ugolini, S. Noether Theorem in Stochastic Optimal Control Problems via Contact Symmetries. Mathematics 2021, 9, 953. https://doi.org/

10.3390/math9090953

Academic Editor: Giulia Di Nunno

Received: 8 February 2021 Accepted: 19 April 2021 Published: 24 April 2021

Publisher’s Note:MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affil- iations.

Copyright: © 2021 by the authors.

Licensee MDPI, Basel, Switzerland.

This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://

creativecommons.org/licenses/by/

4.0/).

1 Institute for Applied Mathematics & Hausdorff Center for Mathematics, Universität Bonn, Endenicher Allee 60, 53115 Bonn, Germany; francesco.devecchi@uni-bonn.de (F.C.D.V.);

mattia.turra@iam.uni-bonn.de (M.T.)

2 Dipartimento di Economia, Università degli Studi dell’Insubria, via Montegeneroso 71, 21100 Varese, Italy;

elisa.mastrogiacomo@uninsubria.it

3 Dipartimento di Matematica, Università degli Studi di Milano, Via Saldini 50, 20113 Milano, Italy

* Correspondence: stefania.ugolini@unimi.it

† These authors contributed equally to this work.

Abstract:We establish a generalization of the Noether theorem for stochastic optimal control prob- lems. Exploiting the tools of jet bundles and contact geometry, we prove that from any (contact) symmetry of the Hamilton–Jacobi–Bellman equation associated with an optimal control problem it is possible to build a related local martingale. Moreover, we provide an application of the theoreti- cal results to Merton’s optimal portfolio problem, showing that this model admits infinitely many conserved quantities in the form of local martingales.

Keywords: Noether theorem; stochastic optimal control; contact symmetries; Merton’s optimal portfolio problem

MSC:93E20; 58D19; 91G10; 60H15

1. Introduction

The concept of symmetry of ordinary or partial differential equations (ODEs and PDEs) was introduced by Sophus Lie at the end of the 19th century with the aim of extending the Galois theory from polynomial to differential equations. Actually, all the theory of Lie groups and algebras was developed by Lie himself as well as the principal tools for facing the problem of symmetries of differential equations (see [1] for an historical introduction to the subject and [2,3] for some modern presentations).

One of the most important applications of the study of symmetries in physical systems was provided by Emmy Noether. She understood that when an equation comes from a variational problem, such as in Lagrangian mechanics, general relativity or, more generally, field theory, it is possible to relate each symmetry of the equation to a conserved quantity, i.e., a function of the state of the system that does not change during the evolution of the dynamics, and conversely, to each conserved quantity it is possible to associate a symmetry of the motion. The simplest examples are, in Newtonian and Lagrangian mechanics, the conservation of energy, which is related to the invariance with respect to time translation, and the conservation of angular momentum, which is correlated to the invariance with respect to rotations. The classical Noether theorem (see, e.g., [2,4] for an exposition of the subject) has found many generalizations in deterministic optimal control theory (see, e.g., [5,6] and also [7–9] on the related problem of commuting Hamiltonians and Hamilton–

Jacobi multi-time equations).

The development of a Lie symmetry analysis for stochastic differential equations (SDEs) and general random systems is relatively recent (see, e.g., [10–20] for some recent developments in the non-variational case). For stochastic systems arising from a variational framework, it is certainly interesting to study the relation between their symmetries and

Mathematics 2021, 9, 953. https://doi.org/10.3390/math9090953 https://www.mdpi.com/journal/mathematics

functionals that are conserved by their flow, and, in particular, to establish stochastic generalizations of the Noether theorem.

The problem of finding some kinds of conservation laws for SDEs was discussed in various papers (see [21–30]). We could summarize three different approaches to this problem. The first one was considered by Misawa in [26,27,31], where the author studied the case in which some Markovian functions of solutions of SDEs are exactly conserved during time evolution.

The second approach was adopted by Zambrini and co-authors in a number of works.

They put themselves in the framework of Euclidean quantum mechanics, which repre- sents a geometrically consistent stochastic deformation of classical mechanics where a Gaussian noise is added to a classical system. This setting has a close connection with optimal transport and optimal control (see, e.g., [32] for an introduction to the topic). More precisely, in [29], a generalization of the Noether theorem has been proved: to any one- parameter symmetry of a variational problem it is possible to associate a martingale that is independent both from the initial and final condition of the system. This first step was quite important since it stressed that the suitable generalization of conserved quantities in a stochastic setting is not a function that remains constant during the time evolution of a stochastic system, but a function that is constant in mean. Another remarkable advance in the study of variational symmetries was achieved in [24,25,30], where it was noted that the symmetries of the Hamilton–Jacobi–Bellman (HJB) equation of the considered variational problem are the correct objects to be associated to the aforementioned martingales and the contact geometry is a good framework in which a stochastic version of the Noether theorem can be formulated. Indeed, to each Lie point symmetry of the HJB equation it is possible to associate a martingale for the evolution of the system. It is worth also mentioning the papers [21,28], where a suitable notion of integrable system, i.e., a system with a number of martingales and symmetries equal to the number of the dimension, is discussed.

The third approach was proposed by Baez and Fong in [22] (see also [23]). The authors showed a method to build martingales applying the action of symmetries to solution to backward Kolmogorov equation, that can be interpreted as a linear version of HJB equation obtained when the control and the objective function are trivial.

In our paper, we generalize at least along two directions the approach proposed by Zambrini and co-authors, as listed above. First, we work in a different optimal control setting that can be seen as a generalization of the variational framework described in their articles. Second, we do not only restrict to Lie point symmetries but we take advantage of the general notion of contact symmetry, namely a transformation preserving the contact structure of the jet space (see Section3).

We prove here a Noether theorem (Theorem12) that relates to any contact symmetry of the HJB equation associated with an optimal control problem, a martingale that is given by the generator of the contact symmetry. More precisely, if we consider the generator Ω(t, x, u, ux)of a contact symmetry (which is a regular function defined on the jet space J¹(Rⁿ,R), i.e., a map depending on a function u and on its first derivatives ux), a regular solution U(t, x)to the HJB equation and the solution Xtto the optimal control problem, then the process Ot=Ω(t, Xt, U(t, Xt),∇U(t, Xt)), obtained by composing the generator Ω with the function U and the process X, is a local martingale.

Furthermore, we generalize the Noether theorem also to the case where the coeffi- cients and the Lagrangian of the control problem are random. Indeed, we establish that the Noether theorem holds also in the case of stochastic HJB equation, introduced in [33]

by Peng to study the optimal control problem with stochastic final condition or stochas- tic Lagrangian, provided that we restrict ourselves to a subset of Lie point symmetries (Theorem13and Corollary1).

Finally, the present paper provides an application of our theory to a non-trivial interesting problem arising in mathematical finance, that is, Merton’s optimal portfolio problem. First proposed by Merton in [34], this model finds nowadays many different applications and generalizations (see [35] for a review of the original problem and various

generalizations and [36–39] for some more recent works on the subject). A particular form of the Noether theorem for this problem can be found in [40]. We show here that the HJB equation of this optimal control system admits infinitely many contact symmetries. It is important to notice that the contact symmetry generalization is essential in this specific problem, since, when we restrict to Lie point symmetries as it is done in the aforementioned literature, the equation admits only a finite number of infinitesimal invariants. The presence of infinitely many contact symmetries yields the possibility to construct infinitely many martingales whose means are preserved by the evolution of the system. Moreover, we also point out that, when the final condition is random or the coefficients of the evolution of the stock are general adapted processes, our stochastic generalization of the Noether theorem (Corollary1) allows us to construct some non-trivial martingales for this classical mathematical model. We think that the presence of these martingales could be related to the existence of many explicit solutions for Merton’s problem, and therefore we expect that the methods presented here can be used to build other explicit solutions for it. We plan to study in a future work the financial consequences of the conservation laws individuated in this paper.

Since the stochastic and geometrical frameworks are not so commonly put together, we also provide a concise introduction to both these subjects.

Plan of the Paper

The paper is organized as follows. Section 2 introduces stochastic optimal con- trol problems both in the deterministic and stochastic case, presenting also the HJB equation, and it is useful also to fix the notations that we adopt throughout the paper.

Contact symmetries and their properties in the PDEs setting are discussed in Section3.

Section4contains the main theoretical results of the paper, namely the Noether theorems for deterministic and stochastic HJB equations. The application of such results to Merton’s optimal portfolio problem is given in Section5.

2. A Brief Survey on Stochastic Optimal Control

We give here an overview of some results about stochastic control problems, referring the interested reader to [41–44] for further investigations on such results, though more precise references will be given throughout the section. The main aim of this section is to introduce the topics we will deal with and to give the tools from the stochastic optimal control theory that we will use later on in the paper.

2.1. Deterministic Optimal Control and Lagrange Mechanics

We start recalling some notions about deterministic optimal control and, in particu- lar, we focus on Lagrangian-type optimal control problems, i.e., problems arising from Lagrangian formulation of classical mechanics. More precisely, we consider a system of controlled ODEs of the form

dXⁱ_t=αⁱ_tdt, (1)

where X = (X¹, . . . , Xⁿ):[t0, T] → Rⁿis in C¹([t0, T],Rⁿ), t0, T∈ R, with t06T, are the initial time and the final time horizon, respectively, and α= (α¹, . . . , αⁿ) ∈C([t0, T],Rⁿ)is the control function. We want to maximize the following objective functional

J(t₀, x, α) = Z _T

t₀ L(X^t_s^0,x, αs)ds+g(X^t_T^0,x). (2) where X_t^t0,xis the solution to the ODE (1) such that X_t^t₀^0,x=x∈ Rⁿ.

We suppose that there exists only one smooth functionA:Rⁿ× Rⁿ → R^{nsuch that}

∑

Aⁱ(x, p)pi+L(x,A(x, p)) = sup

a∈Rⁿ

( _n

i=1

∑

aⁱpi+L(x, a) )

, (x, p) ∈ Rⁿ× Rⁿ,

and also that, for any x ∈ Rⁿ, the map A(x,·) := (A¹(x,·), . . . ,Aⁿ(x,·)) is smoothly invertible in all its variables as a function fromRⁿinto itself. Define then the PDE

ut−H(x, ux) =ut−

∑

Aⁱ(x, ux)u_xi+L(x,A(x, ux))

!

=0, (3)

where ux= (u_x1, . . . , uxⁿ). Equation (3) is usually referred to as Hamilton–Jacobi equation in the context of Lagrangian mechanics or Hamilton–Jacobi–Bellman equation in the context of optimal control theory.

We state now the deterministic version of the so-called verification theorem.

Theorem 1. Let U(t, x) ∈C¹([t₀, T] × Rⁿ,R)be a solution to Hamilton–Jacobi Equation (3).

Then the optimal control problem (1) with objective functional (2) admits a unique solution, for any x∈ Rⁿ, given, for every i=1, . . . , n, by

αⁱ_t= Aⁱ(Xt,∇U(t, Xt)), for every t∈ [t0, T]. Proof. See, e.g., Theorem 4.4 in [41].

Remark 1. It is important to note that, in the deterministic case and when U∈C^1,2([t0, T] × Rⁿ,R)^, i.e., U is differentiable one time with respect to time t and two times with respect to space x∈ Rⁿ, the function t7→α_tis C¹([t0, T],Rⁿ)and it satisfies the Euler–Lagrange equations

d

dt(∂_aiL(Xt, αt)) −∂_xiL(Xt, αt) =0, (4) where i=1, . . . , n.

2.2. Classical Stochastic Optimal Control Problem

An optimal control problem consists of maximizing an objective functional, depending on the state of a dynamical system, on which we can act through a control process.

Let K be a (convex) subset of R^d and fix a final time T > 0. Denote by W an m-dimensional Brownian motion on a filtered probability space(_Ω,F,(F_t)_t≥0,P)^{, where} (F_t)_t≥0is the natural filtration generated by W. We assume that the state of the system is modeled by the following stochastic differential equation (SDE)

(dXt=µ(t, Xt, αt)dt+σ(t, Xt, αt)dWt, t0<t≤T,

Xt₀ =x, (5)

where µ :R+× Rⁿ× R^d → Rⁿand σ :R+× Rⁿ× R^d→ R^n×mare measurable functions that are also Lipschitz-continuous on the set K, i.e., there exists a constant C>0, such that, for every t∈ R+, x, y∈ Rⁿ, a∈K,

|µ(t, x, a) −µ(t, y, a)| + kσ(t, x, a) −σ(t, y, a)k 6C|x−y|, (6) wherekσk²=tr(σ^∗σ).

We will also use the notation µ= (µⁱ)_i=1,...,nand σ= (σ<sub>`</sub><sup>i</sup>)i=1,...,n,`=1,...,m.

The control process α= (αs), appearing in (5), is a K-valued progressively measurable process with respect to the filtration(F_t)_t≥0. We denote byKthe set of control processes α such that

E

_{Z T}

0 (|µ(t, 0, αt)|²+ kσ(t, 0, αt)k²)dt



< +_∞. (7)

We call X_t^t0,x, t∈ [t0, T]the solution to the SDE (5).

Remark 2. Conditions(6) and (7) imply that, for any initial condition(t0, x) ∈ [0, T) × Rⁿand for all α∈ K, there exists a unique strong solution X_t^x,t0 to the SDE (5) (see, e.g., Theorem 2.2 in Chapter 4 of [45]).

Let L :R+× Rⁿ× R^d→ Rand g :Rⁿ → Rbe two measurable functions, such that g satisfies the quadratic growth condition|g(x)| 6C(1+ |x|²), for every x∈ Rⁿ, for some constant C independent of x.

For(t₀, x) ∈ [0, T) × Rⁿ, we denote byK_L(t₀, x)the subset of controls inKsuch that

E

Z _T

t₀ |L(t, X^t_t^0,x, αt)|dt



< +∞.

We consider an objective function of the following form

J(t0, x, α) = E

Z _T

t₀ L(s, X^ts^0,x, αs)ds+g(X^t_T^0,x)

 .

We are now in position to introduce the stochastic optimal control problem.

Definition 1. Fixed (t₀, x) ∈ [_{0, T}) × Rⁿ, the stochastic optimal control problem consists of maximizing the objective function J(t₀, x, α)over all α∈ K_L(t₀, x)subject to the SDE (5). The associated value function is then defined as

U(t0, x) = max

α∈KL(t₀,x)E

Z _T

t₀ L(t, X^t_t^0,x, αt)dt+g(X^t_T^0,x)

 .

Given an initial condition(t0, x) ∈ [0, T) × Rⁿ, we call α^∗∈ K_L(t0, x)an optimal control if J(t0, x, α^∗) =U(t0, x).

We call Hamilton–Jacobi–Bellman equation (HJB) the PDE







∂tϕ(t, x) +sup

a∈K

{L^a_tϕ(t, x) +L(t, x, a)} =0, (t, x) ∈ [t0, T) × Rⁿ, ϕ(T, x) =g(x), x∈ Rⁿ,

(8)

whereL^a_tis the Kolmogorov operator associated with Equation (5), namely, for ψ∈C²(Rⁿ),

L^a_tψ(x) =¹ 2

∑

η^ij(t, x, a)∂_xix^jψ(x) +

∑

µⁱ(t, x, a)∂_xiψ(x), (t, x, a) ∈ R+× Rⁿ×K,

with η^ijdefined, for every i, j∈ {1, . . . , n}, as

η^ij(t, x, a) = (σσ^>)^ij(t, x, a) =

∑

`=1

σ_{`</sub><sup>i</sup>(t, x, a)σ<sub>`}^j(t, x, a)_, (t, x, a) ∈ R+× Rⁿ×K.

We also write, for x∈ R^{n, p}∈ R^{nand q}∈ R^n×n,

H(t, x, p, q) =sup

a∈K

(1 2

∑

η^ij(t, x, a)q^ij+

∑

µⁱ(t, x, a)pⁱ+L(t, x, a) )

,

so that the HJB Equation (8) can be written also in the following way (

∂tϕ(t, x) +H(t, x,∇ϕ, D²ϕ) =0, (t, x) ∈ [t0, T) × Rⁿ,

ϕ(T, x) =g(x), x∈ Rⁿ. (9)

We state here the classical verification theorem.

Theorem 2. Let ϕ∈C^1,2([0, T) × Rⁿ) ∩C⁰([0, T] × Rⁿ)be a solution to the HJB Equation (9) for t0=0, satisfying the following quadratic growth, for some constant C,

|ϕ(t, x)| 6C(₁+ |x|²)_, for all(t, x) ∈ [_{0, T}] × Rⁿ. (10) Suppose that there exists a measurable function A^∗(t, x),(t, x) ∈ [0, T) × Rⁿ, taking values in K, such that

(i) We have

0=∂tϕ(t, x) +H(t, x,∇ϕ, D²ϕ) =∂tϕ(t, x) + L_t^A∗(t,x)ϕ(t, x) +L(t, x, A^∗(t, x)), (ii) The SDE

dXs =µ(s, Xs, A^∗(s, Xs))ds+σ(s, Xs, A^∗(s, Xs))dWs, with initial condition Xt=x, admits a unique solution X_s^∗,

(iii) The process A^∗(s, X^∗_s), s∈ [t, T]lies inK_L(t, x). Then

ϕ(t, x) =U(t, x)_, (t, x) ∈ [_{0, T}] × Rⁿ,

and A^∗(·, X^∗·)is an optimal control for the stochastic optimal control problem in Definition1.

Proof. See, e.g., Theorem 3.5.2 in [42]. Some other references for the verification theorem are also Theorem 4.1 in [41], Theorem 5.7 in [43], and Theorem 4.1 in [44].

Remark 3. The quadratic growth condition(10) is used in Theorem2only to guarantee that the local martingale part of the semi-martingale decomposition of ϕ(t, Xt), namely, by Itô formula,

∑

∑

`=1 Z _t

0 σ<sub>`</sub><sup>i</sup>(s, Xs, A<sup>∗</sup>(s, Xs))∂<sub>x</sub>iϕ(s, Xs)dW<sub>s</sub><sup>`</sup>, (11) is in L¹and a martingale (and not only a local martingale). This means that the statement of Theorem2holds assuming only that (11) is a L¹martingale, i.e., without condition (10).

2.3. Stochastic Hamilton–Jacobi–Bellman Equation

The present section generalizes the aforementioned Hamilton–Jacobi–Bellman equa- tion to its stochastic counterpart. Let us first recall the Itô–Kunita formula.

Theorem 3(Itô–Kunita formula). Let F(t, x),(t, x) ∈ [0, T] × Rⁿbe a random field that is continuous in(t, x)almost surely, such that

(i) For every t∈ [0, T], F(t,·)is a C²-map fromR^nintoR^,P^-a.s.,

(ii) For each x∈ R^{n, F}(·, x)is a continuous semi-martingaleP-a.s., and it satisfies

F(t, x) =F(0, x) +

∑

Z _t

0 f^j(s, x)dY_s^j, for every(t, x) ∈ [0, T] × Rⁿ, a.s., where Ys^j, j=1, . . . , m, are m continuous semi-martingales, f^j(s, x), x∈ Rⁿ, s∈ [0, T], are random fields that are continuous in(s, x)and satisfy the following properties:

(a) For every s∈ [0, T], f^j(s,·)is a C²-map fromR^ntoR^,P^-a.s., (b) For every x∈ Rⁿ, f^j(·, x)is an adapted process.

Let Xt= (X_t¹, . . . , Xⁿ_t)be continuous semi-martingales. Then we have, for t∈ [0, T],

F(t, Xt) =F(0, X0) +

∑

Z _t

0 f^j(s, Xs)dYs^j+

∑

Z _t

0 ∂_xiF(s, Xs)dX_sⁱ +

∑

∑

Z _t

0 ∂_xif^j(s, Xs)d[Y^j, Xⁱ]_s+

∑

Z _t

0 ∂_xixkF(s, Xs)d[Xⁱ, X^k]_s, where[·,·]_s stands for the quadratic variation of semi-martingales. Furthermore, if F ∈C³and

f^j∈C³,P-a.s., then we have, for i=1, . . . , n,

∂_xiF(t, x) =∂_xiF(_{0, x}) +

∑

Z _t

0 ∂_xif^j(s, x)_dYs^j_, for every(t, x) ∈ [_{0, T}] × Rⁿ, P^-a.s.

Proof. See, e.g., the article [46] or the book [47], both by H. Kunita.

Sticking, where possible, with the notation introduced in Section2.2, we consider a stochastic optimal control problem where also the functions L, g, µ, and σ are random. More precisely, they depend also on ω ∈Ω in a predictable way, namely, L(t, x, a,·), g(x,·), µ(t, x,·), and σ(t, x,·)areF_t-measurable, for any(t, x, a) ∈ R+× Rⁿ×K. In order to distinguish them from the functions in the previous section and recall that the following are stochastic terms, we write also

L^S(t, x, a) =L(t, x, a,·), g^S(x) =g(x,·). We want then to maximize the objective functional

E

_{Z T}

t₀ L^S(t, Xt, αt)dt+g^S(XT)



, (12)

where X solves the SDE

(dXt=µ(t, Xt, αt, ω)dt+σ(t, Xt, αt, ω)dWt, t0<t<T,

Xt₀ =x. (13)

and α∈ K_L.

Let us introduce, in a completely analogous way as in the previous section, the value function

U(t, x, ω) =max

α∈KL

E

Z _T

t L^S(s, Xs, αs)ds+g^S(XT)    

Ft

 .

From now on, we may omit the explicit dependence on ω ∈ Ω of the functions.

Then, for any fixed x, U(t, x)is anFt-adapted process but, a priori, it is not of bounded variation. We can anyway expect that it is a continuous semi-martingale, and therefore, by

the representation theorem for semi-martingales and martingales (see, e.g., Section IV.31 and Section IV.36 in [48]), that it can be written as follows,

U(t, x) =ΓT(x) −Γt(x) − Z _T

t Ys(x)dWs, x∈ Rⁿ, 06t6T,

where, for every x∈ Rⁿ,Γt(x)_{and Yt}(x)_areFt-adapted processes andΓt(x)is of bounded variations. In this case, ifΓt(x)and Yt(x)are almost surely continuous in(t, x),Γt(x)is differentiable with respect to t, and both of them are sufficiently smooth with respect to x, then the pair(U, Y)should satisfy a stochastic Hamilton–Jacobi–Bellman equation (SHJB).

More precisely, we say that(ϕ,Ψ)solves the SHJB related with the optimal control problem (12) and (13) if(ϕ,Ψ)satisfies the following backward stochastic partial differen- tial equation







dϕt(x) +H^S(r, x,∇ϕ, D²ϕ,∇_Ψ)dt=

∑

`=1

Ψ^{`</sup>t(x)dW<sub>t</sub><sup>`}, (t, x) ∈ [t0, T) × Rⁿ,

ϕ_T(x) =g(x), x∈ Rⁿ,

(14)

where

H^S(t, x, ux, uxx, ψx) ≡H(t, x, ux, uxx, ψx, ω) =sup

a∈K

H^S(t, x, ux, uxx, a, ψx),

and

H^S(t, x, ux, uxx, a, ψx) =

∑

µⁱ(t, x, a, ω)u_xi+¹ 2

∑

∑

`=1

σ_{`</sub><sup>i</sup>(t, x, a, ω)σ<sub>`}^j(t, x, a, ω)u_xix^j

+

∑

∑

`=1

σ<sub>`</sub><sup>i</sup>(t, x, a, ω)ψ<sub>x</sub><sup>`</sup>i +L^S(t, x, a),

with ψx= (ψ<sup>`</sup><sub>xi</sub>)i=1,...,n,`=1,...,m∈ R^n×m. See, e.g., Section 3.1 in [33] for more details about the derivation of Equation (14) and Section 4 in the same reference for results concerning the well-posedness of such an equation.

We state here the verification theorem, which tells us that a sufficiently smooth solution of the SHJB equation coincides with the value function v.

Theorem 4. Let(ϕ,Ψ)be a smooth solution of the SHJB Equation (14) with t0=0 and assume that the following conditions hold:

(i) For each t∈ [0, T], x7→ (ϕt(x),Ψt(x))is a C²-map fromR^nintoR × R^m,P^-a.s.,

(ii) For each x∈ Rⁿ, t7→ (ϕt(x),Ψt(x))and t7→ (∇ϕt(x), D²ϕt(x),∇_Ψt(x))are continuous F_t-adapted processes.

Suppose further that there exists a predictable admissible control A^∗(_{t, x, ω})_{such that} H^S(t, x,∇ϕ, D²ϕ,∇_Ψ) = H^S(t, x,∇ϕ, D²ϕ, A^∗(t, x, ω),∇_Ψ),

and that it is regular enough so that the SDE (13) is well-posed with solution X. Then(ϕ,Ψ) = (V, Y)and moreover, for any initial data(0, x)with x∈ Rⁿ, A^∗(t, Xt, ω)maximizes the objective function U.

Proof. See Section 3.2 in [33].

Remark 4. Under suitable regularity conditions on µ, σ, L^S, and g^S, it is possible to prove that the SHJB Equation (14) admits a unique solution satisfying the hypotheses of Theorem4. A rigorous proof of this fact can be found in Section 4 of [33]. For further developments on SHJB equations and

the related stochastic optimal control problems we refer the reader to, e.g., [49–52], as well as the already mentioned paper by Peng [33].

3. Solutions of PDEs via Contact Symmetries

In this section, we recall some basic facts concerning the theory of symmetries on which our results are based, referring to [2,3,53–55] for a complete treatment of these topics.

We start with a formal introduction on jet spaces (for an extended introduction to the subject see, e.g., [56,57]), and then proceed with contact symmetries and their applications in solving PDEs. Despite the fact that these results are well-known, we insert here a small survey for the ease of the reader, as well as we introduce the notation that will be adopted in the rest of the paper.

3.1. Jet Spaces and Jet Bundles

The jet space is a generalization of the notion of tangent bundle of a manifold. Let M and N be two open subsets ofR^mandRⁿ, respectively, and consider a smooth function f : M → N. Take a standard coordinate system x = (x¹, . . . , x^m) in M and let u = (u¹, . . . , uⁿ) = f(x) ∈N. We can then consider the k-th prolongation u^(k)=pr^(k) f(x), that is defined by the relations u_x^j_i =∂_xif^j(x), u_x^j_ixl =∂_xix^lf^j(x), . . ., up to order k. For example, if m=2 and n=1, then pr⁽²⁾f(x¹, x²)is given by

(u; u_x1, u_x2; u_x1x¹, u_x1x², u_x2x²) = (f ; ∂_x1f , ∂_x2f ; ∂_x1x¹f , ∂_x1x²f , ∂_x2x²f)(x¹, x²). The k-th prolongation can also be looked at as the Taylor polynomial of degree k for f at the point x. The space whose coordinates represent the independent variables, the dependent variables and the derivatives of the dependent variables up to order k is called the k-th order jet space of the underlying space N×M, and we denote it by J^k(M, N). It is a smooth vector bundle on M with projection π_k,−1: J^k(M, N) → M given by

π_k,−1(x, u, ux, uxx, . . .) =x.

More explicitly, J^k(M, N) = M×N×N₍₁₎× · · · ×N_(k), where N_(i), is the space of i-th order derivatives of u with respect to x. It is clear that N_(i)⊆ Rⁿⁱ with

n_i=

 m+i−1 i

 .

To any function f ∈C^k(M, N), where C^k(M, N)is the infinite-dimensional Fréchet space of k times differentiable functions on M taking values in N, we associate a continuous section of the bundle(J^k(M, N), M, π_k,−1)in the following way

f 7→D^k(f)(x) = (x, u= f(x)_{, ux}= ∇f(x)_{, uxx} =D²f(x), . . . , D^kf(x))_,

where Dⁱf(x)is the vector collecting all the i-th order derivatives of f with respect to x.

In this setting, a differential equation is a sub-manifold∆E ⊂ J^k(M, N). For example, in the scalar case N= R, we usually consider∆E as the null set of some regular functions, i.e.,∆E = {Eⁱ(x, u, ux, uxx, . . .) =_{0, i}∈ {1, . . . , p}}.

Definition 2. Consider a (finite) set Eⁱ: J^k(M, N) → R, for i=1, ..., p where p∈ Nand p>0, of smooth functions defining a sub-manifold∆E = {Eⁱ(x, u, ux, uxx, . . .) =0, i∈ {1, . . . , p}}of J^k(M, N). We say that a smooth function f : M→N is a solution to the equationE(represented by the sub-manifold∆E) if, for any x ∈ M, we have D^kf(x) ∈∆E. The set of all solutions to equationEwill be denoted bySE.

For instance, in the previous case where N= Rand∆E = {Eⁱ(x, u, ux, . . .) = 0, i∈ {1, . . . , p}}, f is a solution to equationEif Eⁱ(x, f(x),∇f(x), . . .) =0, for every i=1, . . . , p, x∈ M.

Remark 5. For technical reasons, it is usually not possible to consider generic equationsE(cor- responding to generic sub-manifold ∆E ⊂ J^k(M, N)). In the following, we always consider non-degenerate systems of differential equations in the sense of Definition 2.70 in [2]. This condition assures that, for each fixed x₀∈M and each set of derivatives(u⁰, u⁰_x, u⁰_xx, . . .), there exists a solu- tion to the equation defined in a neighborhood of x0with the prescribed derivatives(u⁰, u⁰_x, u⁰_xx, . . .) at the point x0. Since the precise formulation of this condition is quite technical and the evolution equations considered in Section4always satisfy such an assumption, we refer to Section 2.6 of [2]

for complete details.

3.2. Contact Transformations

We want to introduce a class of transformations induced by diffeomorphisms of J^k(M, N). For simplicity, we consider the case k =2, M ⊂ Rⁿand N = R. Consider a diffeomorphismΦ : J²(M, N) → J²(M, N)given by the following relations

˜xⁱ =_Φ^xi(x, u, ux, uxx), u˜=Φ^u(x, u, ux, uxx),

˜

u_xi =_Φ^uxi(x, u, ux, uxx),

˜

u_xix^j =_Φ^{uxi xj}(x, u, ux, uxx).

Hereafter, we use the notationΦ^x = (Φ^x1,· · ·,Φ^xn),Φ^ux = (Φ^ux1,· · ·,Φ^uxn)and Φ^uxx = (Φ^{uxi xj})|i,j=1,...,n.

We now aim to define a transformation F_Φon the space of smooth functions induced by the mapΦ on the jet space. Let U∈ C^∞(M, N)and consider the map C_U,Φ: M→ M given by

CU,Φ(x) =_Φ^x(x, U(x),∇U(x), D²U(x)).

LetF_Φ⊂C^∞(M, N)be the subset of smooth functions U ∈C^∞(M, N)such that C_U,Φ is a diffeomorphism from M into itself.

Definition 3. We say that the diffeomorphismΦ generates the (nonlinear) operator F_Φon the space of functionsF_Φ, if there is a map F_Φ:F_Φ→C^∞(M, N)such that

F_Φ(U)(x) =_Φ^u(C_U,Φ⁻¹ (x), U(C⁻¹_U,Φ(x)),∇U(C_U,Φ⁻¹ (x)), D²U(C⁻¹_U,Φ(x))),

∂_xiF_Φ(U)(x) =_Φ^uxi(C_U,Φ⁻¹ (x), U(C_U,Φ⁻¹ (x)),∇U(C⁻¹_U,Φ(x)), D²U(C⁻¹_U,Φ(x))),

∂_xix^jF_Φ(U)(x) =_Φ^{uxi xj}(C⁻¹_U,Φ(x), U(C⁻¹_U,Φ(x)),∇U(C_U,Φ⁻¹ (x)), D²U(C⁻¹_U,Φ(x))). Not every diffeomorphismΦ : J²(M, N) →J²(M, N)generates an operator F_Φon the space of functionsF_Φ. For example, consider M= Rand the mapΦ^x(x, u, ux, uxx) =λx, Φ^u(x, u, ux, uxx) = u,Φ^ux(x, u, ux, uxx) = ux, andΦ^uxx(x, u, ux, uxx) =uxx, where λ >0.

In this case, for any U∈C^∞(M, N), the map CU,Φis given by CU,Φ(x) =λxand, thus, it does not depend on U and it is always a diffeomorphism fromRinto itself, since λ6=0.

This implies thatF_Φ=_C^∞(_{M, N})and also that, if the map F_Φexists, then it must satisfy F_Φ(U) =U(λ⁻¹x),

for any U∈C^∞(M, N). On the other hand, we have

∂xF_Φ(U) =λ⁻¹U⁰(λ⁻¹x) 6=

6=U⁰(λ⁻¹x) =_Φ^ux(C_U,Φ⁻¹ (x), U(C_U,Φ⁻¹ (x)),∇U(C_U,Φ⁻¹ (x)), D²U(C_U,Φ⁻¹ (x))).

This simple counterexample shows that a diffeomorphismΦ : J²(M, N) → J²(M, N) must satisfy some additional conditions in order to generate an operator F_Φ. For this reason, we introduce the following definition.

Definition 4. A diffeomorphismΦ : J²(M, N) → J²(M, N)is said to be a contact transformation if it generates a (nonlinear) operator F_Φin the sense of Definition3.

It is possible to give a nice geometric characterization of the set of contact transforma- tions. From now on, we writeΛ¹Jⁿ(M, N)for the vector space of 1-forms on Jⁿ(M, N). In particular, consider the following 1-forms,

κ=du−

∑

u_xidxⁱ, (15)

κ_xi =du_xi−

∑

u_xix^jdx^j.

We denote by C⊂ _Λ¹J²(M, N)the contact structure, also called Cartan distribution in [56], which is generated by

C=span{κ, κ_xi, i=1, ..., n}.

Theorem 5. A diffeomorphismΦ : J²(M, N) → J²(M, N)is a contact transformation in the sense of Definition4if and only if it preserves the contact structure C, that is,

Φ^∗(C) =C,

whereΦ^∗is the pull-back of differential forms on J²(M, N)induced byΦ.

Proof. See, e.g., Chapter 2 in [56], Section 4 in [53], Chapter 21 in [3], and the refer- ences therein.

Remark 6. The contact transformationΦ is uniquely determined by its action on J¹(M, N). In particular,Φ^x,Φ^u, andΦ^ux depend only on(x, u, ux)and they do not depend on uxx (see, e.g., Chapter 2 in [56]).

Remark 7. In contact geometry, a contact structure on a(2n+1)-dimensional manifoldMis a 1-form ζ, which is maximally non-integrable, namely, ζ∧ (dζ)ⁿ 6=0, and the contact transformations are the diffeomorphismsΨ ofMsuch thatΨ^∗(ζ) = f·ζ, for some f ∈C^∞(M,R)(see, e.g., [58,59] for an introduction to the subject and [60] for an historical overview). This definition is satisfied by J¹(M,R) with the 1-form ζ=κ defined in(15).

In the study of the geometry of jet spaces (see, e.g., Chapter 6 of [57]), the term “contact structure” is often used to express the set of forms C. This custom is due to the fact that, as explained in Remark6, the contact transformations are extensions of diffeomorphisms on J¹(M, N), i.e., the set of transformations considered here is in one-to-one correspondence with the one usually considered in contact geometry.

In the following, we will not consider just a single contact transformation but one- parameter groups of contact transformationsΦ_λ, which means thatΦ·:R ×J²(M, N) →

J²(M, N) is C^∞, Φ_λ is a contact transformation for each λ ∈ R, Φ0(x, u, ux, uxx) = (x, u, ux, uxx), and, for each λ1, λ2∈ R,

Φλ₁◦_Φλ2 =_Φλ1+λ2.

In general, a one-parameter group of diffeomorphismsΦY,λ, where λ∈ R, is generated by a vector field Y∈T J²(M, N), i.e., belonging to the tangent bundle of J²(M, N), which in local coordinates has the expression

Y=

∑

Y^xi(x, u, ux, uxx)∂_xi+Y^u(x, u, ux, uxx)∂u

+

∑

Y^uxi(x, u, ux, uxx)∂u_xi +

∑

Y^{uxi xj}(x, u, ux, uxx)∂u

xi xj,

(16)

by the following relations

∂_λΦ^x_Y,λⁱ (x, u, ux, uxx) =Y^xi◦Φ_λ(x, u, ux, uxx)_,

∂_λΦ^u_Y,λ(x, u, ux, uxx) =Y^u◦Φ_λ(x, u, ux, uxx),

∂_λΦ^u_Y,λ^xi(x, u, ux, uxx) =Y^uxi ◦Φ_λ(x, u, ux, uxx),

∂_λΦ^u_Y,λ^{xi xj}(x, u, ux, uxx) =Y^{uxi xj} ◦Φ_λ(x, u, ux, uxx)_{, (17)}

for any λ ∈ Rand (x, u, ux, uxx) ∈ J²(M, N). It is useful to introduce the following natural notion.

Definition 5. A vector field Y (of the form(16)) on J²(M, N)is called an infinitesimal contact transformation if it generates (through Equation (17)) a one-parameter group of diffeomorphisms Φλof contact transformations.

The following theorem characterizes all the infinitesimal contact transformations on J²(M, N).

Theorem 6. A vector field Y on J²(M, N)is an infinitesimal contact transformation (in the sense of Definition5) if and only if there exists a unique smooth map Ω : J¹(M, N) → Rsuch that Y=Y_Ω, where Y_Ωis a vector field on J²(M, N)defined as

Y_Ω = −

∑

∂u_xiΩ∂_xi+ Ω−

∑

u_xi∂u_xiΩ

!

∂u+

∑

(∂_xiΩ+u_xi∂uΩ)∂u_xi

+

∑



∂_xix^jΩ+u_xj∂_xiuΩ+u_xjx^k∂_xiu_xkΩ+u_xi∂_xjuΩ +u_xiu_xj∂uuΩ+u_xiu_xjx^k∂uu_xkΩ+u_xix^j∂uΩ

+u_xix^k∂_xju_xkΩ+u_xix^ku_xj∂u_xkuΩ+u_xix^ku_xjx<sup>`</sup>∂u<sub>xk</sub>u<sub>x`</sub>Ω

∂u

xi xj.

(18)

Proof. The proof can be found in Chapter 21 of [3] and references therein.

Remark 8. We say that a vector field of the form Y_Ωsatisfying the hypotheses of Theorem6is the infinitesimal contact transformation generated by the (contact generating) functionΩ. Under this terminology, Theorem5guarantees that any infinitesimal contact transformation is generated in a unique way by some smooth functionΩ : J¹(M, N) → R.

There is a special subset of vector fields of the type (18) arising from coordinate transformations involving only the dependent and independent variables(x, u).

Definition 6. We say that Y_{ΩLie, f ,g} is a (projected) Lie point transformation if it is a contact transformation of the form

Y_{ΩLie, f ,g}=

∑

fⁱ(x)∂_xi+g(_{x, u})∂u+

∑

Y^uxi(_{x, u, ux})∂u_xi+

∑

Y^{uxi xj}(_{x, u, ux}, uxx)∂u_{xi xj}, (19)

where fⁱ ∈ C^∞(M,R)^{, g} ∈ C^∞(J⁰(M, N),R)^{, Yuxi} ∈ C^∞(J¹(M, N),R) ^and Y^{uxi xj} ∈C^∞(J²(M, N),R)^.

Remark 9. It is simple to see that a Lie point transformation Y_{ΩLie, f ,g} can be reduced to a standard vector field ˜Y=_∑ifⁱ(x)∂_xi+g(x, u)∂_uon J⁰(Rⁿ,R), i.e., ˜Y is the generator of a one-parameter group of transformations involving only the dependent and independent variables(x, u).

Remark 10. Another important property of Lie point transformations is the following. Denoting byΦLie, f ,g,λ, where λ ∈ R, the one-parameter group generated by the Lie point transformation

Y_{ΩLie, f ,g}, we have that, for any λ ∈ R, the domainF_ΦLie, f ,g,λ of the nonlinear operator F_ΦLie, f ,g,λ

generated byΦLie, f ,g,λis the whole C^∞(M, N) = F_ΦLie, f ,g,λ.

For what follows, we introduce the (formal) operators D_xi: C^∞(J^k(M, N)) → C^∞(J^k+1(M, N))given by

D_xi =∂_xi +u_xi∂u+

∑

u_xix^j∂u

xj +. . .+

∑

u_xj1···xj`xⁱ∂u

xj1···xj` +. . . (20) In a similar way, we writeD<sub>x</sub>ix<sup>j</sup>(·) = D<sub>x</sub>i(D<sub>x</sub>j(·)),D<sub>x</sub>ix<sup>j</sup>x<sup>`</sup>(·) = D_xi(D_xj(D_x`(·))), etc.

We can characterize more precisely the general form of Lie point transformations.

Theorem 7. The vector field Y_{ΩLie, f ,g}is a (projected) Lie point transformation if and only if it is generated by a function of the form

Ω_{Lie, f ,g}(x, u, ux) =g(x, u) −

∑

fⁱ(x)u_xi, (21)

namely, Y_{ΩLie, f ,g}has the following expression

Y_ΩLie,g,f:=

∑

fⁱ(x)∂_xi +g(x, u)∂u+

∑

(−D_xif^j(x)u_xj+ D_xi(g))∂u_xi

+

∑

(−D_xix^j(f^k)u_xk− D_xi(f^k)u_xkx^j+ D_xix^j(g))∂u_{xi xj}.

Proof. The theorem is a direct application of Theorem6to vector fields of the form (19).

If n=1, and the coordinate system of J⁰(R,R)is given by(x, u), some examples of Lie point transformations are:

• The dilation of independent variable x, i.e., ˜Y=x∂x(see the notation in Remark9), related to the generator functionΩ= −xuxand generating the one parameter group defined by

Φ^x_λ(x, u) =e^λx Φ^u_λ(x, u) =u

Φ_λ^ux(x, u, ux) =e^−λux Φ_λ^uxx(x, u, ux, uxx) =e^−2λuxx.

• The dilation of dependent variable u, namely, ˜Y = u∂u related to the generator functionΩ=u and generating the one parameter group defined by

Φ^x_λ(x, u) =x Φ^u_λ(x, u) =e^λu

Φ_λ^ux(x, u, ux) =e^λux Φ_λ^uxx(x, u, ux, uxx) =e^λuxx.

We conclude this section providing the definition of symmetry of a differential equation.

Definition 7. A contact transformationΦ : J²(M, N) → J²(M, N)is a (contact) symmetry of the differential equationEif, for any solution U∈C^∞(M, N) ∩ F_Φto the equationE, also F_Φ(U) is a solution toE, where F_ΦandF_Φare the operator generated by the contact transformationΦ and the domain of F_Φ, respectively (see Definition3).

We say that an (infinitesimal) contact transformation Y_Ω is an (infinitesimal contact) sym- metry of the differential equationEif the one-parameter groupΦY_Ω,λgenerated by Y_Ω is a set of symmetries of the equationE.

Remark 11. With an abuse of language, we say that the functionΩ ∈ C^∞(J¹(M, N),R)is a contact symmetry of the equationE if the corresponding contact vector field Y_Ω is a symmetry ofE. Remark 12. If Y is a Lie point transformation and it is a contact symmetry of the equationE, then we say that Y is a Lie point symmetry of the equationE.

It is possible to give a completely geometric characterization of the contact symmetries of a differential equationE.

Theorem 8(Determining equations). A contact transformationΦ is a symmetry of the equa- tionErepresented by the sub-manifold∆E ⊂J²(M, N)of the form

∆E = {Eⁱ(x, u, ux, uxx) =0, i∈ {1, . . . , p}}, where p∈ N, p>0 and Eⁱ ∈C^∞(J²(M, N),R), if and only if

Φ(∆E) =∆E.

The infinitesimal contact transformation Y_Ωis a symmetry of the non-degenerate differential equationE(see Remark5for the definition of non-degenerate differential equation) if and only if

Y(Eⁱ(x, u, ux, uxx))|_∆E =0, (22) where i=1, . . . , p.

Proof. The proof is given in Theorem 2.27 and Theorem 2.71 in [2] for the case of Lie point symmetries that are diffeomorphisms of J^k(M, N), for k ≥ 0. Since the contact transformations are diffeomorphism of J^h(M, N), for any h≥1 (see, e.g., Chapter 21 of [3]), the case of contact transformations can be proved using the same methods.

3.3. Symmetries and Classical Noether Theorem

Let us discuss here the classical Noether theorem in the Lagrangian mechanics setting described in Section2.1. Heuristically, the Noether theorem says that to any infinitesimal transformation leaving invariant the optimal control problem, namely Equation (1) and the Lagrangian L, a constant of motion is associated.