NewapproachestoMeanFieldGameTheory CorsodiDottoratoinMatematica Universita'degliStudidiPisa

(1)

Universita' degli Studi di Pisa

Facolta' di Scienze Matematiche Fisiche e Naturali

Corso di Dottorato in Matematica

New approaches to Mean Field Game

Theory

Pisa, 19/05/2017

Tesi di Dottorato

Relatore

Candidata

(2)

(3)

3

A Marco, a Mamma e Papa', a Lorenzo, le mie Radici e le mie Ali.

(4)

(5)

Abstract

The present thesis studies new possible approaches to Mean Field Game inspired by Mechanism Design and Mean Field Theory with moderate interactions.

The structure of the thesis is the following.

The rst part of the thesis studies mean eld games in which the particles inter-act like in mean eld theory and in mean eld theory with moderate interinter-actions. Chapter 2 studies competitions in which the agents interact in a mean eld way and convergence of the system to a proper mean eld equation is proved. Chapter 3, starting from the ideas of chapter 2, consider the situation in which the agents interact locally, as done by [21] in the context of mean eld theory.

The second part of the thesis deals with the idea, inspired by Mechanism Design, of introducing, in the context of mean eld games, an external player, the Principal, who can choose the rules of the game in order to achieve a specic outcome. Chapter 4 introduces the Principal in the context of classical mean eld games; optimality for the payo of the Principal is studied. Chapter 5 consider the presence of the Principal in the context of mean eld games with mean eld interactions: existence of minimum for Principal's payo is proved.

(6)

(7)

Chapter 1 Introduction

1.1 State of art

The starting point of this thesis is Mean Field Game Theory, which is a fascinating theory developed by Lasry and Lion dealing with a class of games in which the num-ber of players is very huge but they have symmetrical payos.

The idea is to study competitions of a large number of players in which each agent is a small player, in the sense that each player's behavior very little inuence the overall system; moreover the players are indistinguishable, in the sense that their payos are symmetric. Thanks to the analogy with mean eld models, in which a system of large number of indistinguishable particles is considered, this class of competition has been dened Mean Field Game Theory.

In some way, Mean Field Game Theory can be dened as the study of strategic decision making in very large populations of small interacting individuals with sym-metric payos.

Thanks to this symmetry, it is possible to study the asymptotic case as the number of players goes to innity with a couple of PDEs, a Fokker Planck and a Hamilton Jacobi Bellmann. In this rst section we give an idea of the main denitions and results of Mean Field Game Theory.

(12)

1.1.1 A stochastic optimal control problem: the Mean Field

Game (MFG)

In this section we give the mathematical denition of the object we are dealing with. We have N players acting according the dynamics described by the following stochastic equation: the state of the i − th player is an element Xi

t ∈ Rd such that

dX_ti = αi_tdt +√2dB_ti, where (Bi

t)Ni=1 is a sequence of independent Brownian motions, αi is adapted to the

ltration

(Ft= σ(X0j, B j

s : s ≤ t, j = 1, · · · , N ))

and the initial states Xi

0 have a xed law m0 and are independent.

Each player has a personal payo in the form J_iN(α1, · · · , αN_{) = E} "ˆ _T 0 1 2|α i t| 2 + F X_ti, 1 N − 1 X j6=i δ_Xj t ! dt # . where F : Rd_{× P}1

(Rd) → R satises some hypothesis that will be specied below. Player i−th wants to minimize JN

i ; the mathematical problem we are dealing with

is thus the following: minimize JN

i conditioned to

dX_tj = αj_tdt +√2dB_tj for all j = 1, ..., N.

For doing that, the i−th player can only choose the i−th control αi_{, but cannot decide}

for the other control αj_, _{which somehow determine i−th player payo through X}j_.

We need a specic sense in which we are trying to minimize JN

i : the sense can be

found in the notion of Nash equilibrium here below.

Denition 1 We say that (α∗,1_{, · · · , α}∗,N₎ _{is a Nash equilibrium for (J}N

i )Ni=1 if for

all i and for all α

(13)

1.1. STATE OF ART 13 However it is not possible for us, because of the complexity of the problem, to give a complete characterization of the Nash equilibrium of the problem; thus we will give an approximation of it, that is the - Nash equilibrium:

Denition 2 We say that (α∗,1

, · · · , α∗,N) is an - Nash equilibrium for (J_iN)N_i=1 if for all i and for all α

J_iN(α∗,1, · · · , α∗,N) ≤ J_iN((α∗,j)j6=i, α) +

The minimization will be in the sense of - Nash equilibrium and it will be shown in the next sections.

1.1.2 From game to PDEs: the Mean Field Equations (MFE)

In some sense that will be specied below, the mean eld game evolves to a system of two PDEs, called mean eld equation (MFE for short):

       −∂tu − ∆u + 1₂|∇u|2 = F (x, m) ∂tm − ∆m−div(m∇u) = 0 u(x, T ) = 0 m(0) = m0,

Notice that the rst equation is an Hamilton Jacobi Bellman and the second is a Fokker Planck. The two equations are coupled by the term F and the system is for-ward - backfor-ward: it is not immediate clear that this system admits unique solution. This section is devoted to the explanation of this fact.

In order to prove existence and uniqueness we need some hypothesis: 1. F : Rd_{× P}1_(Rd_{) → R, such that}

|F (x, m)| ≤ C0,

|F (x, m) − F (x0, m0)| ≤ C0(|x − x0| + d1(m, m0)).

where d1 is the 1 Wasserstein distance dened by

d1(p, ν) = sup [φ]Lip≤1 ˆ φ(x)p(dx) − ˆ φ(x)ν(dx) ;

(14)

2. m0 is absolutely continuous with respect to Lebesgue measure with Holder

continuous density and satises ˆ Rd |x|2m0(dx) < +∞. 3. for all m, m0 _{∈ P}1 (Rd), m 6= m0, ˆ Rd (F (x, m) − F (x, m0))d(m − m0)(x) > 0.

Under these hypothesis, we are looking for classical solutions, dened by the follow-ing:

Denition 3 We say that a pair (u, m) is a classical solution of the MFE if • u, m : Rd_{× [0, T ] → R are continuous;}

• u, m are C2 _{in space and C}1 _{in time;}

• u, m satisfy the MFE in classical sense. We can give now the existence theorem for MFE:

Theorem 4 Under the above assumptions, there is at least one classical solution to MFE.

Proof. The proof is split in the following steps:

1. Consider a proper convex and compact subset C of C([0, T ] : P1₎_;

2. Build a map Ψ : C → C in the following way: • associate to some µ ∈ C the solution u of

−∂tu − ∆u + 1₂|∇u|2 = F (x, µ)

u(x, T ) = 0 • associate to u the solution m of

∂tm − ∆m−div(m∇u) = 0

(15)

1.1. STATE OF ART 15 3. Apply a xed point to Ψ : µ 7→ m.

Let's start with the rst step. We dene C as the set of maps µ ∈ C([0, T ] : P1_(Rd₎₎

such that sup s6=t d1(µ(s), µ(t)) |t − s|1/2 ≤ C and sup t∈[0,T ] ˆ Rd |x|2_{µ(t)(dx) ≤ C.}

The set C is clearly convex and it is compact in the topology dened by d(µ, ν) = sup

t∈[0,T ]

d1(µ(t), ν(t)).

At this point, let's build the function Ψ on C. Consider an element µ ∈ C and associate to µ the solution u of

−∂tu − ∆u + 1₂|∇u|2 = F (x, µ)

u(x, T ) = 0

To see that a solution exists and is unique, we use Cole Hopf transform and put w = e−u/2;

then w has to satisfy

−∂tw − ∆w = wF (x, µ)

w(x, T ) = 1

This last equation admits unique solution thanks to the following classical result (see [16]):

Theorem 5 Fix s ∈ N, α ∈ (0, 1]. Denote with Cs+α _{the set of maps z : R}d _×

[0, T ] → R such that

• the derivatives ∂k

tDxlz exist if 2k + l ≤ s

(16)

If a, b, f, w0 ∈ Cα, then there exists a unique weak solution to

∂_tw − ∆w + ha(x, t), ∇wi + b(x, t)w = f (x, t) w(x, 0) = w0(x).

Moreover w ∈ C2+α _{for t strictly positive.}

We can apply this result to our situation since in our situation we have a = 0, f = 0, b = F, w0 = 1 and, thus, it is sucient to control that (x, t) 7→ F (x, µ(t)) ∈ Cα :

|F (x, µ(t)) − F (x0, µ(t0))| ≤ C(|x − x0| + d1(µ(t), µ(t0)))

≤ C(|x − x0| + |t − t0|1/2).

Thanks to theorem 5 we get existence and uniqueness for w ∈ C2+α _{and thus for}

u ∈ C2+α_.

Studying the Hamilton Jacobi Bellman equation satised by u, it is possible to show that u is bounded and lipschitzian.

At this point we associate to u the solution m of

∂_tm − ∆m−div(m∇u) = 0 m(0) = m0.

The equation can be rewritten in the following form:

∂tm − ∆m − h∇m, ∇ui − m∆u = 0

m(0) = m0,

in order to apply theorem 5 again. Thanks to the fact that ∇u, ∆u ∈ Cα _(because

u ∈ C2+α), we get the existence of a solution m ∈ C.

We have built the function Ψ, which is moreover continuous. We just need to verify compactness property of Ψ, in order to apply the following xed point theorem: Theorem 6 Let X be a locally convex topological vector space. Let K ⊂ X be a non-empty, convex and compact set. For any continuous function

f : K → K, there exists x ∈ K such that f(x) = x.

(17)

1.1. STATE OF ART 17 Let's start with the estimation of u. Fix x ∈ Rd and t ∈ [0, T ]; for all control α

consider the solution Xα

s of the following SDE:

dXα s = αsds + √ 2dBs Xt = x; then u(x, t) = inf α E ˆ T t 1 2|αs| 2_{+ F (X}α s, µs)ds .

Thanks to boundedness of F we immediately get that u ≤ C0T. Let's study ∇u.

Fix another point y ∈ Rd _{and consider the solution Y}α _of

dYα s = αsds + √ 2dBs Yt = y; then u(y, t) = inf α E ˆ T t 1 2|αs| 2 + F (Y_sα, µs)ds .

Denote αx _{the optimal control for u(x, t) and α}y _{the optimal control for u(y, t). We}

have:

u(x, t) − u(y, t) ≤ inf

α E ˆ T t 1 2|α y s| 2₋ 1 2|α y s| 2_{+ F (X}αy s , µs) − F (Yα y s , µs)ds ≤ inf α E ˆ T t C0|Xα y s − Y αy s |ds ≤ inf α E ˆ T t C0|x − y|ds ≤ T C0|x − y|.

We can repeat the same computation for u(y, t) − u(x, t) and get that |u(x, t) − u(y, t)| ≤ T C0|x − y|,

(18)

that is

|∇u| ≤ T C0.

Notice that, thanks to the uniform boundedness of F , these estimates does not depend on µ. Thanks to this, in order to get compactness properties of Ψ it is sucient to show that there exists a constant C depending on C0 and T such that

the solution m belongs to the set C. At this point, this is an easy task using the fact that m is the law of the solution of

dXt= −∇u(Xt, t)dt +

√ 2dBt

L(X0) = m0.

As to uniqueness of the solution of MFE, we need the following hypothesis on F : for all m, m0 _{∈ P}1_(Rd_{), m 6= m}0_,

ˆ

Rd

(F (x, m) − F (x, m0))d(m − m0)(x) > 0.

Under the previous hypothesis, we can state the following uniqueness result:

Theorem 7 Under the above assumption, there exists a unique classical solution to MFE.

The proof is quite technical and not so much interesting for the purposes of this thesis, so we skip it.

1.1.3 The link between MFG and MFE

In this section we are going to discuss the link between mean eld games and mean eld equation: we will show that the mean eld equation represents the approximate asymptotic solution for the mean eld game (in the sense of Nash equilibrium) when the number of players goes to innity.

In order to show that, we need to consider the following abstract control problem. We have

(19)

1.1. STATE OF ART 19 • a functional: J(α) = Eh´₀T 1 2|αt| 2_{+ F (X} t, mt)dt i , • a state: dXt = αtdt + √ 2dBt. We want to nd inf α J (α).

The solution of this abstract control problem is given by the following:

Proposition 8 Consider the solution of the MFE (u, m); denote ¯X as the solution of d ¯Xt = −∇u( ¯Xt, t)dt + √ 2dBt and put ¯ αt= −∇u( ¯Xt, t); Then J ( ¯α) = inf α J (α).

Proof. Consider a generic control α: using Ito formula, we get 0 = E[u(XT, T )] = E u(X0, 0) + ˆ T 0

∂tu(Xs, s) + hαs, ∇u(Xs, s)i + ∆u(Xs, s)ds

= E u(X0, 0) + ˆ T 0 1 2|∇u(Xs, s)| 2 + hαs, ∇u(Xs, s)i − F (Xs, ms)ds notag ≥ E u(X0, 0) + ˆ T 0 −1 2|αs| 2_{− F (X} s, ms)ds = E [u(X0, 0)] − J (α). Then J (α) ≥ E [u(X0, 0)] ;

(20)

moreover we get equality when using the control ¯

αt= −∇u( ¯Xt, t),

which means that

E [u(X0, 0)] = J ( ¯α).

In other words,

J (α) ≥ J ( ¯α) for all controls α.

Now we have the tools to study the link between mean eld games and mean eld equation. Recall that we have the following situation:

• a payo for each player: J_iN(α1, · · · , αN_{) = E} "ˆ _T 0 1 2|α i t|2+ F Xti, 1 N − 1 X j6=i δ_Xj t ! dt #

• a state for each player:

dX_ti = αi_tdt +√2dB_ti

The following theorem explains in which sense mean eld equation is the asymptotic solution of mean eld game.

Theorem 9 Fix (u, m) solution to MFE. For all i, put d ¯X_ti = −∇u( ¯X_ti, t)dt +√2dB_ti

¯

αi_t= −∇u( ¯X_ti, t) Then (¯α1_{, · · · , ¯}_αN₎ _{is a}

N- Nash equilibrium for (J1N, · · · , JNN) with N → 0 as

N → ∞.

Proof. We have to evaluate:

J_iN( ¯α1, · · · , ¯αN) − J_iN(( ¯αj)j6=i, α)

which is dominated by

(21)

1.2. MEAN FIELD GAMES WITH MEAN FIELD INTERACTIONS 21 since ¯α solves the abstract optimal control problem.

It is sucient to show that

J_iN( ¯α1, · · · , ¯αN) − J ( ¯αi) → 0 and J (α) − J_iN(( ¯αj)j6=i, α) → 0. Consider the rst: J_iN( ¯α1, · · · , ¯αN) − J ( ¯αi_{) ≤ E} "ˆ _T 0 d1 m(t), 1 N − 1 X j6=i δ_X¯j t ! dt # ,

which goes to zero, since ¯Xj _{are independent and identically distributed with law m,}

as the following equations state:

d ¯X_ti = −∇u( ¯X_ti, t)dt +√2dB_tj ∂tm − ∆m − div(m∇u) = 0.

So we get

1_N = J_iN( ¯α1, · · · , ¯αN) − J ( ¯αi) → 0.

For the second term, we can repeat the same argument and get that 2_N = J (α) − J_iN(( ¯αj)j6=i, α) → 0.

This implies that

J_iN( ¯α1, · · · , ¯αN) − J_iN(( ¯αj)j6=i, α) ≤ N

with

N = 1N + 2 N → 0.

1.2 Mean eld games with mean eld interactions

The chapter studies mean eld games where the players interact as inspired by mean eld theory: we will see that this means that the drift of the i−th player depends on the empirical measure of the other player.

The problem has already been studied in [11] in the context of crowd dynamics with an applied perspective. Here we give an equivalent formulation of the result.

(22)

1.2.1 Introduction

Consider N players and for all i = 1, · · · , N the dynamics of the i−th player is given by: dX_ti,αi = αi_tdt + b X_ti,αi, 1 N N X j=1 δ Xj,αjt ! +√2dB_ti where (α1_{, · · · α}N₎ are the controls chosen by the players.

Notice that the mean eld interaction term, that is b, does not depend on N; the situation in which b depends on N will be studied in chapter 5.

For each player, we can consider a payo function given by: Ji(α1, · · · , αN_{) = E} "ˆ _T 0 1 2|α i t| 2 + F X_ti,αi, 1 N − 1 X j6=i δ X_tj,αj ! dt # .

We want to minimize the functional in a proper sense, that is in the sense of Nash equilibrium.

1.2.2 Main result

The main result of the chapter is the following:

Theorem 10 Fix a solution (u, µ) of the following mean eld equation:       

−∂tu − ∆u − hb(·, µ), ∇ui + 1₂|∇u|2 = F (x, µ)

∂tµ − ∆µ−div(µ∇u)−div(µb(·, µ)) = 0

u(x, T ) = 0 µ(0) = µ0,

Put for all i

dX_ti = −∇u(X_ti)dt + b X_ti, 1 N N X j=1 δ_Xj t ! +√2dB_ti and ¯ αi = −∇u(Xi). Then (¯α1_{· · · , ¯}_αN₎is an

N-Nash equilibrium for (J1, · · · , JN),with N → 0as N →

(23)

1.2. MEAN FIELD GAMES WITH MEAN FIELD INTERACTIONS 23 The theorem gives us a minimization result in the sense of -Nash equilibrium. Let's x some notation.

Consider a general control α1_{. We have that (X}i₎N

i=1 are the solutions of the system

of SDEs given by dX_ti = −∇u(X_ti)dt + b X_ti, 1 N N X j=1 δ_Xj t ! +√2dB_ti for all i = 1, · · · , N. We write SN,ˆi = 1 N − 1 X j6=i δXj and SN = 1 N X i≥1 δ_Xi. We denote with ( ¯Xi₎N

i=1 the solutions of the system of SDEs given by

d ¯X_ti = −∇u(X_ti)dt + b X¯_ti, 1 N N X j=1 δ_X¯j t ! +√2dB_ti for i ≥ 2 and d ¯X_t1 = α1_tdt + b X¯_t1, 1 N N X j=1 δ_X¯j t ! +√2dB_t1. We write ¯ SN,ˆi = 1 N − 1 X j6=i δ_X¯j and ¯ SN = 1 N X i≥1 δ_X¯i.

(24)

1.2.3 The three main ingredients of the proof

In order to prove the main result we need three ingredients.

We rstly need to solve an abstract problem dealing with our problem. The lemma is the following:

Lemma 11 Consider the payo function J (α) = E ˆ T 0 1 2αt+ F (Xt, µt)dt conditioned to dXt= αtdt + b(Xt, µt)dt + √ 2dBt. Put dX_tµ = −∇u(X_tµ)dt + b(X_tµ, µt)dt + √ 2dBt and ¯ αµ = −∇u(Xµ). Then ¯αµ _{minimizes the functional J}α_.

Secondly, we need convergence of the empirical measure of the optimal states (when the players choose the controls ¯αi) to the solution of the Fokker Planck of the mean

eld equation:

Theorem 12 The empirical measure SN _{converges to µ in the following sense:}

lim N →∞E ˆ T 0 d1(SsN, µs)ds → 0.

In order to get convergence we use tightness argument.

Thirdly, we need convergence of the empirical measure of the modied problem (the case in which the players choose the optimal controls except one who chooses a general control αi_{) again to the solution of the Fokker Planck.}

Theorem 13 The empirical measure ¯SN _{converges to µ in the following sense:}

lim N →∞E ˆ T 0 d1( ¯SsN, µs)ds → 0.

To prove this result we study the distance between ¯SN and SN and prove that it

(25)

1.2. MEAN FIELD GAMES WITH MEAN FIELD INTERACTIONS 25

1.2.4 Proof of the main result

Putting together the three ingredients, we can prove the main theorem. Proof. Consider a general control αi_{. We have}

Ji( ¯α1, · · · , ¯αN) − Ji(( ¯αj)j6=iαi) = E ˆ T 0 1 2| ¯α i t| 2_{+ F} _Xi t, S N,î t dt − E ˆ T 0 1 2|α i t| 2_{+ F} ¯_Xi t, ¯S N,î t dt = E ˆ T 0 1 2| ¯α i,µ t |2+ F X i,µ t , µt dt − E ˆ T 0 1 2|α i t| 2_{+ F} _X_¯i,µ t , µt dt + E ˆ T 0 F X_ti, S_tN,î− F X_ti,µ, µt dt + E ˆ T 0 F X¯_ti,µ, µt − F ¯Xti, ¯S N,î t dt + E ˆ T 0 1 2| ¯α i t| 2₋ 1 2| ¯α i,µ t | 2 where

dX_ti,µ = −∇u(X_ti,µ)dt + b(X_tµ, µt)dt +

√ 2dB_ti, αi,µ = −∇u(Xi,µ)

and

d ¯X_tµ= αi_tdt + b( ¯X_tµ, µt)dt +

√ 2dB_ti. Thanks to the abstract problem studied in section 3, we get that

E ˆ T 0 1 2| ¯α i,µ t | 2 + F X_ti,µ, µt dt − E ˆ T 0 1 2|α i t| 2 + F X¯_ti,µ, µt dt ≤ 0, thus Ji( ¯α1, · · · , ¯αN) − Ji(( ¯αj)j6=iαi) ≤ I1+ I2+ I3 where I1 = E ˆ T 0 F X_ti, S_tN,ˆi − F X_ti,µ, µt dt ,

(26)

I2 = E ˆ T 0 F X¯_ti,µ, µt − F ¯Xti, ¯S N,ˆi t dt , I3 = E ˆ T 0 1 2|α i t| 2₋ 1 2|α i,µ t |2 .

Consider I1. We have that, thanks to the properties of F ,

I1 ≤ E ˆ T 0 |X_ti− X_ti,µ|dt + E ˆ T 0 d1(StN,ˆi, µt)dt .

As to the rst term we have that , thanks to lipschitizianity of ∇u and b |Xi t− X i,µ t | ≤ C ˆ T 0 |Xi t− X i,µ t |dt + C ˆ T 0 d1(StN, µt)dt;

using the convergence of SN _{to µ as proved in section 4, we get, by Gronwall lemma,}

that E ˆ T 0 |Xi t− X i,µ t |dt → 0, and than I1 → 0.

The same argument can be applied to prove that I2 goes to 0. As to I3, we have

I3 ≤ k∇uk∞E ˆ T

0

|∇u(X_ti) − ∇u(X_ti,µ) ≤ CE ˆ T 0 |X_ti− X_ti,µ|dt → 0.

1.2.5 Existence and uniqueness for the solution of the MFE

In this section we discuss existence and uniqueness for the solution of the MFE. As to existence we give the following theorem.

(27)

1.2. MEAN FIELD GAMES WITH MEAN FIELD INTERACTIONS 27 Theorem 14 In our hypothesis, there exists a solution (u, µ) to

      

−∂tu − ∆u − hb(·, µ), ∇ui + 1₂|∇u|2 = F (x, µ)

u(x, T ) = 0 µ(0) = µ0,

with u ∈ C2+1_.

The proof of the theorem consists in applying a xed point theorem to proper spaces, that are: ∆C =u ∈ C2,1(Rd× [0, T ]) : kuk∞≤ (1 + T )C, kDxuk∞ ≤ C CC = ( µ ∈ C0([0, T ], P1) : sup t,s∈[0,T ] d1(µ(t), µ(s)) |t − s|1/2 ≤ C; sup t∈[0,T ] ˆ Rd |x|2µ(t)(dx) ≤ C ) .

We associate to some m ∈ CC, the solution u of the HJB of the mean eld system

with m and prove that u belongs to ∆C. At this point we associate to u the solution

µ of the Fokker Planck with u. In this way we dene a map from CC to CC which

satises some compactness inequalities that allow us to apply a x point theorem. As to uniqueness we recall the paper [13] where weak uniqueness for the system is stated with the following theorem:

Theorem 15 In our hypothesis the system       

−∂tu − ∆u − hb(·, µ), ∇ui + 1₂|∇u|2 = F (x, µ)

u(x, T ) = 0 µ(0) = µ0,

(28)

1.3 Mean eld games with moderate interaction

We consider a mean eld game model where N players also interact by a "moderate" interaction, in the language of [21].

The idea is to generalize the approach studied in the previous chapter by using a kernel depending on the number of players.

1.3.1 Introduction

We consider a mean eld game where N players interact both in the cost functionals and in the dynamics. The new feature is that the dynamical interaction is of the so called "moderate" type, intermediate between the more classical mean eld interac-tion and a strictly local (let us say "nearest neighbor") interacinterac-tion. In applicainterac-tions, mean eld (namely long range) interaction is often unrealistic. Nearest neighbor interaction would be very important for applications, for instance in biology the con-tact interactions between cells modeled as particles, but it requires to deal with a so called hydrodynamic limit, which in the case of interacting diusions is poorly understood. The case of intermediate interaction, when each particle interact only with an innitesimal proportion of particles, but still innitely many in the limit, is a rst step in this direction. The limit PDE system is made only of local dierential operators, as it should be for local interactions. So it could be considered a rst approximation of the purely local case. This is the motivation for this investigation. Consider N controlled interacting particles Xi,N

t , living in Rd, i = 1, · · · , N, governed by the equations dX_ti,N = Gt, X_ti,Ndt − 1 N N X j=1 V_N0(X_ti,N− X_tj,N)dt +√2dB_ti, X₀i,N = X₀i. (1.1) We denote by SN

t the empirical measure of the system, that is

S_tN = 1 N N X i=1 δ_Xi,N t .

We will need also a regularization of SN t :

pN_t = WN ∗ StN

Here G (t, ·) is a feedback control, {Bi

t}i∈N is a sequence of independent Brownian

motions in Rd _{dened on a ltered probability space (Ω, F, F}

(29)

1.3. MEAN FIELD GAMES WITH MODERATE INTERACTION 29 sequence of independent Rd-valued random variables on (Ω, F

0, P ), and VN : Rd→ R

is an interaction potential which becomes more and more localized as N → ∞, but still allow interaction between a large number of players. We assume VN of the form

VN(x) = NβV Nβ/dx

for β > 0, so the range of interaction scales with N. The classical mean eld case is when VN = V is independent of N, namely β = 0. The local interaction case is

when the support of VN has size comparable to the typical minimal distance between

particles, namely N−1/d_{, which corresponds to the case β = 1. We consider an}

intermediate case between these ones, called moderate interactions in the language of [21], namely 0 < β < 1. In addition, for technical reasons, we suppose that

β < d d + 2.

We assume that an external agent may choose G and wants to choose it in order to minimize, for each particle, a functional of the form

Ji,N_{(G) = E} "ˆ _T 0 1 2|G t, X_ti,N|2_{+ F} _Xi,N t , 1 N N X j=1 δ_Xj,N t ! dt # . (1.2) This is a variant of mean eld game theory. Opposite to the classical case, here the particles are not "agents", they cannot choose their own controls to optimize their own costs. Here particles are passive, in a sense, and an external agent may modify their dynamic with the choice of a feedback control G, equal for all of them; as equal are the cost functionals. Justication for this new approach will be given in the next sections.

Let us remark about a minor detail. Above in the interaction and below in the cost functional we use 1

N PN j=1 in place of 1 N −1 P

j6=i, as it should be to exclude

self-interaction. The nal result is the same and the notations are much easier in the case of 1

N

PN

j=1, so we accept this minor approximation with respect to reality.

The mean eld game system of PDEs arising in this model is       

∂tu + ∆u = 1₂|∇u|2+ ∇p · ∇u − F (x, p)

∂tp = ∆p + 1₂∆p2+ div(p∇u)

u(x, T ) = 0 p(x, 0) = p0(x) .

(1.3) The peculiarity is the porous media term ∆p2 in the equation for the density of

(30)

1.3.2 Convergence of X

i,N

_{and S}

N

The theorem stating convergence of the processes Xi,N _{and their empirical measure}

is the following.

Theorem 16 Let (Xi,N

t )i=1,··· ,N be the solution of

dX_ti,N = Gt, X_ti,Ndt − 1 N N X j=1 V_N0(X_ti,N− X_tj,N)dt +√2dB_ti, X₀i,N = X₀i. (1.4) and (Xi,G

t )i∈N be the solutions of equations

dX_ti,G = G(t, X_ti,G)dt − ∇pG(t, X_ti,G)dt +√2dB_ti, X₀i,G = X₀i, (1.5) where ∂_tpG= ∆pG+ 1₂∆(pG)2+ div(G · pG) p(x, 0) = p0(x) . (1.6) Then lim N →∞E " sup t∈[0,T ] X i,N t − X i,G t dt # = 0 (1.7) lim N →∞E ˆ T 0 d1(StN, p G t )dt = 0. (1.8)

The proof of this facts is very hard and involves some advanced techniques developed by [21]. We will see the details of the proof in chapter 3.

An important corollary of this theorem is the follwing result: Corollary 17 Fix a solution (u, p) of

      

∂tu + ∆u = 1₂|∇u|2+ ∇p · ∇u − F (x, p)

∂tp = ∆p + 1₂∆p2+ div(p∇u)

u(x, T ) = 0 p(x, 0) = p0(x) .

(1.9) Consider the processes

d ¯X_ti,N = −∇ut, ¯X_ti,Ndt − 1 N N X j=1 V_N0( ¯X_ti,N − ¯X_tj,N)dt +√2dBi_t, X¯₀i,N = X₀i, (1.10)

(31)

1.3. MEAN FIELD GAMES WITH MODERATE INTERACTION 31 and

dX_ti,−∇u = −∇u(t, X_ti,−∇u)dt − ∇p(t, X_ti,−∇u)dt +√2dB_ti, X₀i,−∇u = X₀i. (1.11) Putting ¯ SN t = 1 N N X i=1 δ_X¯i,N t , then we have lim N →∞E " sup t∈[0,T ] ¯ X_ti,N − X_ti,−∇u dt # = 0 (1.12) lim N →∞E ˆ T 0 d1( ¯StN, pt)dt = 0. (1.13)

1.3.3 A rst open problem and a new approach

Notice that, dierently from usual mean eld game theory, here we have modied the controls for all the agents, since the feedback control G acts on the drift of each player. Consider the classical situation in which only one players, namely the rst, modies his control; we get a system in which the equation for the i-th player are the same as before for i = 2, ..., N while the rst player has an equation on the form

dX_t1,N = αtdt − 1 N N X j=1 V_N0(X_t1,N − X_tj,N)dt +√2dB_t1.

To study convergence of the processes Xi _{we can repeat the previous proof and get}

convergence of the processes Xi only for i = 2, ..., N; in fact, the key point of our

proof is exchangeability for the processes and in this situation the i-th processes, for i = 2, ..., N are exchangeable while the rst process is not since it has a dierent drift. In other words, we cannot get convergence of X1,N to the solution of

dX_t1,α= αtdt − ∇p(t, Xt1,α)dt +

√ 2dB1_t.

This technical fact makes impossible Nash equilibrium analysis, which require a modication of just a process.

In this sense, thanks to our convergence theorem for simultaneous feedback control and the technical problem arising when modifying just a process, it is natural to

(32)

study the case in which all the controls are changed simultaneously by a feedback control. Moreover there are biological motivations justifying this approach: the next sections will be devoted to discuss them.

We assume that an external agent may choose G and wants to choose it in order to minimize, for each particle, a functional of the form

Ji,N_{(G) = E} "ˆ _T 0 1 2|G t, X_ti,N|2_{+ F} _Xi,N t , 1 N N X j=1 δ_Xj,N t ! dt # . (1.14) The criterion that however we adopt, to be precise, is N-optimal, that is

Denition 18 A feedback control G∗ _{is called}

N-optimal if there is an innitesimal

sequence (N)_{N ∈N} such that, for each N ∈ N, each i = 1, ..., N and each feedback

control G one has

Ji,N(G∗) ≤ Ji,N(G) + N. (1.15)

Our conjecture is that an N-optimal feedback control, with limN →∞N = 0, is given

by controls of feedback form

G = −∇u.

1.3.4 Some motivating examples: angiogenesis

Here we give two motivating example for our dissertation on mean eld game with moderate interactions. The examples are related to two biological problems.

Angiogenesis is the physiological process through which new blood vessels form from pre-existing ones. This is a normal process in growth and development, however, it is also a fundamental step in the transition of tumors from the benign to the malig-nant state. Tumoral hypoxic cells (that is cells with low level of oxygen) produce a growth factor signal, the VEGF, which is sent to near blood vessel and stimulate the formation of new capillaries towards the tumor, to provide oxygen to hypoxic cells. The cells moving towards the tumor from vessels are called tip cells.

(33)

1.3. MEAN FIELD GAMES WITH MODERATE INTERACTION 33 Here we try to model this situation in the context of mean eld games. Consider N tip cells; their state is given by

dX_ti = G(X_ti, t)dt − (V_N0 ∗ SN t )(X i t)dt + √ 2dB_ti, where G is a vector eld heading the cells in a proper way.

Each cell is trying to minimize the distance of the tumor from the near vessels so we can suppose that the i−th cell has a payo of the form

Ji _{= E} ˆ T 0 1 2|G(X i t, t)| 2 + |X_ti− x0|dt .

where x0 is a point inside the tumor.

The cells need to optimize their payo; we can suppose that, thanks to Darwinian selection, the nature chooses a vector eld G in order to minimize Ji_{: we will see}

that the optimal vector eld is −∇u, with (u, p) solving ∂tu + ∆u = 1 2|∇u| 2_{+ ∇p · ∇u + F (x)} ∂tp = ∆p + ∆p2+ div(p∇u), where F (x) = |x − x0|.

Let's denote uVEGF _{the solution of the rst equation. The vector eld −∇u}VEGF _is

the term pushing the tip cells towards tumor to get oxygen supply.

Notice that the usual coupling term F of mean eld games here only depends on the position Xi. However, thanks to moderate interactions between particles, in the

PDEs solving the mean eld game we have the term ∇p · ∇u which is coupling the system. Without moderate interactions, the problem would be a classical optimizing problem; the presence of moderate interactions gives us a non trivial problem even in the case in which F does not couple the system.

1.3.5 Some motivating examples: cell adhesion

This example is related to the problem of cells adhesion in tissue.

Tissues are treated as liquids consisting of mobile cells. According to dierential adhesion hypothesis (DAH), cells move to be near to other cells of similar adhesive

(34)

strength: similar cells are forced to be near.

In this sense, we can consider for each cell an equation on the form dX_ti = G(X_ti, t)dt − (V_N0 ∗ SN t )(X i t)dt + √ 2dBi_t,

where G is a proper vector eld, as before. In this situation, each cell wants to move near to similar cells, so the payo to be minimized has the form

Ji _{= E} ˆ T 0 1 2|G(X i t, t)| 2 + (K ∗ S_tN)(x)dt .

for a proper attractive kernel K.

As before, we can suppose that, trough selection, nature has chosen a proper vector eld G in order to optimize Ji: the vector eld optimizing Ji is −∇u, where (u, p)

is the solution of ∂tu + ∆u = 1 2|∇u| 2 + ∇p · ∇u + K ∗ p ∂tp = ∆p + ∆p2+ div(p∇u).

In this situation the coupling takes place both in the equations, thanks to moderate interactions, and in the functional, since F depends on the empirical measure. The interesting fact is that the eects are opposite in the two situations: at the level of SDE there is a repulsive eect, while the minimization of the functional Ji _{creates an}

eect of aggregation of particles. This is a very interesting capability of our system: modeling attractive interactions between particles is a very hard problem and of recent interest, studied, for example, in [7] with completely dierent tools. In this situation we present a new approach modeling adhesion eect through the coupling term F .

1.3.6 A second open problem

In this subsection we give a conjecture for the characterization of the optimal feed-back control. In the rst subsection we state an abstract limit control problem and conjecture its solution. Supposing the conjecture to be true, in the second subsection we give the -optimal feedback control to our problem.

(35)

1.3. MEAN FIELD GAMES WITH MODERATE INTERACTION 35 In order to get −optimality for our system, we need to deal with the following limit problem. Problem 19 Calculate min G J (G) where J (G) = E ˆ T 0 1 2|G(t, Xt)| 2_{+ F (X} t, pt)dt , conditioned to dXt = G(t, Xt)dt − ∇p(t, Xt)dt + √ 2dBt and p(t, ·) = L(Xt).

At this moment, we do not have a solution for this problem, but we conjecture the following result.

Conjecture 20 The solution of 45 is the feedback control G = −∇u

where (u, p) solves       

∂tu + ∆u = 1₂|∇u|2+ ∇p · ∇u − F (x, p)

∂tp = ∆p + 1₂∆p2+ div(p∇u)

u(x, T ) = 0 p(x, 0) = p0(x) .

(1.16)

1.3.7 − optimality

Here we state the conjectured -optimal result, showing how the abstract problem stated in 45 is linked with our optimal control problem.

Theorem 21 Suppose the conjecture 46 to be true. Let (u, p) be a solution of the system       

∂tu + ∆u = 1₂|∇u|2+ ∇p · ∇u + F (x, p)

∂tp = ∆p + 1₂∆p2+ div(p∇u)

u(x, T ) = 0 p(x, 0) = p0(x) .

(1.17) Then the feedback control G∗_{(t, x) = −∇u(t, x)} _is

(36)

Proof. We have

Ji,N(−∇u) − Ji,N(G) = E ˆ T 0 1 2|∇u t, X_ti,−∇u |2+ F (X_ti,−∇u, p−∇u_t ) dt − E ˆ T 0 1 2|G t, X_ti,G|2_{+ F (X}i,G t , p G t) dt + (1)_N − (2)_N where (1)_N = Ji,N_{(−∇u) − E} ˆ T 0 1 2|∇u t, X_ti,−∇u|2_{+ F (X}i,−∇u t , p −∇u t ) dt (2)_N = Ji,N_{(G) − E} ˆ T 0 1 2|G t, X_ti,G |2+ F (X_ti,G, pG_t ) dt

and where (u, p) is the given solution of system (3.20), Xi,−∇u

t is the solution of

the limit problem with control −∇u, Xi,G _{is the solution of the limit problem with}

control G.

Supposing 46 to be true, we get E ˆ T 0 1 2|∇u t, X_ti,−∇u|2_{+ F (X}i,−∇u t , p −∇u t ) dt ≤ E ˆ T 0 1 2|G t, X_ti,G|2+ F (X_ti,G, pG_t ) dt hence

Ji,N(−∇u) − Ji,N(G) ≤ N

where N = (1) N − (2) N .

It is thus sucient to show that limN →∞ (i) N = 0, i = 1, 2. One has (1)_N = 1 2E ˆ T 0

|∇ut, ¯X_ti,N|2_{− |∇u}_{t, X}i,−∇u t |2_dt + E ˆ T 0

F ( ¯X_ti,N, ¯S_tN) − F (X_ti,−∇u, p−∇u_t )

dt

(37)

1.4. COMPETITION WITH A MAJOR PLAYER 37 ≤ k∇uk_∞ D2u _∞E ˆ T 0 ¯ X_ti,N − X_ti,−∇u dt + LFE ˆ T 0 ¯ X_ti,N − X_ti,−∇u + d1( ¯S N t , p −∇u t ) dt .

We then apply Theorem 42 and deduce limN →∞ (1) N = 0.

Finally, one has

(2)_N = 1 2E ˆ T 0 |Gt, X_ti,N|2_{− |G}_{t, X}i,G t |2_dt + E ˆ T 0 F (X_ti,N, S_tN) − F (X_ti,G, pG_t )dt ≤ kGk_∞kDGk_∞_E ˆ T 0 X i,N t − X i,G t dt + LFE ˆ T 0 X i,N t − X i,G t + d1(S N t , p G t ) dt .

We then apply Theorem 42 and deduce limN →∞ (2)

N = 0. The proof of Theorem 47

is complete.

1.4 Competition with a major player

1.4.1 The philosophy: Mean Field Game and Mechanism

De-sign

The starting point of the paper is the theory of Mechanism Design, which can be introduced by the following words:

Philosophers and social scientists have long realized that it is not necessary that all citizens strive to enhance social welfare for the outcome of their joint actions to be nevertheless good for society at large. Adam Smith's classical metaphor of the "invisible hand" suggests precisely this: how markets under ideal conditions lead to an ecient allocation of resources even when all agents are motivated by their self-interest.

(38)

This is how Professor Weibull started his speech to confer The Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel, 2007, to Leonid Hurwicz, Eric S. Maskin, Roger B. Myerson for having laid the foundations of mechanism design theory.

Mechanism design theory provides general methods for the analysis and development of mechanisms for resource allocation. This analysis is carried out in three steps. First, one makes a prediction of the behavior that is expected under given rules. Here, game theory comes to use. Thereafter, one evaluates, according to the given goal, the resource allocations - such as consumption, production and environmental stress - that result. Finally, one looks for the mechanism, with due regard to its behavioral implications, that best meets the goal.

The idea behind Mechanism Design is the introduction of a major player, called the Principal, who has the capability of establishing the rule of the game in order to obtain a particular result, for example minimizing a payo function: in other words the principal can choose the rule of the game in order to induce the players to act according to what is convenient for him.

The philosophy of Mechanism Design is here used in the context of Mean Field Games: we consider situations in which there is a huge number of indistinguishable players with symmetrical payos and a major one, that is the principal, who can choose the rules of the competition in order to achieve the minimization of a func-tional, representing principal's cost.

The mathematical setting is not easy to be dealt with because of its complexity: we are going to give approximated results for existence of minimum for principal's payo.

1.4.2 A motivating example

The problem of a principal who has to decide the rules of a competition is very re-alistic and can have applications in many decision making problems. Here we give a rst example of the importance of such a problem.

Consider a CEO of a company handling supermarkets who has to decide how dis-tributing n supermarkets in an area. He faces the following facts:

(39)

1.4. COMPETITION WITH A MAJOR PLAYER 39 2. each supermarket can receive a nite number of persons, so people don't want

to be too much concentrated at any point;

3. the company has to face costs depending on the number of supermarkets and their collocation (if they are too far one from each other it arises the costs); 4. minimum cost and maximum gain want to be achieved.

In such a competition between the players, the rules of the game are represented by the distribution of the supermarkets.

We model the situation both from the players (buyers) and from the major player (CEO) point of view. The players have to face the following problem: each of them decides a drift αi _{(the velocity) in order to minimize a payo on the form:}

Ji _{= E} "ˆ _T 0 1 2|α i t| 2 + n X k=1 |X_ti− ak| + 1 N − 1 X j6=i K(X_ti− X_tj)dt # , where Xj

t is the position for the i-th player at time i, ak is the position of the k−th

supermarket and K is a function growing near zero. The rst term of the integral represents the energy spent by the i-th player to move, the second term is the distance between the i−th player and the supermarkets, the last term represents the fact that two players do not want to be too crowded. The dynamic described by this game models in some way the point 1 and 2.

As previous said, the CEO wants to choose the rules of the games in order to minimize costs and maximize gains, that is he decides the number n and the positions ak in

order to minimize a functional on the form JP _{= E} "ˆ T 0 1 N N X j=1 n X k=1 |X_tj − ak|dt # +1 2 X h6=k |ak− ah|.

Is it possible for the CEO to nd the number n and the positions ak to minimize its

functional JP? More generally, can the major player choose the rules of the game

in order to minimize its functional? We want to give a rst answer to this problem from a more general point of view, considering a generic competition with a huge number of players and a major one who has the capability of deciding the rules of the game.

(40)

1.4.3 The mathematical setting: principal's payo

Consider the following problem. We have N players and a major one, called the Principal, who can establish the rules of the game. The N players act as in the classical theory of Mean Field Games: each player can choose a control αi,which

determines the state Xi

t through a SDE of the form

dX_ti = αi_tdt +√2dB_ti with the aim to minimize the payo

J_iN(α1, · · · , αN_{) = E} "ˆ _T 0 1 2|α i s| 2_{+ F} _Xi s, 1 N − 1 X j6=i δ_Xj s !# ,

which however depends on the controls chosen by the other players; thus the players will accept the compromise of a Nash equilibrium.

Here the Principal appears: he has the capability of choosing the rules of the com-petition. In this context we choose as the rules of the game the functional F as it represents the way through which the players interact. The principal can choose the functional F in order to achieve a particular result: in this context the aim of the principal is minimizing a payo function.

At this point we need to introduce principal's payo function: we suppose that it is in the form J_NP(α1, · · · , αN_{, F ) = E} "ˆ _T 0 φ 1 N N X i=1 δ_Xi t ! dt # + J0(F )

where Xi _{are the states associated to the controls α}i_.

We suppose that J0 _{is a semi continuous functional such that if}

kFnklip→ ∞

and

kFnk∞→ ∞

then

J0(Fn) → ∞.

The payo of the Principal depends on the actions αi of the players. This arises a

(41)

1.4. COMPETITION WITH A MAJOR PLAYER 41 The diculty is similar to the classical one in game theory with the dierence that the Principal is not like another player and a concept of Nash equilibrium between the N players and the Principal does not look suitable: because the Principal acts rst, choosing the rules of the game, and only then the players make their choices. The way we exit by this ambiguity of formulation is to prescribe the strategy of the players, given F . Assume we have a map F 7→ (α1,F_{, · · · , α}N,F₎_{which prescribes the}

choice (α1,F_{, · · · , α}N,F₎ _{made by the player, for each given F . Clearly the Principal}

payo becomes dependent only on F

J_NP(F ) := J_NP(α1,F, · · · , αN,F, F ) and we may formulate a classical minimization problem.

1.4.4 A sub-optimal result

Trying to minimize the functional JP

N is a very hard task, so we introduce an

approx-imation of it: we consider ˜ JP_{(F ) = E} ˆ T 0 φ(mF_t) + J0(F )

where (uF_{, m}F₎ _{is the solution of the mean eld equation with the functional F .}

Let us introduce some notation: if F is xed, we dene ˜

α(x, t) = −∇xuF(x, t);

if Xi,F solves the SDE

dX_ti,F = ˜α(X_ti,F, t)dt +√2dB_ti. we put

αF,i = ˜α(X_ti,F, t)

The link between the two functionals is given by the following theorem: Theorem 22 Suppose that F∗ _{∈ arg min ˜}_JP_{(F )}. Then

J_NP(αF∗,1, · · · , αF∗,N, F∗) ≤ inf F J P N(α F,1_{, · · · , α}F,N_{, F ) +} N. with N → 0 as N → ∞.

(42)

Let us summarize the intuition behind this result. The Principal may choose F but not α1_{, · · · , α}N

; the Principal has no way to minimize the payo JP

N(α1, · · · , αN, F )

unconditionally. The only way is to work conditionally to a presumed strategy of action of the players. Here we presume the Mean Field action, which is motivated by its simplicity for a very complex game problem. Conditional to this belief, the Principal could try to minimize the functional JP

N(F )given by equation (1), but this

is still a formidable task. Hence the Principal may choose F by means of the Mean Field payo ˜JP, getting a sub-optimal result, which is much easier than J_NP(F ). Principal's belief about the fact that the agents will act according to a Nash equilib-rium strategy is justied by classical game theory: in some way, before choosing F , the principal trusts the agents to act according to Nash equilibrium (thanks to game theory) and following the Mean Field model (justied by their symmetrical payos and the theory of Mean Field Game). Supposing that the agents will act according the rules predicted by Mean Field Games, the principal chooses the function F in order to minimize ˜JP _{. The next step is studying existence of minimum for the}

approximating functional.

1.4.5 Continuity of the approximating functional

In order to study continuity of the approximating functional, we need to know in-formation about continuous dependence between the functional F and the solutions (uF_{, m}F₎ _{of the mean eld equation associated to F given by:}

       −∂tu − ∆u + 1₂|∇u|2 = F (x, m) ∂tm − ∆m−div(m∇u) = 0 u(x, T ) = 0 m(0) = m0,

Lion has proved the following fact:

Theorem 23 Suppose that the following hypothesis are satised:

• Uniform boundedness and lipschitzianity: there exists a constant C0 for which

sup

x∈Rd_,m∈P 1

|F (x, m)| ≤ C0,

(43)

1.4. COMPETITION WITH A MAJOR PLAYER 43 for all x, x0

∈ Rd, for all m, m0 _{∈ P}

1, where d1 the Kantorovitch- Rubinstein

distance dened as d1(µ, ν) = inf γ∈Π(µ,ν) ˆ Rd |x − y|dγ(x, y) ,

where Π(µ, ν) is the set of Borel probability measures on R2d _{such that γ(A ×}

Rd) = µ(A) and γ(Rd× B) = ν(B) for all A, B Borel sets of Rd. • Weak monotonicity: for all m, m0 _{∈ P}

1, we have

ˆ

Rd

(F (x, m) − F (x, m0))d(m − m0)(x) ≥ 0. Then we have strong existence and weak uniqueness.

We will show continuity with respect to the following topologies: • topology for F : uniform convergence on compact sets; • topology for u: uniform convergence on compact sets; • topology for m: topology dened by the distance:

d(ν, µ) = sup

t∈[0,T ]

d1(ν(t), µ(t)),

where d1 is the 1- Wasserstein distance.

The theorem stating continuity with respect to these topologies is the following: Theorem 24 If Fn → F, then uFn → uF and mFn → mF.

The proof is carried out in the following steps:

1. for all F, uF _{and ∇u}F _{are uniformly bounded, m}F _{admits second moment and}

sup

t,s∈[0,T ]

d1(mF(t), mF(s))

|t − s|1/2 ≤ C;

(44)

2. if Fn→ F, then uFn → uFand mFn → mF; the prove of this second fact follows

the steps:

• using Ascoli Arzelá, prove that uFn converges u;

• using compactness, prove that mFn converges to some m;

• prove that (u, m) solves the Mean Field Equation associated to F in a weak sense;

• use weak uniqueness of the Mean Field Equation.

At this point we have the ingredients to study the approximating functional. Theorem 25 The functional ˜JP is semi-continuous.

1.4.6 Minimum for the approximating functional

In this subsection we give an idea of how proving the existence of minimum for the approximating functional.

The main result is the following:

Theorem 26 There exists a minimum for the functional ˜JP. The proof of this result is carried out in the following steps:

1. consider a minimizing sequence Fn;

2. use Ascoli Arzelà theorem and prove that there exists F such that Fn→ F;

3. prove that F belongs to the domain of ˜JP;

4. use semi-continuity of ˜JP_.

In this way existence of minimum is thus proved.

The problem of uniqueness of minimum is very hard because standard methods of convex analysis fail: it is not clear if the functional ˜JP _{has some convexity properties.}

In the next section we give an idea of a dierent approach to the problem which could lead to some results about uniqueness and characterization of minimum.

(45)

1.4. COMPETITION WITH A MAJOR PLAYER 45

1.4.7 An Hamilton Jacobi Bellman approach

Here we give an idea to a new approach to the problem using Hamilton Jacobi Bell-man equations and dynamic programming theory. This approach could lead to some results regarding uniqueness of minimum that can not be obtained using convex analysis.

Fix T > 0. For s, S ∈ [0, T ] with s ≤ S,consider for (t, x) ∈ [s, S] × Rd _{the following}

system:        −∂tu − ∆u + 1₂|∇u|2 = F (x, m) ∂tm − ∆m−div(m∇u) = 0 u(x, S) = G(x) m(s) = µ(x).

We consider (u, m) as the state and F as the control. Consider the functional

J (s, S, µ, G, F ) = ˆ S s E h φ(X_tF,G,µ)idt = ˆ S s f (t, m, u, F )dt, where XF,G,µ

t solves the SDE

dXt= −∇u(t, Xt)dt +

√ 2dBt

L(Xs) = µ

Consider the value function given by V (s, S, µ, G) = inf

F J (s, S, µ, G, F ).

Then we have the following theorem:

Theorem 27 (Bellmann's Principle of optimality (forward)): the value function V satises the following equation

V (s, S, µ, G) = inf F ˆ ¯s s f (t, m, u, F )dt + V (¯s, S, m(¯s), G) for all ¯s ∈ [s, S].

(46)

The proof is carried put in the following steps: 1. dene ¯ V (s, S, µ, G) = inf F ˆ s¯ s f (t, m, u, F, G)dt + V (¯s, S, m(¯s), G) .

2. for all F we have:

V (s, S, µ, G) ≤ J (s, S, µ, G, F ) = ˆ s¯ s f (t, m, u, F, G)dt + ˆ S ¯ s f (t, m, u, F, G)dt = ˆ ¯s s f (t, m, u, F, G)dt + J (¯s, S, m(¯s), G, F ), that is V (s, S, µ, G) ≤ ¯V (s, S, µ, G).

3. let > 0; there exists F such that

V (s, S, µ, G) + ≥ J (s, S, µ, G, F)

≥ ˆ ¯s

s

f (t, m, u, F, G)dt + V (¯s, S, m(¯s), G) ≥ ¯V (s, S, µ, G).

Starting from this equation, it would be interesting to develop the theory of optimiza-tion for this problem and obtain the PDE satised by V : this could give informaoptimiza-tion about uniqueness of the minimum of our problem, using uniqueness of the solution V of the PDE.

1.5 Mean eld interactions with a major player

In this chapter we introduce an external player, the Principal, in the context of mean eld game with mean eld interactions. The Principal has to minimize a payo in a proper sense.

(47)

1.5. MEAN FIELD INTERACTIONS WITH A MAJOR PLAYER 47

1.5.1 Introduction

Consider the situation of mean eld game with mean eld interactions in which there are N players and for all i = 1, · · · , N the dynamics of the i−th player is given by:

dX_ti,αi = αi_tdt + b X_ti,αi, 1 N N X j=1 δ X_tj,αj ! dt +√2dB_ti where (α1_{, · · · α}N₎ _{are the controls chosen by the players.}

Each player wants to minimize a payo function given by Ji(α1, · · · , αN_{) = E} "ˆ T 0 1 2|α i t| 2_{+ F} _Xi,αi t , 1 N − 1 X j6=i δ Xj,αjt ! dt # .

The dynamic of the system depends on the functional F which determines how much each player has to pay.

We want to introduce an external player, the principal, who can choose the rules of the game.

Thus it is natural to suppose that the principal is able to choose the functional F , in order to minimize a functional, having the following form

J_NP(α1, · · · , αN, F ) = E "ˆ _T 0 φ 1 N N X i=1 δ X_ti,αi ! dt # + J0(F ),

where Xi,αi satises

dX_ti,αi = αi_tdt + b X_ti,αi, 1 N N X j=1 δ X_tj,αj ! +√2dB_ti.

The idea is the following: the principal choose the function F which establishes the rules of the game. Then the players choose their strategies: we expect them to act according to Nash equilibrium as Lions has proved. After that, the principal receives the payo JP_{, depending on the strategies the players have played.}

(48)

1.5.2 A sub-optimal result

Computationally speaking, it is dicult to work with such a functional. We are going to introduce a functional supporting us:

˜ JP_{(F ) = E} ˆ T 0 φ X_tF dt + J0(F ). where XF

t is a random variable with law µF and (uF, µF)is the solution of the mean

eld equation given by       

−∂tu − ∆u − hb(·, µ), ∇ui + 1₂|∇u|2 = F (x, µ)

u(x, T ) = 0 µ(0) = µ0,

For xed F , we dene

˜

α(x, t) = −∇uF(x, t); if Xi,F _{solves the SDE}

dX_ti,F = ˜α(X_ti,F, t)dt + b(X_ti,F, µF_t )dt +√2dB_ti. we put

αF,i = ˜α(X_ti,F, t)

Now the following theorem explain why JP _{is related to ˜}_JP _:

Theorem 28 Suppose that F∗ _{∈ arg min ˜}_JP_{(F )}_{. Then}

J_NP(αF∗,1, · · · , αF∗,N, F∗) ≤ inf

F J P

N(αF,1, · · · , αF,N, F ) + N.

with N → 0as N → ∞.

The idea behind the result is that the minimization of ˜JP _{gives us a sub-optimal}

minimization to JP, since dealing directly with the problem of minimizing JP is a

very hard task.

(49)

1.5. MEAN FIELD INTERACTIONS WITH A MAJOR PLAYER 49

1.5.3 Semicontinuity of ˜

J

P

The rst step to get semicontinuity of ˜JP is the continuous dependence between F

and the solution of the mean eld equation, as stated by the following theorem: Theorem 29 Suppose that a sequence Fn converges to some F locally uniformly.

Then, up to a subsequence, uFn converges to uF locally uniformly and µFn converges

to µF _{in d.}

Thanks to this theorem, semicontinuity is an easy task: Theorem 30 The functional ˜JP _{is semi continuous.}

Proof. Consider a sequence Fn → F locally uniformly. Let γt be a measure on R2d

with marginal measures µF

|J1_(F n) − J1(F )| ≤ C ˆ T 0 d1(µFt , µ Fn t ) ≤ CT d(µ F_{, µ}Fn_{) → 0.}

(50)

1.5.4 Existence of minimum for ˜

J

P

At this point we show existence of minimum for ˜JP.

Theorem 31 There exists a minimum for ˜JP_.

Proof. Consider a minimizing sequence Fn. Using Ascoli Arzela' and a diagonal

procedure, it is possible to get a subsequence nk and a function F such that

Fnk → F

locally uniformly. It is very easy to check that F belongs to Γc. Thanks to

semicon-tinuity, we get that

˜

JP(Fnk) → ˜J

P_{(F ),}

(51)

Part I

(52)

(53)

Chapter 2 Mean eld game with mean eld

interaction

2.1 Introduction

In this chapter we study mean eld games where the players interact as inspired by mean eld theory.

The problem has already been studied in [11] in the context of crowd dynamics with an applied perspective. Here we give an equivalent formulation of the result.

Consider N players and for all i = 1, · · · , N the dynamics of the i−th player is given by: dX_ti,αi = α_tidt + b X_ti,αi, 1 N N X j=1 δ Xtj,αj ! +√2dB_ti where (α1_{, · · · α}N₎ are the controls chosen by the players.

As to the initial states, we suppose that

E[X0i] ≤ C.

Notice that the mean eld interaction term, that is b, does not depend on N; this situation will be studied in chapter 5.

We suppose that b satises the following hypothesis: 53

(54)

1. b is bounded;

2. b is lipschtzian, that is

|b(x, m) − b(x0, m0)| ≤ C(|x − x0| + d1(m, m0)).

3. ∇b is lipschitizian, that is

|∇b(x, m) − ∇b(x0, m0)| ≤ C(|x − x0| + d1(m, m0)).

For example, b could be in the form

b(x, µ) = (k ∗ µ)(x)

for a proper kernel k being in L∞ _{such that k and ∇k is lipschitzian.}

In fact we have: |b(x, µ) − b(x0, µ0)| ≤ ˆ k(x − y)µ(dy) − ˆ k(x0− y)µ0(dy) ≤ ˆ k(x − y)µ(dy) − ˆ k(x0− y)µ(dy) + ˆ k(x0 − y)µ(dy) − ˆ k(x0− y)µ0(dy) ≤ ˆ |k(x − y) − k(x0− y)|µ(dy) + lip(k) ˆ 1 lip(k)k(x 0_{− y)µ(dy) −} ˆ 1 lip(k)k(x 0_{− y)µ}0 (dy) ≤ C(|x − x0_{| + d} 1(µ, µ0)).

For each player, we can consider a payo function given by: Ji(α1, · · · , αN_{) = E} "ˆ _T 0 1 2|α i t| 2_{+ F} _Xi,αi t , 1 N − 1 X j6=i δ X_tj,αj ! dt # .

(55)

2.2. MAIN RESULT 55

2.2 Main result

In this section, we give the main result of the minimization problem stated in the previous section: the minimization is given in a − Nash sense.

Theorem 32 Fix a solution (u, µ) of the following mean eld equation:       

−∂tu − ∆u − hb(·, µ), ∇ui + 1₂|∇u|2 = F (x, µ)

u(x, T ) = 0 µ(0) = µ0,

Put for all i

dX_ti = −∇u(X_ti)dt + b X_ti, 1 N N X j=1 δ_Xj t ! +√2dB_ti and ¯ αi = −∇u(Xi).

Then (¯α1_{· · · , ¯}_αN₎ _{is an -Nash equilibrium for (J}1_{, · · · , J}N₎_.

To prove the main theorem, we need to x some notation. Consider a general control α1_{. Recall that (X}i₎N

i=1 are the solutions of the system of

SDEs given by dX_ti = −∇u(X_ti)dt + b X_ti, 1 N N X j=1 δ_Xj t ! +√2dB_ti for all i = 1, · · · , N. We write SN,ˆ1 = 1 N − 1 X i≥2 δXi and SN = 1 N X i≥1 δXi.

(56)

We denote with ( ¯Xi₎N

i=1 the solutions of the system of SDEs given by

d ¯X_ti = −∇u(X_ti)dt + b X¯_ti, 1 N N X j=1 δ_X¯j t ! +√2dB_ti for i ≥ 2 and d ¯X_t1 = α1_tdt + b X¯_t1, 1 N N X j=1 δ_X¯j t ! +√2dB_t1. We write ¯ SN,ˆ1 = 1 N − 1 X i≥2 δ_X¯i and ¯ SN = 1 N X i≥1 δ_X¯i.

In order to prove the main result we need three ingredients. We rstly need to solve an abstract problem dealing with our problem: it will be discussed in section 3. Secondly, we need convergence of the empirical measure of the optimal states (when the players choose the controls ¯αi) to the solution of the Fokker Planck of the mean

eld equation; it will be discussed in section 4. Thirdly, we need convergence of the empirical measure of the modied problem (the case in which the players choose the optimal controls except one who chooses a general control αi) again to the solution

of the Fokker Planck; it will be discussed in section 5.

2.3 An abstract problem

To solve our problem we need the following lemma. Lemma 33 Consider the payo function

J (α) = E ˆ T 0 1 2αt+ F (Xt, µt)dt conditioned to dXt= αtdt + b(Xt, µt)dt + √ 2dBt.

(57)

2.4. CONVERGENCE OF SN ₅₇ Put dX_tµ= −∇u(X_tµ)dt + b(X_tµ, µt)dt + √ 2dBt and ¯ αµ = −∇u(Xµ). Then ¯αµ _{minimizes the functional J}α_.

The proof is very easy: it is sucient to apply Ito formula and the use the mean eld equation.

2.4 Convergence of S

N

In order to get convergence we use tightness argument.

2.4.1 Tightness

Denote with QN the law of SN. Let's show the following proposition.

Proposition 34 Write

QN = L(SN) ∈ P(C([0, T ] : P1_(Rd))). Then the sequence QN _{is tight.}

Proof. Consider the set KM,R = ( µ ∈ P(C([0, T ] : P1_(Rd)))| sup t∈[0,T ] ˆ |x|µt(dx) ≤ M, ˆ T 0 ˆ T 0 d1(µt, µs) |t − s|1+αp ≤ R )

It is sucient to show that

(58)

for a proper choice of M, R. We have QN(K_M,Rc _{) ≤ P} ( sup t∈[0,T ] ˆ |x|S_tN(dx) > M ) + P ˆ T 0 ˆ T 0 d1(SsN, StN) |t − s|1+αp > R .

Consider the rst term: P ( sup t∈[0,T ] ˆ |x|SN t (dx) > M ) ≤ 1 ME " sup t∈[0,T ] ˆ |x|SN t (dx) # ≤ 1 M 1 N N X j=2 Esup |Xtj| We have that X_ti = X₀i + ˆ t 0 1 N X j K(X_si − Xj s) − ∇u(X i s)ds + √ 2dB_si, so |Xi t| ≤ |X0| + tkKk∞+ tk∇uk∞+ √ 2|B_ti|; then we get Esup |Xtj| ≤ C and P ( sup t∈[0,T ] ˆ |x|SN t (dx) > M ) ≤ C M → 0 as M → ∞.

Consider the second term: we have

P ˆ T 0 ˆ T 0 d1(SsN, StN) |t − s|1+αp dtds > R ≤ 1 RE ˆ T 0 ˆ T 0 d1(SsN, StN) |t − s|1+αp dtds = 1 R ˆ T 0 ˆ T 0 1 |t − s|1+αpE[d1(S N s , StN)p]dtds.

NewapproachestoMeanFieldGameTheory CorsodiDottoratoinMatematica Universita'degliStudidiPisa

Universita' degli Studi di Pisa

Facolta' di Scienze Matematiche Fisiche e Naturali

Corso di Dottorato in Matematica

New approaches to Mean Field Game

Theory

Pisa, 19/05/2017

Tesi di Dottorato

Relatore

Candidata

Abstract

Contents

I

51

II

103

Chapter 1

Introduction

1.1 State of art

1.1.1 A stochastic optimal control problem: the Mean Field

Game (MFG)

1.1.2 From game to PDEs: the Mean Field Equations (MFE)

1.1.3 The link between MFG and MFE

1.2 Mean eld games with mean eld interactions

1.2.1 Introduction

1.2.2 Main result

1.2.3 The three main ingredients of the proof

1.2.4 Proof of the main result

1.2.5 Existence and uniqueness for the solution of the MFE

1.3 Mean eld games with moderate interaction

1.3.1 Introduction

1.3.2 Convergence of X

and S

1.3.3 A rst open problem and a new approach

1.3.4 Some motivating examples: angiogenesis

1.3.5 Some motivating examples: cell adhesion

1.3.6 A second open problem

1.3.7 − optimality

1.4 Competition with a major player

1.4.1 The philosophy: Mean Field Game and Mechanism

De-sign

1.4.2 A motivating example

1.4.3 The mathematical setting: principal's payo

1.4.4 A sub-optimal result

1.4.5 Continuity of the approximating functional

1.4.6 Minimum for the approximating functional

1.4.7 An Hamilton Jacobi Bellman approach

1.5 Mean eld interactions with a major player

1.5.1 Introduction

1.5.2 A sub-optimal result

1.5.3 Semicontinuity of ˜

J

1.5.4 Existence of minimum for ˜

J

Part I

Chapter 2

Mean eld game with mean eld

interaction

2.1 Introduction

2.2 Main result

2.3 An abstract problem

2.4 Convergence of S

2.4.1 Tightness

1.2 Mean eld games with mean eld interactions

1.3 Mean eld games with moderate interaction

_{and S}

1.3.3 A rst open problem and a new approach

1.3.7 − optimality

1.4.3 The mathematical setting: principal's payo

1.5 Mean eld interactions with a major player

Mean eld game with mean eld