This session aims to mix a few talks concerning the asymptotics of E(·|G

This session aims to mix a few talks concerning the asymptotics of

E(·|G

)

when the conditioning σ-fields G

move. In the spirit of the martingale convergence theorem. Among the various questions:

• Is there a σ-field G such that E(Z|G

) → E(Z|G), in some sense, for sufficiently many random variables Z ?

• What about E(Z

|G

) if the Z

move as well ? Example:

G

= σ(X

, . . . , X

),

for some sequence X

, and Z

= 1

n+1∈B}

, so that

E(Z

|G

) = P (X

∈ B|X

, . . . , X

) predictive measure

1

• Assume a regular version of the conditional distribution given G

exists and define

a

(ω)(A) = P

(A|G

)

for each ω and each A in some sub-σ-field. Let µ

be a sequence of random probability measures and d a distance between probability measures. What about

d[ a

(ω), µ

(ω)] ?

Example: G

= σ(X

, . . . , X

),

a

(·) = P (X

∈ ·|X

, . . . , X

) predictive measure, µ

= (1/n)

δ

i

empirical measure

In some frameworks, µ

can be regarded as an ”estimate” of a

and it is natural to investigate the asymptotics of d(a

, µ

)

ASYMPTOTIC PREDICTIVE INFERENCE WITH

EXCHANGEABLE DATA

Patrizia Berti, Luca Pratelli, Pietro Rigo

Universita’ di Modena e Reggio-Emilia Accademia Navale di Livorno

Universita’ di Pavia

Torino, June 20, 2017

Framework

• (X

: n ≥ 1) sequence of random variables with values in a (nice) measurable space (S, B)

• µ

= (1/n)

δ

i

empirical measure

• a

(·) = P (X

∈ · | X

, . . . , X

) predictive measure

Under some conditions, µ

(B)−a

(B) → 0 a.s. for each B ∈ B. Thus, we fix D ⊂ B, and we investigate uniform convergence over D, i.e., we focus on

||µ

− a

|| = sup

|µ

(B) − a

(B)|

Motivations

• Bayesian predictive inference

• Empirical processes for non-ergodic data

• Predictive distributions of exchangeable sequences

• Frequentistic approximations of Bayesian procedures

4

Assumptions and possible generalizations

• From now on, (X

) is exchangeable. However, most results are still valid if (X

) is conditionally identically distributed, namely,

P (X

∈ · | X

, . . . , X

) = P (X

∈ · | X

, . . . , X

) a.s.

for each k > n ≥ 0

• Similarly, in most results, σ(X

, . . . , X

) can be replaced by any filtration G

which makes (X

) adapted

• To avoid measurability problems, D is countably determined, i.e.

sup

|α(B) − β(B)| = sup

|α(B) − β(B)|

for some countable D

⊂ D and all probabilities α and β on B

Goals

• To give conditions for

||µ

− a

|| → 0 a.s. or in probability

• To determine the rate of convergence, i.e., to fix the asymptotic behavior of

r

||µ

− a

||

for suitable constants r

> 0, e.g.

r

= √

n or r

=

log log n

• Another issue, not discussed in this talk, is the limiting distribution (if any) of r

(µ

− a

) under some distance on the space l

(D)

6

Theorem 1

Recall that, because of exchangeability, there is a random probability measure µ on B such that

µ

(B) → µ(B) a.s. for each B ∈ B

If ||µ

− µ|| → 0 a.s. then ||µ

− a

|| → 0 a.s.

Examples: ||µ

− a

|| → 0 a.s. provided

• D is a universal Glivenko-Cantelli-class

• D = B and µ a.s. discrete (usual in Bayesian nonparametrics !)

• S = R

, D = {closed convex sets} and µ << λ a.s. where λ is a

(non random) σ-finite product measure

Theorem 2

Fix the constants r

> 0 and define M = sup

r

||µ

− µ||. Define also the ”exchangeable empirical process”

W

= √

n (µ

− µ)

If E(M ) < ∞ then lim sup

||µ

− a

|| < ∞ a.s.

Examples:

•

||µ

− a

|| → 0 a.s. if sup

E{||W

||

} < ∞ for some p > 2

•

||µ

− a

|| ≤

2

a.s. if D is a VC-class

8

Theorem 3

If sup

E{||W

||

} < ∞, for some p ≥ 1, then

r

||µ

− a

|| → 0 in probability whenever

n

→ 0

Remark: Thus, if sup

E{||W

||

} < ∞, then

q _n

log log n

||µ

− a

|| → 0 in probability

but a.s. convergence can not be obtained by the LIL

Theorem 4 Define

h

(B) = P {X

∈ B | µ

(B)}

Suppose that µ(B) has absolutely continuous distribution for each B ∈ D such that 0 < P (X

∈ B) < 1. Then, under mild technical conditions, one obtains

√ n ||µ

− a

|| → 0 in probability

if and only if

√ n {a

(B) − h

(B)} → 0 in probability for each B ∈ D

Open problem: Characterize the above condition. More generally, in addition to µ

(B), which further information is required to evaluate a

(B) ?

10