ASYMPTOTIC PREDICTIVE INFERENCE WITH EXCHANGEABLE DATA

ASYMPTOTIC PREDICTIVE INFERENCE WITH

EXCHANGEABLE DATA

Patrizia Berti, Luca Pratelli, Pietro Rigo

Universita’ di Modena e Reggio-Emilia Accademia Navale di Livorno

Universita’ di Pavia

Roma, September 12, 2016

Framework

• (X

: n ≥ 1) sequence of random variables with values in a (nice) measurable space (S, B)

• µ

= (1/n)

δ

i

empirical measure

• a

(·) = P (X

∈ · | X

, . . . , X

) predictive measure

Under some conditions, µ

(B)−a

(B) → 0 a.s. for each B ∈ B. Thus, we fix D ⊂ B, and we investigate uniform convergence over D, i.e., we focus on

||µ

− a

|| = sup

|µ

(B) − a

(B)|

Motivations

• Bayesian predictive inference

• Empirical processes for non-ergodic data

• Predictive distributions of exchangeable sequences

• Frequentistic approximations of Bayesian procedures

Assumptions and possible generalizations

• From now on, (X

) is exchangeable. However, most results are still valid if (X

) is conditionally identically distributed, namely,

P (X

∈ · | X

, . . . , X

) = P (X

∈ · | X

, . . . , X

) a.s.

for each k > n ≥ 0

• Similarly, in most results, σ(X

, . . . , X

) can be replaced by any filtration G

which makes (X

) adapted

• To avoid measurability problems, D is countably determined, i.e.

sup

|α(B) − β(B)| = sup

|α(B) − β(B)|

for some countable D

⊂ D and all probabilities α and β on B

Goals

• To give conditions for

||µ

− a

|| → 0 a.s. or in probability

• To determine the rate of convergence, i.e., to fix the asymptotic behavior of

r

||µ

− a

||

for suitable constants r

> 0, e.g.

r

= √

n or r

=

log log n

• Another issue, not discussed in this talk, is the limiting distribution

(if any) of r

(µ

− a

) under some distance on the space l

(D)

Theorem 1

Recall that, because of exchangeability, there is a random probability measure µ on B such that

µ

(B) → µ(B) a.s. for each B ∈ B

If ||µ

− µ|| → 0 a.s. then ||µ

− a

|| → 0 a.s.

Examples: ||µ

− a

|| → 0 a.s. provided

• D is a universal Glivenko-Cantelli-class

• D = B and µ a.s. discrete (usual in Bayesian nonparametrics !)

• S = R

, D = {closed convex sets} and µ << λ a.s. where λ is a

(non random) σ-finite product measure

Theorem 2

Fix the constants r

> 0 and define M = sup

r

||µ

− µ||. Define also the ”exchangeable empirical process”

W

= √

n (µ

− µ)

If E(M ) < ∞ then lim sup

||µ

− a

|| < ∞ a.s.

Examples:

•

||µ

− a

|| → 0 a.s. if sup

E{||W

||

} < ∞ for some p > 2

•

||µ

− a

|| ≤

2

a.s. if D is a VC-class

Theorem 3

If sup

E{||W

||

} < ∞, for some p ≥ 1, then

r

||µ

− a

|| → 0 in probability whenever

n

→ 0

Remark: Thus, if sup

E{||W

||

} < ∞, then

q _n

log log n

||µ

− a

|| → 0 in probability

but a.s. convergence can not be obtained by the LIL

Theorem 4 Define

h

(B) = P {X

∈ B | µ

(B)}

Suppose that µ(B) has absolutely continuous distribution for each B ∈ D such that 0 < P (X

∈ B) < 1. Then, under mild technical conditions, one obtains

√ n ||µ

− a

|| → 0 in probability

if and only if

√ n {a

(B) − h

(B)} → 0 in probability for each B ∈ D

Open problem: Characterize the above condition. More generally,

in addition to µ

(B), which further information is required to evaluate