COMPATIBILITY OF CONDITIONAL DISTRIBUTIONS

COMPATIBILITY OF CONDITIONAL DISTRIBUTIONS

Patrizia Berti, Emanuela Dreassi, Pietro Rigo

Universit` a di:

Modena e Reggio-Emilia, Firenze, Pavia

Pisa, December 7, 2014

THE PROBLEM

Up to technicalities, the problem can be stated as follows

Let X and Y be real random variables. Suppose we are given the (measurable) kernels

α = {α(x, ·) : x ∈ R} and β = {β(x, ·) : x ∈ R}

where α(x, ·) and β(x, ·) are probability measures on R.

Is there a joint distribution for (X, Y ) such that

P (Y ∈ · | X = x) = α(x, ·) and P (X ∈ · | Y = y) = β(y, ·)

for almost all (x, y) ∈ R

?

MORE GENERALLY

The previous problem is actually a special case of the following Let X

, . . . , X

be real random variables and

Y

= (X

, . . . , X

, X

, . . . , X

), 1 ≤ j ≤ k.

Suppose we are given the (measurable) kernels α

= {α

(y, ·) : y ∈ R

}

where α

(y, ·) is a probability measure on R.

Is there a joint distribution for (X

, . . . , X

) such that P (X

∈ · | Y

= y) = α

(y, ·)

for all 1 ≤ j ≤ k and almost all y ∈ R

?

If yes, the kernels α

, . . . , α

are compatible (or consistent).

MOTIVATIONS AND REMARKS

(i) We focus on real random variables for the sake of simplicity.

Nothing important changes if each X

takes values in a measurable space (X

, B

). Similarly, (X

, . . . , X

) can be replaced by an infinite sequence X

, X

, . . .

(ii) Let C be a collection of probability measures on R

. The kernels α

, . . . , α

are C-compatible if they are the conditional distributions induced by some member of C.

(iii) Conceptually, we are just dealing with a coherence problem within the Kolmogorovian setting. Indeed, those people familiar with de Finetti’s coherence can think of this problem as

Coherence + Disintegrability

Technically, however, the problem is much harder than Finetti’s co-

herence

(iv) In some statistical frameworks, such as

Gibbs sampling, Multiple data imputation, Spatial statistics the conditional distributions of

X

given Y

= (X

, . . . , X

, X

, . . . , X

)

are needed. Sometimes, such conditional distributions are not derived from a joint distribution for (X

, . . . , X

). Rather:

One first selects the kernels α

, . . . , α

, with each α

regarded as the conditional distribution of X

given Y

, and then checks compatibility of α

, . . . , α

.

Note that, if α

, . . . , α

fail to be compatible, any subsequent anal- ysis does not make sense !!!

(v) Compatibility issues also occur in

Statistical mechanics, Machine learning, Bayesian statistics

RESULTS

In BDR (Stochastics, 2013; J. Multivariate Analysis, 2014) compat- ibility of α

, . . . , α

is characterized under the assumptions that

(i) Each X

has compact support or

(ii) Each kernel α

has a density with respect to some reference measure λ

, namely α

(y, dx) = f

(x | y) λ

(dx)

Furthermore,

(iii) C-compatibility of α

, . . . , α

is characterized for C = {exchangeable laws on R

}

C = {identically distributed laws on R

}

We next focus on (iii)

THM1: Let C = {exchangeable laws on R

}. Suppose α

= . . . = α

= α and α(y, ·) = α[π(y), ·]

for all permutations π. Then,

α

, . . . , α

are compatible ⇔ α

, . . . , α

are C-compatible

THM2: Let C = {identically distributed laws on R

}. Suppose k = 2; X

, X

take values in a finite set; α

irreducible

Then, α

, α

are C-compatible if and only if α

(x, y) > 0 ⇔ α

(y, x) > 0

Q_n

i=1

α

(x

, x

) =

α

(x

, x

) if x

= x

where α

(x, y) = α

(x, {y}). Such characterization easily extends to

X

, X

taking values in a countable set

EXAMPLE Suppose

α

= . . . = α

= α with α(y, ·) = N (b y, 1)

where b ∈ R and y = (1/(k − 1))

y

is the sample mean.

By THM1, those values of b which make α

, . . . , α

compatible can be determined. For instance,

For k = 2: α

, α

are compatible ⇔ b ∈ (−1, 1)

For k = 3: α

, α

, α

are compatible ⇔ b ∈ (−2, 1)

and so on. Also, for any k, α

, . . . , α

are compatible if and only if

they are induced by an exchangeable law on R

EXAMPLE

Suppose k = 2 and X

, X

take values in Z = {. . . , −1, 0, 1, . . .}

Let α be the kernel of the symmetric random walk on Z:

α(i, i − 1) = α(i, i + 1) = 1/2

for each i ∈ Z, where α(i, j) = α(i, {j}).

Are there a kernel β on Z and a joint distribution for (X

, X

) such that

X

∼ X

and

P (X

= j | X

= i) = α(i, j), P (X

= j | X

= i) = β(i, j) ?

The answer is NO, as a consequence of THM2