Show that PnXn+h= h−1 X j=1 ϕjPnXn+h−j+ p X j=h ϕjXn+h−j+ q X j=h ϑn+h−1,j(Xn+h−j− ˆXn+h−j) where the sums are considered empty if the lower index is larger than the upper, and ϕj≡ 0 for j &gt

Exercises # 6 6.1. Let {Xt} be a stationary process with mean 0. Define

P_n = P_L(X1,...,Xn) and Xˆ_n+1= P_nX_n+1. (a) Show that, for h > 1, PnXn+h= PnXˆn+h.

(b) Show that

PnXn+h=

n+h−1

X

j=h

ϑn+h−1,j(Xn+h−j− ˆXn+h−j) where ϑ_n,j are the coefficients determined by the innovations algorithm.

(c) Let {Xt} be a causal ARMA(p,q) process, and let n > max{p, q}. Show that

PnXn+h=

h−1

X

j=1

ϕjPnXn+h−j+

p

X

j=h

ϕjXn+h−j+

q

X

j=h

ϑn+h−1,j(Xn+h−j− ˆXn+h−j)

where the sums are considered empty if the lower index is larger than the upper, and ϕ_j≡ 0 for j > p.

(d) The previous formula is a recursive method for computing PnXn+hstarting from h = 1.

Write explicitely the formula for PnXn+2 in case of an ARMA(1,1) process.

(e) Define σ²(h) = E(X^n+h−PnXn+h)²the h-step prediction error. Prove for an ARMA(p,q) process

σ²(2) = vn+1+ (ϕ1+ ϑn+1,1)²vn for n > max{p, q}

Hint: compute σ²(2) by writing Xn+2− PnXn+2= (Xn+2− ˆXn+2) + ( ˆXn+2− PnXn+2).

6.2. Let ˜Pn the projection on Mn the linear space generated by {Xj, j ≤ n}.

Assume that {Xt} is a causal and invertible ARMA(p,q) process, i.e. we can write

X_t=

∞

X

j=0

ψ_jZ_t−j Z_t= X_t+

∞

X

j=1

π_jX_t−j.

(a) Show that ˜PnXn+hcan be computed recursively from P˜_nX_n+h= −

h−1

X

j=1

π_jP˜_nX_n+h−j−

∞

X

j=h

π_jX_n+h−j.

(b) Show also that

P˜nXn+h=

∞

X

j=h

ψjZn+h−j.

(c) Conclude that

σ˜²(h) = E(Xn+h− ˜P_nX_n+h)²= σ²

h−1

X

j=0

ψ_j².

6.3. Let {Xt} be a stationary process with mean 0. Define ui(t) (i ≤ t < n) be the difference between the value of X_n−t+i and the best linear prediction using the previous i values, i.e.

ui(t) = Xn−t+i− P_{L(Xn−t,...,Xn−t+i−1)}Xn−t+i.

Symmetrically, let v_i(t) (1 ≤ t < n) be the difference between the value of X_n−t and the best linear prediction using the following i values, i.e.

vi(t) = Xn−t− P_L(Xn−t+1,...,X_n−t+i)Xn−t

For i = 0, one defines u0(t) = v0(t) = Xn−t.

(a) Show, using the definition of projection, that

u_i(t) = X_n−t+i−

i

P

k=1

φ_i,kX_n−t+i−k vi(t) = Xn−t−

i

P

k=1

φi,kXn−t+k

and derive the equations satisfied by the coefficients φi,j.

(b) Remembering that the coefficients φi,j can be chosen according to Durbin-Levinson algorithm that yields for j = 1, . . . , i − 1 (i > 1)

φi,j= φi−1,j− φi,iφi−1,i−j

show that one obtains the recursion

u_i(t) = u_i−1(t − 1) − φ_i,iv_i−1(t) v_i(t) = v_i−1(t) − φ_i,iu_i−1(t − 1), 1 ≤ i ≤ t < n. (1) (c) Define

d(i) =

n−1

X

t=2

(u²_i(t − 1) + v_i²(t))

and

σ_i²= 1 2(n − 1)

n−1

X

t=i

(u²_i(t) + v²_i(t))

Show, using (1), that

σ²_i = 1

2(n − 1) d(i − 1) − 4φ_i,i

n−1

X

t=i

u_i−1(t − 1)v_i−1(t) + φ²_i,id(i − 1)

! .

(d) Compute the minimum of σ²_i with respect to φi,i showing that it is obtained at

φ^B_i,i= 2 d(i − 1)

n−1

X

t=i

ui−1(t−1)vi−1(t) and (σ²_i)^B= 1

2(n − 1)d(i−1) 1 − (φ^B_i,i)²
(2)

[the superscript B stands for Burg].

(e) Show that one obtains finally the iteration

d(i) = d(i − 1) 1 − (φ^B_i,i)²
− u²_i(n − 1) − v²_i(i).

(f) Show that φ^B_1,1 is very close to ˆρ(1) and specify the difference. Analogously show that φ^B_2,2 is an estimate of α(2), the partial correlation coefficient.

6.4. (5.11 of ITSM ) Given two observations x1and x2 from the causal AR(1) process satisfying Xt= ϕXt−1+ Zt, {Zt} ∼ W N (0, σ²)

and assuming that |x₁| 6= |x2|, find the maximum likelihood estimates of ϕ and σ².

6.5. (5.12 of ITSM ) Derive a cubic equation for the maximum likelihood estimate of the coefficient ϕ of a causal AR(1) process based on the observations X₁, . . . , X_n.