Consider the general linear model Y = Xβ + E, where Y = (y1

Statistical models A.Y. 2014/15

Written exam of March 27, 2015.

1. Consider the general linear model Y = Xβ + E, where Y = (y1, . . . , yn) is a vector in Rⁿ, X is a matrix n × p and E are random variables with mean 0.

(a) Show that ˆβ = (X^tX)⁻¹X^tY is an unbiased estimator of β. Compute also V ( ˆβ). This choice is generally called the least square method; what does this exactly mean?

(b) Two possible motivations for the choice of ˆβ are maximum likelihood estimation, or Gauss-Markov theorem. State precisely the results (proofs are not necessary, but, if time allows, they are welcome) and the assumptions on the errors E for either result.

(c) Let ˆε = Y − X ˆβ the observed residuals; prove that V (ˆε_i) = σ²(1 − H_ii) where H = X(X^tX)⁻¹X^t and σ² = V (E_j) for each j. Explain why, if H_ii is close to 1 for some value of i, this results makes us expect that ˆy_iwill be close to yi.

(d) Are ˆεiand ˆεj independent for i 6= j? [Hint: check the proof of the previous result]

2. (a) Write down the matrix X and the vector β in order to set the linear regression y_i = a + bx_i+ ε_i, i = 1 . . . n as a general linear model Y = Xβ + E.

(b) How can be obtain a confidence interval for the parameter b? Write down the elements of the formula, not necessarily the explicit computation.

(c) When is the matrix (X^tX) not invertible?

(d) Assume that the number of points n = 10 and there exists a qualitative variable Z such that Zi = A for i = 1 . . . 4 and Zi = B for i = 5 . . . 10. Write down the model

y_i =

(µ_A+ bx_i+ ε_i if Z_i = A µ_B+ bx_i+ ε_i if Z_i = B

as a general linear model specifying the matrix X and the vector β (there are several ways to do this, but there is one that is preferable if we then wish to test the hypothesis µ_A= µ_B).

3. Write down, if possible, the following models as linear models, possibly after some transfor- mation? In all cases ε_i represent independent and equidistributed error terms, E(εi) = 0; a, b, c . . . parameters to be estimated.

(a) yi =

(a + b(x_i− 30) + ε_i if x_i < 30 a + c(30 − x_i) + ε_i if x_i > 30 (b) yi = ax^b_i(1 + ε_i)

(c) yi = a + bx_i x_i+ c + ε_i

(d) y_i = a cos(ωx_i+ ϕ) + b sin(ωx_i + ϕ) + ε_i

1