12 Multiple Linear Regression

Testo completo

(1)c12. .qxd 5/20/02 9:31 M Page 410 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 12. Multiple Linear Regression. CHAPTER OUTLINE. 12-1 MULTIPLE LINEAR REGRESSION MODEL 12-1.1 Introduction 12-1.2 Least Squares Estimation of the Parameters 12-1.3 Matrix Approach to Multiple Linear Regression. 12-1.4 Properties of the Least Squares Estimators 12-2 HYPOTHESIS TESTS IN MULTIPLE LINEAR REGRESSION 12-2.1 Test for Significance of Regression 12-2.2 Tests on Individual Regression Coefficients and Subsets of Coefficients 12-2.3 More About the Extra Sum of Squares Method (CD Only) 12-3 CONFIDENCE INTERVALS IN MULTIPLE LINEAR REGRESSION. 12-3.2 Confidence Interval on the Mean Response. 12-4 PREDICTION OF NEW OBSERVATIONS 12-5 MODEL ADEQUACY CHECKING 12-5.1 Residual Analysis 12-5.2 Influential Observations 12-6 ASPECTS OF MULTIPLE REGRESSION MODELING 12-6.1 Polynomial Regression Models 12-6.2 Categorical Regressors and Indicator Variables 12-6.3 Selection of Variables and Model Building 12-6.4 Multicollinearity 12-6.5 Ridge Regression (CD Only) 12-6.6 Nonlinear Regression Models (CD Only). 12-3.1 Confidence Intervals on Individual Regression Coefficients. LEARNING OBJECTIVES After careful study of this chapter, you should be able to do the following: 1. Use multiple regression techniques to build empirical models to engineering and scientific data 2. Understand how the method of least squares extends to fitting multiple regression models. 410.

(2) c12. .qxd 5/20/02 9:31 M Page 411 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 12-1 MULTIPLE LINEAR REGRESSION MODEL. 411. 3. Assess regression model adequacy 4. Test hypotheses and construct confidence intervals on the regression coefficients 5. Use the regression model to estimate the mean response and to make predictions and to construct confidence intervals and prediction intervals 6. Build regression models with polynomial terms 7. Use indicator variables to model categorical regressors 8. Use stepwise regression and other model building techniques to select the appropriate set of variables for a regression model CD MATERIAL 9. Understand how ridge regression provides an effective way to estimate model parameters where there is multicollinearity. 10. Understand the basic concepts of fitting a nonlinear regression model. Answers for many odd numbered exercises are at the end of the book. Answers to exercises whose numbers are surrounded by a box can be accessed in the e-Text by clicking on the box. Complete worked solutions to certain exercises are also available in the e-Text. These are indicated in the Answers to Selected Exercises section by a box around the exercise number. Exercises are also available for some of the text sections that appear on CD only. These exercises may be found within the e-Text immediately following the section they accompany.. 12-1 12-1.1. MULTIPLE LINEAR REGRESSION MODEL Introduction Many applications of regression analysis involve situations in which there are more than one regressor variable. A regression model that contains more than one regressor variable is called a multiple regression model. As an example, suppose that the effective life of a cutting tool depends on the cutting speed and the tool angle. A multiple regression model that might describe this relationship is Y 0 1x1 2x2 . (12-1). where Y represents the tool life, x1 represents the cutting speed, x2 represents the tool angle, and is a random error term. This is a multiple linear regression model with two regressors. The term linear is used because Equation 12-1 is a linear function of the unknown parameters 0, 1, and 2. The regression model in Equation 12-1 describes a plane in the three-dimensional space of Y, x1, and x2. Figure 12-1(a) shows this plane for the regression model E1Y 2 50 10x1 7x2 where we have assumed that the expected value of the error term is zero; that is E() 0. The parameter 0 is the intercept of the plane. We sometimes call 1 and 2 partial regression coefficients, because 1 measures the expected change in Y per unit change in x1 when x2 is held constant, and 2 measures the expected change in Y per unit change in x2 when x1 is held constant. Figure 12-1(b) shows a contour plot of the regression model— that is, lines of constant E(Y ) as a function of x1 and x2. Notice that the contour lines in this plot are straight lines..

(3) c12. .qxd 5/20/02 9:31 M Page 412 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 412. CHAPTER 12 MULTIPLE LINEAR REGRESSION x2 10 220 240. 8 203. 200 160 E(Y) 120. 6. 80. 4. 186 10. 40 0. 8 6 0. 2. 4 x1. 6. 8. 10 0. 2. 4. x2. 169. 2 67 0. 0. 84. 2. 101 4. (a). Figure 12-1. 118 6. 135 8. 152 10 x1. (b). (a) The regression plane for the model E(Y ) 50 10x1 7x2. (b) The contour plot.. In general, the dependent variable or response Y may be related to k independent or regressor variables. The model Y 0 1x1 2x2 p k x k ˛. ˛. (12-2). is called a multiple linear regression model with k regressor variables. The parameters j, j 0, 1, p , k, are called the regression coefficients. This model describes a hyperplane in the kdimensional space of the regressor variables {xj}. The parameter j represents the expected change in response Y per unit change in xj when all the remaining regressors xi (i j) are held constant. Multiple linear regression models are often used as approximating functions. That is, the true functional relationship between Y and x1, x2, p , xk is unknown, but over certain ranges of the independent variables the linear regression model is an adequate approximation. Models that are more complex in structure than Equation 12-2 may often still be analyzed by multiple linear regression techniques. For example, consider the cubic polynomial model in one regressor variable. Y 0 1x 2x 2 3 x 3 . (12-3). If we let x1 x, x2 x 2, x3 x 3, Equation 12-3 can be written as Y 0 1x1 2x2 3 x3 . (12-4). which is a multiple linear regression model with three regressor variables. Models that include interaction effects may also be analyzed by multiple linear regression methods. An interaction between two variables can be represented by a cross-product term in the model, such as Y 0 1x1 2x2 12 x1x2 If we let x3 x1x2 and 3 12, Equation 12-5 can be written as Y 0 1x1 2x2 3 x3 which is a linear regression model. Figure 12-2(a) and (b) shows the three-dimensional plot of the regression model Y 50 10x1 7x2 5x1x2. (12-5).

(4) c12. .qxd 5/20/02 9:31 M Page 413 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 413. 12-1 MULTIPLE LINEAR REGRESSION MODEL. 800 600 1000. 400. 800. 200. 10 8 0. 2. 4 x1. 6. 8. 10. 0. 2. 4. 600 400. 6. 0. E(Y). x2. 200. 10 8 6. 0 0. (a). 2. 4 x1. x2 10. 720 653. 8. 586. 6. 8. 10. 2. 4. x2. (a). x2 10. 25. 8. 100. 519. 6. 452 385. 4. 6 175 4. 318 2. 251 117. 0. 0. 0. 2. 4. 6. 550 700 625 800 750. 2. 325250 400 475. 184 8. 10 x1. 0. 0. 2. 4. 6. 10 x1. 8. (b). (b). Figure 12-2 (a) Three-dimensional plot of the regression model E(Y ) 50 10x1 7x2 5x1x2. (b) The contour plot.. Figure 12-3 (a) Three-dimensional plot of the regression model E(Y ) 800 10x1 7x2 8.5x21 5x22 4x1x2. (b) The contour plot.. and the corresponding two-dimensional contour plot. Notice that, although this model is a linear regression model, the shape of the surface that is generated by the model is not linear. In general, any regression model that is linear in parameters (the ’s) is a linear regression model, regardless of the shape of the surface that it generates. Figure 12-2 provides a nice graphical interpretation of an interaction. Generally, interaction implies that the effect produced by changing one variable (x1, say) depends on the level of the other variable (x2). For example, Fig. 12-2 shows that changing x1 from 2 to 8 produces a much smaller change in E(Y ) when x2 2 than when x2 10. Interaction effects occur frequently in the study and analysis of real-world systems, and regression methods are one of the techniques that we can use to describe them. As a final example, consider the second-order model with interaction Y 0 1 x1 2 x2 11x 21 22 x 22 12 x1x2 . (12-6). If we let x3 x 21, x4 x 22, x5 x1x2, 3 11, 4 22, and 5 12, Equation 12-6 can be written as a multiple linear regression model as follows: Y 0 1x1 2 x2 3 x3 4 x4 5 x5 Figure 12-3(a) and (b) show the three-dimensional plot and the corresponding contour plot for E1Y 2 800 10x1 7x2 8.5x 21 5x 22 4x 1x2 ˛.

(5) c12. .qxd 5/20/02 9:31 M Page 414 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 414. CHAPTER 12 MULTIPLE LINEAR REGRESSION. These plots indicate that the expected change in Y when x1 is changed by one unit (say) is a function of both x1 and x2. The quadratic and interaction terms in this model produce a moundshaped function. Depending on the values of the regression coefficients, the second-order model with interaction is capable of assuming a wide variety of shapes; thus, it is a very flexible regression model.. 12-1.2 Least Squares Estimation of the Parameters The method of least squares may be used to estimate the regression coefficients in the multiple regression model, Equation 12-2. Suppose that n k observations are available, and let xij denote the ith observation or level of variable xj. The observations are 1xi 1, xi 2, p , xik, yi 2, ˛. i 1, 2, p , n. ˛. nk. and. ˛. It is customary to present the data for multiple regression in a table such as Table 12-1. Each observation (xi1, xi2, p , xik , yi ), satisfies the model in Equation 12-2, or y i 0 1xi1 2 xi 2 p k xik i ˛. k. 0 a j xij i. i 1, 2, p , n ˛. (12-7). ˛. j1. The least squares function is n. n. k. 2. L a 2i a ayi 0 a j xij b ˛. i1. ˛. (12-8). ˛. i1. j1. We want to minimize L with respect to 0, 1, p , k. The least squares estimates of 0, 1, p , k must satisfy n k L ` 2 a ayi ˆ 0 a ˆ j xij b 0 0 ˆ 0,ˆ 1, p , ˆ k i1 j1. (12-9a). and n k L ` 2 a ayi ˆ 0 a ˆ j xij b xij 0 j ˆ 0,ˆ 1, p, ˆ k i1 j1. j 1, 2, p , k. (12-9b). Simplifying Equation 12-9, we obtain the least squares normal Equations n. n. nˆ 0 ˆ 1 a xi1 ˛. i1 n. n. ˆ 2 ˆ 0 a xi1 1 a xi1 ˛. ˛. i1. i1. o. o. n. n. ˆ 2 a xi 2 ˛. ˛. ˛. ˛. i1. i1. n. n. n. i1. i1. ˆ 2 a xi1 xi2 p ˆ k a xi1xik a xi1 yi o n. o. o n. 2 ˆ 0 a xik ˆ 1 a xik xi1 ˆ 2 a xik xi2 p ˆ k a xik i1 i1 ˛. a yi. i1. i1. n. n. p ˆ k a xik. ˛. i1. i1. n. a xik yi. (12-10). i1. Note that there are p k 1 normal Equations, one for each of the unknown regression coefficients. The solution to the normal Equations will be the least squares estimators of the.

(6) c12. .qxd 5/20/02 2:58 PM Page 415 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 415. 12-1 MULTIPLE LINEAR REGRESSION MODEL. Table 12-1 Data for Multiple Linear Regression y y1 y2. x1 x11 x21. x2 x12 x22. ... ... .... xk x1k x2k. o yn. o xn1. o xn2. .... o xnk. ˆ , ˆ ,p, ˆ . The normal Equations can be solved by any method regression coefficients, 0 1 k appropriate for solving a system of linear Equations.. EXAMPLE 12-1. In Chapter 1, we used data on pull strength of a wire bond in a semiconductor manufacturing process, wire length, and die height to illustrate building an empirical model. We will use the same data, repeated for convenience in Table 12-2, and show the details of estimating the model parameters. A three-dimensional scatter plot of the data is presented in Fig. 1-13. Figure 12-4 shows a matrix of two-dimensional scatter plots of the data. These displays can be helpful in visualizing the relationships among variables in a multivariable data set. Specifically, we will fit the multiple linear regression model Y 0 1x1 2 x2 where Y pull strength, x1 wire length, and x2 die height. From the data in Table 12-2 we calculate 25. n 25, a yi 725.82 ˛. i1 25. 25. a xi1 206, a xi 2 8,294 ˛. i1. i1. 25. 25. 2 2 a x i1 2,396, a x i2 3,531,848 ˛. i1. i1. 25. 25. 25. a xi1xi2 77,177, a xi1 yi 8,008.37, a xi2 yi 274,811.31 ˛. ˛. i1. i1. i1. Table 12-2 Wire Bond Data for Example 11-1 Observation Number. Pull Strength y. Wire Length x1. Die Height x2. Observation Number. Pull Strength y. 1 2 3 4 5 6 7 8 9 10 11 12 13. 9.95 24.45 31.75 35.00 25.02 16.86 14.38 9.60 24.35 27.50 17.08 37.00 41.95. 2 8 11 10 8 4 2 2 9 8 4 11 12. 50 110 120 550 295 200 375 52 100 300 412 400 500. 14 15 16 17 18 19 20 21 22 23 24 25. 11.66 21.65 17.89 69.00 10.30 34.93 46.59 44.88 54.12 56.63 22.13 21.15. Wire Length Die Height x1 x2 2 4 4 20 1 10 15 15 16 17 6 5. 360 205 400 600 585 540 250 290 510 590 100 400.

(7) c12. .qxd 5/20/02 2:58 PM Page 416 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 416. CHAPTER 12 MULTIPLE LINEAR REGRESSION. 54.15 Strength 24.45. 15.25 Length 5.75. 462.5 Height 187.5. 24.45. 5.75. 54.15. 15.25. 187.5. 462.5. Figure 12-4 Matrix of scatter plots (from Minitab) for the wire bond pull strength data in Table 12-2.. For the model Y 0 1 x1 2 x 2 , the normal Equations 12-10 are n. n. nˆ 0 ˆ 1 a xi1 ˛. i1 n. n ˛. ˛. i1 n. ˆ 0 a xi1 ˆ 1 a xi12 i1 n. n. ˆ 2 a xi 2. ˛. ˛. i1 n. i1 n. ˆ 2 a xi1xi2 a xi1 yi ˛. ˛. i1 n. i1 n. ˆ 0 a xi 2 ˆ 1 a xi1xi 2 ˆ 2 a x2i 2 ˛. i1. ˛. i1. a yi. ˛. i1. a xi 2 yi ˛. i1. Inserting the computed summations into the normal equations, we obtain 25ˆ 0 . 206ˆ 1 . 8294ˆ 2 725.82. 206ˆ 0 . 2396ˆ 1 . 77,177ˆ 2 8,008.37. 8294ˆ 0 77,177ˆ 1 3,531,848ˆ 2 274,811.31 The solution to this set of equations is ˆ 0 2.26379,. ˆ 1 2.74427,. ˆ 2 0.01253. Therefore, the fitted regression equation is yˆ 2.26379 2.74427x1 0.01253x2 This equation can be used to predict pull strength for pairs of values of the regressor variables wire length (x1) and die height (x2). This is essentially the same regression model given in Equation 1-7, Section 1-3. Figure 1-14 shows a three-dimentional plot of the plane of predicted values yˆ generated from this equation..

(8) c12. .qxd 5/20/02 9:31 M Page 417 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 417. 12-1 MULTIPLE LINEAR REGRESSION MODEL. 12-1.3 Matrix Approach to Multiple Linear Regression In fitting a multiple regression model, it is much more convenient to express the mathematical operations using matrix notation. Suppose that there are k regressor variables and n observations, (xi1, xi2, p , xik , yi), i 1, 2, p , n and that the model relating the regressors to the response is yi 0 1xi1 2 xi 2 p k xik i. i 1, 2, p , n. This model is a system of n equations that can be expressed in matrix notation as y X . (12-11). where. y ≥. y1 y2 o yn. ¥. X ≥. 1 1. x11 x21. x12 x22. o 1. o. o. xn1. xn2. p p. x1k x2k o. p. ¥. xnk. ≥. 0 1 o k. ¥. and. ≥. 1 2 o n. ¥. In general, y is an (n 1) vector of the observations, X is an (n p) matrix of the levels of the independent variables, is a ( p 1) vector of the regression coefficients, and is a (n 1) vector of random errors. ˆ , that minimizes We wish to find the vector of least squares estimators, n. L a 2i ¿ 1y X2¿1y X2 ˛. i1. ˆ is the solution for in the equations The least squares estimator L 0 We will not give the details of taking the derivatives above; however, the resulting equations that must be solved are. ˆ Xy XX. (12-12). Equations 12-12 are the least squares normal equations in matrix form. They are identical to the scalar form of the normal equations given earlier in Equations 12-10. To solve the normal equations, multiply both sides of Equations 12-12 by the inverse of X¿X. Therefore, the least squares estimate of is. ˆ (XX)1 Xy . (12-13).

(9) c12. .qxd 5/20/02 2:58 PM Page 418 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 418. CHAPTER 12 MULTIPLE LINEAR REGRESSION. Note that there are p k 1 normal equations in p k 1 unknowns (the values of ˆ 0, ˆ 1, p , ˆ k 2. Furthermore, the matrix XX is always nonsingular, as was assumed above, so the methods described in textbooks on determinants and matrices for inverting these matrices can be used to find 1X¿X2 1. In practice, multiple regression calculations are almost always performed using a computer. It is easy to see that the matrix form of the normal equations is identical to the scalar form. Writing out Equation 12-12 in detail, we obtain. n. n n. a xi1 H i1 o n. n. n. a xi1. a xi2. i1 n. i1 n 2. i1 n. a xi1. a xi1xi2. i1. i1. o. ˆ 0. a yi i1 n. ˆ a xi1xik 1 a xi1 yi X H X H i1 X o o o. p. i1. o. n. n. a xik. p. n. n. a xik. a xik xi1. a xik xi2. i1. i1. i1. n 2. a xik. p. ˆ k. i1. a xik yi ˛. i1. If the indicated matrix multiplication is performed, the scalar form of the normal equations (that is, Equation 12-10) will result. In this form it is easy to see that XX is a ( p p) symmetric matrix and Xy is a ( p 1) column vector. Note the special structure of the XX matrix. The diagonal elements of XX are the sums of squares of the elements in the columns of X, and the off-diagonal elements are the sums of cross-products of the elements in the columns of X. Furthermore, note that the elements of Xy are the sums of cross-products of the columns of X and the observations 5yi 6. The fitted regression model is k. yˆ i ˆ 0 a ˆ j xi j ˛. ˛. i 1, 2, p , n ˛. ˛. (12-14). j1. In matrix notation, the fitted model is yˆ Xˆ The difference between the observation yi and the fitted value yˆ i is a residual, say, ei yi yˆ i. The (n 1) vector of residuals is denoted by e y yˆ. EXAMPLE 12-2. (12-15). In Example 12-1, we illustrated fitting the multiple regression model y 0 1 x1 2 x2 ˛. where y is the observed pull strength for a wire bond, x1 is the wire length, and x2 is the die height. The 25 observations are in Table 12-2. We will now use the matrix approach.

(10) c12. .qxd 5/20/02 9:31 M Page 419 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 12-1 MULTIPLE LINEAR REGRESSION MODEL. 419. to fit the regression model above to these data. The X matrix and y vector for this model are 1 1 1 1 1 1 1 1 1 1 1 1 X 1 1 1 1 1 1 1 1 1 1 1 1 1. 2 8 11 10 8 4 2 2 9 8 4 11 12 2 4 4 20 1 10 15 15 16 17 6 5. 50 110 120 550 295 200 375 52 100 300 412 400 500 360 205 400 600 585 540 250 290 510 590 100 400. 9.95 24.45 31.75 35.00 25.02 16.86 14.38 9.60 24.35 27.50 17.08 37.00 y 41.95 11.66 21.65 17.89 69.00 10.30 34.93 46.59 44.88 54.12 56.63 22.13 21.15. The X¿X matrix is 1 X¿X £ 2 50. 1 8 110. p p p. 1 1 1 5 §≥ o 400 1 ˛. 2 8 o 5. 50 25 110 ¥ £ 206 o 8,294 400. 206 2,396 77,177. 8,294 77,177 § 3,531,848. and the X¿y vector is 1 X¿y £ 2 50. 1 8 110. p p p. 9.95 1 725.82 24.45 5 §≥ ¥ £ 8,008.37 § o 400 274,811.31 21.15. The least squares estimates are found from Equation 12-13 as ˆ 1X¿X2 1X¿y.

(11) c12. .qxd 5/20/02 2:58 PM Page 420 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 420. CHAPTER 12 MULTIPLE LINEAR REGRESSION. or 25 ˆ 0 ˆ £ 1 § £ 206 8,294 ˆ 2. 206 2,396 77,177. 0.214653 £ 0.007491 0.000340. 1 8,294 725.82 77,177 § £ 8,008.37 § 3,531,848 274,811.31. 0.007491 0.001671 0.000019. 0.000340 725.82 2.26379143 0.000019 § £ 8,008.47 § £ 2.74426964 § 0.0000015 274,811.31 0.01252781. Therefore, the fitted regression model with the regression coefficients rounded to five decimal places is yˆ 2.26379 2.74427x1 0.01253x2 This is identical to the results obtained in Example 12-1. This regression model can be used to predict values of pull strength for various values of wire length (x1) and die height (x2). We can also obtain the fitted values yˆ i by substituting each observation (xi1, xi2), i 1, 2, . . . , n, into the equation. For example, the first observation has x11 2 and x12 50, and the fitted value is yˆ1 2.26379 2.74427x11 0.01253x12 2.26379 2.74427122 0.012531502 8.38 The corresponding observed value is y1 9.95. The residual corresponding to the first observation is e1 y1 yˆ1 9.95 8.38 1.57 Table 12-3 displays all 25 fitted values yî and the corresponding residuals. The fitted values and residuals are calculated to the same accuracy as the original data. Table 12-3 Observations, Fitted Values, and Residuals for Example 12-2 Observation Number. yi. yî. ei yi yî. 1 2 3 4 5 6 7 8 9 10 11 12 13. 9.95 24.45 31.75 35.00 25.02 16.86 14.38 9.60 24.35 27.50 17.08 37.00 41.95. 8.38 25.60 33.95 36.60 27.91 15.75 12.45 8.40 28.21 27.98 18.40 37.46 41.46. 1.57 1.15 2.20 1.60 2.89 1.11 1.93 1.20 3.86 0.48 1.32 0.46 0.49. Observation Number. yi. yî. ei yi yî. 14 15 16 17 18 19 20 21 22 23 24 25. 11.66 21.65 17.89 69.00 10.30 34.93 46.59 44.88 54.12 56.63 22.13 21.15. 12.26 15.81 18.25 64.67 12.34 36.47 46.56 47.06 52.56 56.31 19.98 21.00. 0.60 5.84 0.36 4.33 2.04 1.54 0.03 2.18 1.56 0.32 2.15 0.15.

(12) c12. .qxd 8/6/02 2:32 PM Page 421. 12-1 MULTIPLE LINEAR REGRESSION MODEL. 421. Computers are almost always used in fitting multiple regression models. Table 12-4 presents some annotated output from Minitab for the least squares regression model for wire bond pull strength data. The upper part of the table contains the numerical estimates of the regression coefficients. The computer also calculates several other quantities that reflect important information about the regression model. In subsequent sections, we will define and explain the quantities in this output. Estimating 2 Just as in simple linear regression, it is important to estimate 2, the variance of the error term , in a multiple regression model. Recall that in simple linear regression the estimate of 2 was obtained by dividing the sum of the squared residuals by n 2. Now there are two parameters in the simple linear regression model, so in multiple linear regression with p parameters a logical estimator for 2 is n 2. a ei ˛. SSE ˆ 2 n p n p i1. (12-16). This is an unbiased estimator of 2. Just as in simple linear regression, the estimate of 2 is usually obtained from the analysis of variance for the regression model. The numerator of Equation 12-16 is called the error or residual sum of squares, and the denominator n p is called the error or residual degrees of freedom. Table 12-4 shows that the estimate of 2 for the wire bond pull strength regression model is ˆ 2 115.222 5.2364. The Minitab output rounds the estimate to ˆ 2 5.2.. 12-1.4. Properties of the Least Squares Estimators The statistical properties of the least squares estimators ˆ 0, ˆ 1, p , ˆ k may be easily found, under certain assumptions on the error terms 1, 2, p , n, in the regression model. Paralleling the assumptions made in Chapter 11, we assume that the errors i are statistically independent with mean zero and variance 2. Under these assumptions, the least squares estimators ˆ 0, ˆ 1, p , ˆ k are unbiased estimators of the regression coefficients 0, 1, p , k. This property may be shown as follows: ˛. E1ˆ 2 E3 1X¿X2 1X¿Y4 E3 1X¿X2 1X¿1X 2 4 E3 1X¿X2 1X¿X 1X¿X2 1X¿4 since E() 0 and (XX)1XX I, the identity matrix. Thus, ˆ is an unbiased estimator of . The variances of the ˆ ’s are expressed in terms of the elements of the inverse of the X¿X matrix. The inverse of X¿X times the constant 2 represents the covariance matrix of the regression coefficients ˆ . The diagonal elements of 2 1X¿X2 1 are the variances of ˆ 0, ˆ 1, p , ˆ k, and the off-diagonal elements of this matrix are the covariances. For example, if we have k 2 regressors, such as in the pull-strength problem, C00 C 1X¿X2 1 £ C10 C20. C01 C11 C21. C02 C12 § C22.

(13) c12. .qxd 5/20/02 3:23 PM Page 422 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 422. CHAPTER 12 MULTIPLE LINEAR REGRESSION. Table 12-4 Minitab Multiple Regression Output for the Wire Bond Pull Strength Data Regression Analysis: Strength versus Length, Height The regression equation is Strength 2.26 2.74 Length 0.0125 Height Predictor Constant Length Height. ˆ0 ˆ1 ˆ2 . Coef 2.264 2.74427 0.012528. S 2.288 PRESS 156.163. SE Coef 1.060 0.09352 0.002798. T 2.14 29.34 4.48. R-Sq 98.1% R-Sq (pred) 97.44%. P 0.044 0.000 0.000. VIF 1.2 1.2. R-Sq (adj) 97.9%. Analysis of Variance Source Regression Residual Error Total. DF 2 22 24. Source Length Height. Seq SS 5885.9 104.9. DF 1 1. SS 5990.8 115.2 6105.9. MS 2995.4 5.2. F 572.17. P 0.000. ˆ 2. Predicted Values for New Observations New Obs 1. Fit 27.663. SE Fit 0.482. 95.0% CI (26.663, 28.663). 95.0% PI (22.814, 32.512). Values of Predictors for New Observations New Obs 1. Length 8.00. Height 275. which is symmetric (C10 C01, C20 C02, and C21 C12) because (XX)1 is symmetric, and we have V 1ˆ j 2 2C jj, ˛. ˛. cov1ˆ i, ˆ j 2 2C ij, ˛. j 0, 1, 2 ij. ˆ is a ( p p) symmetric matrix whose jjth element is the In general, the covariance matrix of ˆ variance of j and whose i, jth element is the covariance between ˆ i and ˆ j, that is,. cov1ˆ 2 2 1X¿X2 1 2 C The estimates of the variances of these regression coefficients are obtained by replacing 2 with an estimate. When 2 is replaced by it’s estimate ˆ 2 , the square root of the estimated variance of the jth regression coefficient is called the estimated standard error of ˆ j or se 1ˆ j 2 2ˆ 2Cjj . These standard errors are a useful measure of the precision of estimation for the regression coefficients; small standard errors imply good precision. Multiple regression computer programs usually display these standard errors. For example, the Minitab output in Table 12-4 reports se 1ˆ 0 2 1.060, se 1ˆ 1 2 0.09352, and.

(14) c12. .qxd 5/20/02 2:58 PM Page 423 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 12-1 MULTIPLE LINEAR REGRESSION MODEL. 423. se1ˆ 1 2 0.002798. The slope estimate is about twice the magnitude of its standard error, and ˆ 1 and ˆ 2 are considerably larger than se 1ˆ 1 2 and se 1ˆ 2 2. This implies reasonable precision of estimation, although the parameters 1 and 2 are much more precisely estimated than the intercept (this is not unusual in multiple regression). EXERCISES FOR SECTION 12-1 12-1. A study was performed to investigate the shear strength of soil ( y) as it related to depth in feet (x1) and moisture content (x2). Ten observations were collected, and the following summary quantities obtained: n 10, gxi1 223, gxi2 553, gyi 1,916, gx2i1 5,200.9, gx2i2 31,729, gxi1xi2 12,352, gxi1 yi 43,550.8, gxi2 yi 104,736.8, and gy2i 371,595.6. (a) Set up the least squares normal equations for the model Y 0 1x1 2x2 . (b) Estimate the parameters in the model in part (a). (c) What is the predicted strength when x1 18 feet and x2 43%? 12-2. A regression model is to be developed for predicting the ability of soil to absorb chemical contaminants. Ten observations have been taken on a soil absorption index ( y) and two regressors: x1 amount of extractable iron ore and x2 amount of bauxite. We wish to fit the model Y 0 1x1 2x2 . Some necessary quantities are: 1.17991 7.30982 E-3 1¿2 1 £ 7.30982 E-3 7.9799 E-5 7.3006 E-4 1.23713 E-4 (a) Estimate the regression coefficients in the model specified above. (b) What is the predicted value of the absorption index y when x1 200 and x2 50? 12-3. A chemical engineer is investigating how the amount of conversion of a product from a raw material (y) depends on reaction temperature (x1) and the reaction time (x2). He has developed the following regression models: 1. yˆ 100 2 x1 4 x2 2. yˆ 95 1.5x1 3x2 2 x1x2 Both models have been built over the range 0.5 x2 10. (a) What is the predicted value of conversion when x2 2? Repeat this calculation for x2 8. Draw a graph of the predicted values for both conversion models. Comment on the effect of the interaction term in model 2. (b) Find the expected change in the mean conversion for a unit change in temperature x1 for model 1 when x2 5. Does this quantity depend on the specific value of reaction time selected? Why? (c) Find the expected change in the mean conversion for a unit change in temperature x1 for model 2 when x2 5. Repeat this calculation for x2 2 and x2 8. Does the result depend on the value selected for x2? Why?. 12-4. The data in Table 12-5 are the 1976 team performance statistics for the teams in the National Football League (Source: The Sporting News). (a) Fit a multiple regression model relating the number of games won to the teams’ passing yardage (x2), the percent rushing plays (x7), and the opponents’ yards rushing (x8). (b) Estimate 2. 2 (c) What are the standard errors of the regression coefficients? (d) Use the model to predict the number of games won when x2 2000 yards, x7 60%, and x8 1800. 12-5. Table 12-6 presents gasoline mileage performance for 25 automobiles (Source: Motor Trend, 1975). (a) Fit a multiple regression model relating gasoline mileage to engine displacement (x1) and number of carburetor barrels (x6). (b) Estimate 2. (c) Use the model developed in part (a) to predict mileage performance for a car with displacement x1 300 and x6 2. 7.3006 E-4 220 1.23713 E-4 § , ¿y £ 36,768 § 4.6576 E-4 9,965 12-6. The electric power consumed each month by a chemical plant is thought to be related to the average ambient temperature (x1), the number of days in the month (x2), the average product purity (x3), and the tons of product produced (x4). The past year’s historical data are available and are presented in the following table: y. x1. x2. x3. x4. 240 236 270 274 301 316 300 296 267 276 288 261. 25 31 45 60 65 72 80 84 75 60 50 38. 24 21 24 25 25 26 25 25 24 25 25 23. 91 90 88 87 91 94 87 86 88 91 90 89. 100 95 110 88 94 99 97 96 110 105 100 98. (a) Fit a multiple linear regression model to these data. (b) Estimate 2..

(15) c12. .qxd 5/20/02 9:32 M Page 424 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 424. CHAPTER 12 MULTIPLE LINEAR REGRESSION. Table 12-5 National Football League 1976 Team Performance Team Washington Minnesota New England Oakland Pittsburgh Baltimore Los Angeles Dallas Atlanta Buffalo Chicago Cincinnati Cleveland Denver Detroit Green Bay Houston Kansas City Miami New Orleans New York Giants New York Jets Philadelphia St. Louis San Diego San Francisco Seattle Tampa Bay. y. x1. x2. x3. x4. x5. x6. x7. x8. x9. 10 11 11 13 10 11 10 11 4 2 7 10 9 9 6 5 5 5 6 4 3 3 4 10 6 8 2 0. 2113 2003 2957 2285 2971 2309 2528 2147 1689 2566 2363 2109 2295 1932 2213 1722 1498 1873 2118 1775 1904 1929 2080 2301 2040 2447 1416 1503. 1985 2855 1737 2905 1666 2927 2341 2737 1414 1838 1480 2191 2229 2204 2140 1730 2072 2929 2268 1983 1792 1606 1492 2835 2416 1638 2649 1503. 38.9 38.8 40.1 41.6 39.2 39.7 38.1 37.0 42.1 42.3 37.3 39.5 37.4 35.1 38.8 36.6 35.3 41.1 38.6 39.3 39.7 39.7 35.5 35.3 38.7 39.9 37.4 39.3. 64.7 61.3 60.0 45.3 53.8 74.1 65.4 78.3 47.6 54.2 48.0 51.9 53.6 71.4 58.3 52.6 59.3 55.3 69.6 78.3 38.1 68.8 68.8 74.1 50.0 57.1 56.3 47.0. 4 3 14 4 15 8 12 1 3 1 19 6 5 3 6 19 5 10 6 7 9 21 8 2 0 8 22 9. 868 615 914 957 836 786 754 797 714 797 984 819 1037 986 819 791 776 789 582 901 734 627 722 683 576 848 684 875. 59.7 55.0 65.6 61.4 66.1 61.0 66.1 58.9 57.0 58.9 68.5 59.2 58.8 58.6 59.2 54.4 49.6 54.3 58.7 51.7 61.9 52.7 57.8 59.7 54.9 65.3 43.8 53.5. 2205 2096 1847 1903 1457 1848 1564 2476 2577 2476 1984 1901 1761 1709 1901 2288 2072 2861 2411 2289 2203 2592 2053 1979 2048 1786 2876 2560. 1917 1575 2175 2476 1866 2339 2092 2254 2001 2254 2217 1686 2032 2025 1686 1835 1914 2496 2670 2202 1988 2324 2550 2110 2628 1776 2524 2241. y: Games won (per 14 game season) x1: Rushing yards (season) x2: Passing yards (season) x3: Punting yards (yds/punt) x4: Field goal percentage (Field goals made/Field goals attempted—season) x5: Turnover differential (turnovers acquired—turnovers lost) x6: Penalty yards (season) x7: Percent rushing (rushing plays/total plays).

(16) c12. .qxd 5/20/02 9:32 M Page 425 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 425. 12-1 MULTIPLE LINEAR REGRESSION MODEL. Table 12-6 Gasoline Mileage Performance for 25 Automobiles Automobile. y. x1. x2. x3. x4. x5. x6. x7. x8. x9. x10. x11. Apollo Nova Monarch Duster Jenson Conv. Skyhawk Scirocco Corolla SR-5 Camaro Datsun B210 Capri II. 18.90 20.00 18.25 20.07 11.2 22.12 34.70 30.40 16.50 36.50 21.50. 350 250 351 225 440 231 89.7 96.9 350 85.3 171. 165 105 143 95 215 110 70 75 155 80 109. 260 185 255 170 330 175 81 83 250 83 146. 8.0:1 8.25:1 8.0:1 8.4:1 8.2:1 8.0:1 8.2:1 9.0:1 8.5:1 8.5:1 8.2:1. 2.56:1 2.73:1 3.00:1 2.76:1 2.88:1 2.56:1 3.90:1 4.30:1 3.08:1 3.89:1 3.22:1. 4 1 2 1 4 2 2 2 4 2 2. 3 3 3 3 3 3 4 5 3 4 4. 200.3 196.7 199.9 194.1 184.5 179.3 155.7 165.2 195.4 160.6 170.4. 69.9 72.2 74.0 71.8 69 65.4 64 65 74.4 62.2 66.9. 3910 3510 3890 3365 4215 3020 1905 2320 3885 2009 2655. A A A M A A M M A M M. Pacer Granada Eldorado Imperial Nova LN Starfire Cordoba Trans Am Corolla E-5 Mark IV Celica GT Charger SE. 19.70 17.80 14.39 14.89 17.80 23.54 21.47 16.59 31.90 13.27 23.90 19.73. 258 302 500 440 350 231 360 400 96.9 460 133.6 318. 110 129 190 215 155 110 180 185 75 223 96 140. 195 220 360 330 250 175 290 NA 83 366 120 255. 8.0:1 8.0:1 8.5:1 8.2:1 8.5:1 8.0:1 8.4:1 7.6:1 9.0:1 8.0:1 8.4:1 8.5:1. 3.08:1 3.0:1 2.73:1 2.71:1 3.08:1 2.56:1 2.45:1 3.08:1 4.30:1 3.00:1 3.91:1 2.71:1. 1 2 4 4 4 2 2 4 2 4 2 2. 3 3 3 3 3 3 3 3 5 3 5 3. 171.5 199.9 224.1 231.0 196.7 179.3 214.2 196 165.2 228 171.5 215.3. 77 74 79.8 79.7 72.2 65.4 76.3 73 61.8 79.8 63.4 76.3. 3375 3890 5290 5185 3910 3050 4250 3850 2275 5430 2535 4370. A A A A A A A A M A M A. Cougar Corvette. 13.90 16.50. 351 350. 148 165. 243 255. 8.0:1 8.5:1. 3.25:1 2.73:1. 2 4. 3 3. 215.5 185.2. 78.5 69. 4540 3660. A A. y: Miles/gallon x1: Displacement (cubic inches) x2: Horsepower (foot-pounds) x3: Torque (foot-pounds) x4: Compression ratio x5: Rear axle ratio x6: Carburetor (barrels) x7: No. of transmission speeds x8: Overall length (inches) x9: Width (inches) x10: Weight (pounds) x11: Type of transmission (A—automatic, M—manual).

(17) c12. .qxd 6/4/02 2:30 PM Page 426 RK UL 6 RK UL 6:Desktop Folder:montgo:. 426. CHAPTER 12 MULTIPLE LINEAR REGRESSION. (c) Compute the standard errors of the regression coefficients. (d) Predict power consumption for a month in which x1 75F, x2 24 days, x3 90%, and x4 98 tons. 12-7. A study was performed on wear of a bearing y and its relationship to x1 oil viscosity and x2 load. The following data were obtained. y. x1. x2. 293 230 172 91 113 125. 1.6 15.5 22.0 43.0 33.0 40.0. 851 816 1058 1201 1357 1115. (a) Fit a multiple linear regression model to these data. (b) Estimate 2 and the standard errors of the regression coefficients. (c) Use the model to predict wear when x1 25 and x2 1000. (d) Fit a multiple linear regression model with an interaction term to these data. (e) Estimate 2 and se(ˆ j) for this new model. How did these quantities change. Does this tell you anything about the value of adding the interaction term to the model? (f) Use the model in (d) to predict when x1 25 and x2 1000. Compare this prediction with the predicted value from part (b) above. 12-8. The pull strength of a wire bond is an important characteristic. The following table gives information on pull strength ( y), die height (x1), post height (x2), loop height (x3), wire length (x4), bond width on the die (x5), and bond width on the post (x6). y. x1. x2. x3. x4. x5. x6. 8.0 8.3 8.5 8.8 9.0 9.3 9.3 9.5 9.8 10.0 10.3 10.5 10.8 11.0 11.3 11.5 11.8 12.3 12.5. 5.2 5.2 5.8 6.4 5.8 5.2 5.6 6.0 5.2 5.8 6.4 6.0 6.2 6.2 6.2 5.6 6.0 5.8 5.6. 19.6 19.8 19.6 19.4 18.6 18.8 20.4 19.0 20.8 19.9 18.0 20.6 20.2 20.2 19.2 17.0 19.8 18.8 18.6. 29.6 32.4 31.0 32.4 28.6 30.6 32.4 32.6 32.2 31.8 32.6 33.4 31.8 32.4 31.4 33.2 35.4 34.0 34.2. 94.9 89.7 96.2 95.6 86.5 84.5 88.8 85.7 93.6 86.0 87.1 93.1 83.4 94.5 83.4 85.2 84.1 86.9 83.0. 2.1 2.1 2.0 2.2 2.0 2.1 2.2 2.1 2.3 2.1 2.0 2.1 2.2 2.1 1.9 2.1 2.0 2.1 1.9. 2.3 1.8 2.0 2.1 1.8 2.1 1.9 1.9 2.1 1.8 1.6 2.1 2.1 1.9 1.8 2.1 1.8 1.8 2.0. (a) Fit a multiple linear regression model using x2, x3, x4, and x5 as the regressors. (b) Estimate 2. (c) Find the se(ˆ j). How precisely are the regression coefficients estimated, in your opinion? (d) Use the model from part (a) to predict pull strength when x2 = 20, x3 = 30, x4 = 90, and x5 = 2.0. 12-9. An engineer at a semiconductor company wants to model the relationship between the device HFE ( y) and three parameters: Emitter-RS (x1), Base-RS (x2), and Emitter-to-Base RS (x3). The data are shown in the following table. x1 Emitter-RS. x2 Base-RS. x3 E-B-RS. y HFE-1M-5V. 14.620 15.630 14.620 15.000 14.500 15.250 16.120 15.130 15.500 15.130 15.500 16.120 15.130 15.630 15.380 14.380 15.500 14.250 14.500 14.620. 226.00 220.00 217.40 220.00 226.50 224.10 220.50 223.50 217.60 228.50 230.20 226.50 226.60 225.60 229.70 234.00 230.00 224.30 240.50 223.70. 7.000 3.375 6.375 6.000 7.625 6.000 3.375 6.125 5.000 6.625 5.750 3.750 6.125 5.375 5.875 8.875 4.000 8.000 10.870 7.375. 128.40 52.62 113.90 98.01 139.90 102.60 48.14 109.60 82.68 112.60 97.52 59.06 111.80 89.09 101.00 171.90 66.80 157.10 208.40 133.40. (a) Fit a multiple linear regression model to the data. (b) Estimate 2. (c) Find the standard errors se 1ˆ j 2. (d) Predict HFE when x1 14.5, x2 220, and x3 5.0. 12-10. Heat treating is often used to carburize metal parts, such as gears. The thickness of the carburized layer is considered a crucial feature of the gear and contributes to the overall reliability of the part. Because of the critical nature of this feature, two different lab tests are performed on each furnace load. One test is run on a sample pin that accompanies each load. The other test is a destructive test, where an actual part is cross-sectioned. This test involves running a carbon analysis on the surface of both the gear pitch (top of the gear tooth) and the gear root (between the gear teeth). Table 12-7 shows the results of the pitch carbon analysis test for 32 parts. The regressors are furnace temperature (TEMP), carbon concentration and duration of the carburizing cycle.

(18) c12. .qxd 5/20/02 9:32 M Page 427 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 12-1 MULTIPLE LINEAR REGRESSION MODEL. 427. Table 12-7 TEMP. SOAKTIME. SOAKPCT. DIFFTIME. DIFFPCT. PITCH. 1650 1650 1650 1650 1600 1600 1650 1650 1650 1650 1650 1650 1650 1650 1650 1650 1650 1650 1650 1650 1650 1650 1650 1650 1650 1650 1650 1700 1650 1650 1700 1700. 0.58 0.66 0.66 0.66 0.66 0.66 1.00 1.17 1.17 1.17 1.17 1.17 1.17 1.20 2.00 2.00 2.20 2.20 2.20 2.20 2.20 2.20 3.00 3.00 3.00 3.00 3.33 4.00 4.00 4.00 12.50 18.50. 1.10 1.10 1.10 1.10 1.15 1.15 1.10 1.10 1.10 1.10 1.10 1.10 1.15 1.15 1.15 1.10 1.10 1.10 1.15 1.10 1.10 1.10 1.15 1.10 1.10 1.15 1.10 1.10 1.10 1.15 1.00 1.00. 0.25 0.33 0.33 0.33 0.33 0.33 0.50 0.58 0.58 0.58 0.58 0.58 0.58 1.10 1.00 1.10 1.10 1.10 1.10 1.10 1.10 1.50 1.50 1.50 1.50 1.66 1.50 1.50 1.50 1.50 1.50 1.50. 0.90 0.90 0.90 0.95 1.00 1.00 0.80 0.80 0.80 0.80 0.90 0.90 0.90 0.80 0.80 0.80 0.80 0.80 0.80 0.90 0.90 0.90 0.80 0.70 0.75 0.85 0.80 0.70 0.70 0.85 0.70 0.70. 0.013 0.016 0.015 0.016 0.015 0.016 0.014 0.021 0.018 0.019 0.021 0.019 0.021 0.025 0.025 0.026 0.024 0.025 0.024 0.025 0.027 0.026 0.029 0.030 0.028 0.032 0.033 0.039 0.040 0.035 0.056 0.068. (SOAKPCT, SOAKTIME), and carbon concentration and duration of the diffuse cycle (DIFFPCT, DIFFTIME). (a) Fit a linear regression model relating the results of the pitch carbon analysis test (PITCH) to the five regressor variables. (b) Estimate 2. (c) Find the standard errors se 1ˆ j 2. (d) Use the model in part (a) to predict PITCH when TEMP 1650, SOAKTIME 1.00, SOAKPCT 1.10, DIFFTIME 1.00, and DIFFPCT 0.80. 12-11. Statistics for 21 National Hockey League teams were obtained from the Hockey Encyclopedia and are shown in Table 12-8.. The variables and definitions are as follows: Wins Number of games won in a season. Pts Points awarded in a season. Two points for winning a game, one point for losing in overtime, zero points for losing in regular time. GF Goals for. Total goals scored during the season. GA Goals against. Goals scored against the team during the season. PPG Power play goals. Points scored while on power play. PPcT Power play percentage. The number of power play goals divided by the number of power play opportunities..

(19) c12. .qxd 5/20/02 9:32 M Page 428 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 428. CHAPTER 12 MULTIPLE LINEAR REGRESSION. Table 12-8 Team. Wins. Pts. GF. GA. PPG. PPcT. SHG. PPGA. PKPcT. SHGA. Chicago Minnesota Toronto St. Louis Detroit Edmonton Calgary Vancouver Winnipeg Los Angeles Philadelphia NY Islanders Washington NY Rangers New Jersey Pittsburgh Boston Montreal Buffalo Quebec Hartford. 47 40 28 25 21 47 32 30 33 27 49 42 39 35 17 18 50 42 38 34 19. 104 96 68 65 57 106 78 75 74 66 106 96 94 80 48 45 110 98 89 80 45. 338 321 23 285 263 424 321 303 311 308 326 302 306 306 230 257 327 350 318 343 261. 268 290 330 316 344 315 317 309 333 365 240 226 283 287 338 394 228 286 285 336 403. 86 91 79 67 37 86 90 65 78 81 60 69 75 71 66 81 67 64 67 61 51. 27.2 26.4 22.3 21.2 19.3 29.3 27 23.8 23.6 23.8 21.6 25.8 20.9 22.4 21.9 22.6 22.2 22.2 21.5 20.7 19.3. 4 17 13 9 7 22 7 5 10 10 15 10 3 12 74 3 8 8 12 6 6. 71 67 83 63 80 89 59 56 67 94 61 55 53 75 78 110 53 68 48 92 70. 76.6 80.7 75 81.3 72.6 77.5 77.1 80.8 72.8 68.2 82 83.4 81.6 76 73.5 72.2 80.7 73.8 82.5 73.6 76.1. 6 20 9 12 9 6 6 13 7 14 7 3 11 8 10 15 6 8 9 6 9. SHG PPGA PKPcT. SHGA. 12-2. Short-handed goals scored during the season. Power play goals against. Penalty killing percentage. Measures a team’s ability to prevent goals while its opponent is on a power play. Opponent power play goals divided by opponent’s opportunities. Short-handed goals against. Fit a multiple linear regression model relating wins to the other variables. Estimate 2 and find the standard errors of the regression coefficients.. 12-12.. Consider the linear regression model Yi ¿0 1 1xi1 x1 2 2 1xi2 x2 2 i. where x1 g xi1n and x2 g xi2n. (a) Write out the least squares normal equations for this model. (b) Verify that the least squares estimate of the intercept in this model is ˆ ¿0 g yin y. (c) Suppose that we use yi y as the response variable in the model above. What effect will this have on the least squares estimate of the intercept?. HYPOTHESIS TESTS IN MULTIPLE LINEAR REGRESSION In multiple linear regression problems, certain tests of hypotheses about the model parameters are useful in measuring model adequacy. In this section, we describe several important hypothesis-testing procedures. As in the simple linear regression case, hypothesis testing requires that the error terms i in the regression model are normally and independently distributed with mean zero and variance 2.. 12-2.1. Test for Significance of Regression The test for significance of regression is a test to determine whether a linear relationship exists between the response variable y and a subset of the regressor variables x1, x2, p , xk. The.

(20) c12. .qxd 5/20/02 2:58 PM Page 429 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 429. 12-2 HYPOTHESIS TESTS IN MULTIPLE LINEAR REGRESSION. appropriate hypotheses are. H0: 1 2 # # # k 0 H1: j Z 0 for at least one j. (12-17). Rejection of H0: 1 2 p k 0 implies that at least one of the regressor variables x1, x2, p , xk contributes significantly to the model. The test for significance of regression is a generalization of the procedure used in simple linear regression. The total sum of squares SST is partitioned into a sum of squares due to regression and a sum of squares due to error, say, SST SSR SSE Now if H0: 1 2 p k 0 is true, SSR 2 is a chi-square random variable with k degrees of freedom. Note that the number of degrees of freedom for this chi-square random variable is equal to the number of regressor variables in the model. We can also show the SSE 2 is a chi-square random variable with n p degrees of freedom, and that SSE and SSR are independent. The test statistic for H0: 1 2 p k 0 is. F0 . SSRk MSR MSE SSE 1n p2. (12-18). We should reject H0 if the computed value of the test statistic in Equation 12-18, f0, is greater than f ,k,np. The procedure is usually summarized in an analysis of variance table such as Table 12-9. We can find a computing formula for SSE as follows: n. n. i1. i1. SSE a 1 yi yˆ i 2 2 a e2i e¿e Substituting e y yˆ y Xˆ into the above, we obtain SSE y¿y ˆ ¿X¿y. (12-19). Table 12-9 Analysis of Variance for Testing Significance of Regression in Multiple Regression Source of Variation. Sum of Squares. Degrees of Freedom. Regression Error or residual Total. SSR SSE SST. k np n1. Mean Square. F0. MSR MSE. MSRMSE.

(21) c12. .qxd 5/20/02 2:58 PM Page 430 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 430. CHAPTER 12 MULTIPLE LINEAR REGRESSION. A computational formula for SSR may be found easily. Now since SST g ni1 y2i n n 1g i1 yi 2 2n y¿y 1 g i1 yi 2 2 n, we may rewrite Equation 12-19 as n. SSE y¿y . 2. n. a a yi b. ≥ ˆ ¿¿y . i1. n. 2. a a yi b i1. n. ¥. or SSE SST SSR Therefore, the regression sum of squares is n. SSR ˆ ¿¿y . EXAMPLE 12-3. 2. a a yi b i1. n. (12-20). We will test for significance of regression (with 0.05) using the wire bond pull strength data from Example 12-1. The total sum of squares is n. SST y¿y . 2. a a yi b i1. n. 27,177.9510 . 1725.822 2 6105.9447 25. The regression sum of squares is computed from Equation 12-20 as follows: n. SSR ˆ ¿X¿y . 2. a a yi b i1. n. 27,062.7775 . 1725.822 2 5990.7712 25. and by subtraction SSE SST SSR y¿y ˆ ¿X¿y 115.1735 The analysis of variance is shown in Table 12-10. To test H0: 1 2 0, we calculate the statistic f0 . MSR 2995.3856 572.17 5.2352 MSE. Since f0 f0.05,2,22 3.44 (or since the P-value is considerably smaller than = 0.05), we reject the null hypothesis and conclude that pull strength is linearly related to either wire length or die height, or both. However, we note that this does not necessarily imply that the.

(22) c12. .qxd 5/20/02 9:32 M Page 431 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 431. 12-2 HYPOTHESIS TESTS IN MULTIPLE LINEAR REGRESSION. Table 12-10 Test for Significance of Regression for Example 12-3 Source of Variation. Sum of Squares. Degrees of Freedom. Regression Error or residual Total. 5990.7712 115.1735 6105.9447. 2 22 24. Mean Square. f0. P-value. 2995.3856 5.2352. 572.17. 1.08E-19. relationship found is an appropriate model for predicting pull strength as a function of wire length and die height. Further tests of model adequacy are required before we can be comfortable using this model in practice. Most multiple regression computer programs provide the test for significance of regression in their output display. The middle portion of Table 12-4 is the Minitab output for this example. Compare Tables 12-4 and 12-10 and note their equivalence apart from rounding. The P-value is rounded to zero in the computer output. R2 and Adjusted R2 We may also use the coefficient of multiple determination R2 as a global statistic to assess the fit of the model. Computationally, R2 . SSR SSE 1 SST SST. (12-21). For the wire bond pull strength data, we find that R2 SSRSST 5990.77126105.9447 0.9811. Thus the model accounts for about 98% of the variability in the pull strength response (refer to the Minitab output in Table 12-4). The R2 statistic is somewhat problematic as a measure of the quality of the fit for a multiple regression model because it always increases when a variable is added to a model. To illustrate, consider the model fit to wire bond pull strength data in Example 11-8. This was a simple linear regression model with x1 wire length as the regressor. The value of R2 for this model is R2 0.9640. Therefore, adding xy die height to the model increases R2 by 0.9811 0.9640 0.0171, a very small amount. Since R2 always increases when a regressor is added, it can be difficult to judge whether the increase is telling us anything useful about the new regressor. It is particularly hard to interpret a small increase, such as observed in the pull strength data. Many regression users prefer to use an adjusted R2 statistic:. R2adj 1 . SSE 1n p2 SST 1n 12. (12-22). Because SSE 1n p2 is the error or residual mean square and SST 1n 12 is a constant, R2adj will only increase when a variable is added to the model if the new variable reduces the error mean square. Note that for the multiple regression model for the pull strength data R2adj 0.979 (see the Minitab output in Table 12-4), whereas in Example 11-8 the adjusted R2 for the one-variable model is R2adj 0.962. Therefore, we would conclude that adding x2 die height to the model does result in a meaningful reduction in unexplained variability in the response..

(23) c12. .qxd 5/20/02 2:58 PM Page 432 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 432. CHAPTER 12 MULTIPLE LINEAR REGRESSION. The adjusted R2 statistic essentially penalizes the analyst for adding terms to the model. It is an easy way to guard against overfitting, that is, including regressors that are not really useful. Consequently, it is very useful in comparing and evaluating competing regression models. We will use R2adj for this when we discuss variable selection in regression in Section 12-6.3.. 12-2.2 Tests on Individual Regression Coefficients and Subsets of Coefficients We are frequently interested in testing hypotheses on the individual regression coefficients. Such tests would be useful in determining the potential value of each of the regressor variables in the regression model. For example, the model might be more effective with the inclusion of additional variables or perhaps with the deletion of one or more of the regressors presently in the model. Adding a variable to a regression model always causes the sum of squares for regression to increase and the error sum of squares to decrease (this is why R2 always increases when a variable is added). We must decide whether the increase in the regression sum of squares is large enough to justify using the additional variable in the model. Furthermore, adding an unimportant variable to the model can actually increase the error mean square, indicating that adding such a variable has actually made the model a poorer fit to the data (this is why R2adj is a better measure of global model fit then the ordinary R2). The hypotheses for testing the significance of any individual regression coefficient, say j, are H0: j 0 H1: j 0. (12-23). If H0: j 0 is not rejected, this indicates that the regressor xj can be deleted from the model. The test statistic for this hypothesis is. T0 . ˆ j 2 ˆ 2 Cjj. . ˆ j. se1ˆ j 2. (12-24). where Cjj is the diagonal element of 1X¿X2 1 corresponding to ˆ j. Notice that the denominator of Equation 12-24 is the standard error of the regression coefficient ˆ j. The null hypothesis H0: j 0 is rejected if 0 t0 0 t 2,np. This is called a partial or marginal test because the regression coefficient ˆ j depends on all the other regressor variables xi (i j) that are in the model. More will be said about this in the following example. EXAMPLE 12-4. Consider the wire bond pull strength data, and suppose that we want to test the hypothesis that the regression coefficient for x2 (die height) is zero. The hypotheses are H0: 2 0 H1: 2 0.

(24) c12. .qxd 5/20/02 9:32 M Page 433 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 433. 12-2 HYPOTHESIS TESTS IN MULTIPLE LINEAR REGRESSION. The main diagonal element of the 1X¿X2 1 matrix corresponding to ˆ 2 is C22 0.0000015, so the t-statistic in Equation 12-24 is t0 . ˆ 2 2 ˆ 2C22. . 0.01253 215.2352210.00000152. 4.4767. Note that we have used the estimate of 2 reported to four decimal places in Table 12-10. Since t0.025,22 2.074, we reject H0: 2 0 and conclude that the variable x2 (die height) contributes significantly to the model. We could also have used a P-value to draw conclusions. The P-value for t0 4.4767 is P 0.0002, so with = 0.05 we would reject the null hypothesis. Note that this test measures the marginal or partial contribution of x2 given that x1 is in the model. That is, the t-test measures the contribution of adding the variable x2 die height to a model that already contains x1 wire length. Table 12-4 shows the value of the t-test computed by Minitab. The Minitab t-test statistic is reported to two decimal places. Note that the computer produces a t-test for each regression coefficient in the model. These t-tests indicate that both regressors contribute to the model. There is another way to test the contribution of an individual regressor variable to the model. This approach determines the increase in the regression sum of squares obtained by adding a variable xj (say) to the model, given that other variables xi (i j) are already included in the regression equation. The procedure used to do this is called the general regression significance test, or the extra sum of squares method. This procedure can also be used to investigate the contribution of a subset of the regressor variables to the model. Consider the regression model with k regressor variables y X . (12-25). where y is (n 1), X is (n p), is (p 1), is (n 1), and p k 1. We would like to determine if the subset of regressor variables x1, x2, . . . , xr (r k) as a whole contributes significantly to the regression model. Let the vector of regression coefficients be partitioned as follows: c. 1 d 2. (12-26). where 1 is (r 1) and 2 is [(p r) 1]. We wish to test the hypotheses. H0: 1 0 H1: 1 0. (12-27). where 0 denotes a vector of zeroes. The model may be written as y X X11 X22 . (12-28). where X1 represents the columns of X associated with 1 and X2 represents the columns of X associated with 2..

(25) c12. .qxd 5/20/02 2:58 PM Page 434 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 434. CHAPTER 12 MULTIPLE LINEAR REGRESSION. For the full model (including both 1 and 2), we know that ˆ 1X¿X2 1 X¿y. In addition, the regression sum of squares for all variables including the intercept is SSR 12 ˆ ¿X¿y. 1 p k 1 degrees of freedom2. and MSE . y¿y ˆ X¿y np. SSR() is called the regression sum of squares due to . To find the contribution of the terms in 1 to the regression, fit the model assuming the null hypothesis H0: 1 0 to be true. The reduced model is found from Equation 12-28 as y X22 . (12-29). The least squares estimate of 2 is ˆ 2 1X¿2 X2 2 1X¿2 y, and ˆ ¿ X¿ y SSR 12 2 2 2. 1p r degrees of freedom2. (12-30). The regression sum of squares due to 1 given that 2 is already in the model is SSR 11 0 2 2 SSR 12 SSR 12 2. (12-31). This sum of squares has r degrees of freedom. It is sometimes called the extra sum of squares due to 1. Note that SSR 11 0 2 2 is the increase in the regression sum of squares due to including the variables x1, x2, p , xr in the model. Now SSR 11 0 2 2 is independent of MSE, and the null hypothesis 1 0 may be tested by the statistic. F0 . SSR 11 | 2 2 r MSE. (12-32). If the computed value of the test statistic f0 f ,r,np, we reject H0, concluding that at least one of the parameters in 1 is not zero and, consequently, at least one of the variables x1, x2, p , xr in X1 contributes significantly to the regression model. Some authors call the test in Equation 12-32 a partial F-test. The partial F-test is very useful. We can use it to measure the contribution of each individual regressor xj as if it were the last variable added to the model by computing SSR 1j 0 0, 1, p , j1, j1, p , k 2,. j 1, 2, p , k. This is the increase in the regression sum of squares due to adding xj to a model that already includes x1, . . . , xj1, xj1, . . . , xk. The partial F-test is a more general procedure in that we can measure the effect of sets of variables. In Section 12-6.3 we show how the partial F-test plays a major role in model building—that is, in searching for the best set of regressor variables to use in the model..

(26) c12. .qxd 5/20/02 2:58 PM Page 435 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 12-2 HYPOTHESIS TESTS IN MULTIPLE LINEAR REGRESSION. EXAMPLE 12-5. 435. Consider the wire bond pull strength data in Example 12-1. We will investigate the contribution of the variable x2 (die height) to the model using the partial F-test approach. That is, we wish to test H0: 2 0 H1: 2 0 To test this hypothesis, we need the extra sum of squares due to 2, or SSR 12 0 1,0 2 SSR 11,2,0 2 SSR 11,0 2 SSR 11,2 0 0 2 SSR 11 0 0 2 In Example 12-3 we have calculated n. SSR 11,2 0 0 2 ˆ ¿X¿y . a a yi b i1. n. 5990.7712. 1two degrees of freedom2. and from Example 11-8, where we fit the model Y 0 1x1 , we can calculate SSR 11 0 0 2 ˆ 1Sxy 12.9027212027.71322 5885.8521 1one degree of freedom2 Therefore, SSR 12 0 1,0 2 5990.7712 5885.8521 104.9191 1one degree of freedom2 This is the increase in the regression sum of squares due to adding x2 to a model already containing x1. To test H0: 2 0, calculate the test statistic f0 . SSR 12 0 1,0 2 1 104.91911 20.04 5.2352 MSE. Note that the MSE from the full model, using both x1 and x2, is used in the denominator of the test statistic. Since f0.05,1,22 4.30, we reject H0: 2 0 and conclude that the regressor die height (x2) contributes significantly to the model. Table 12-4 shows the Minitab regression output for the wire bond pull strength data. Just below the analysis of variance summary in this table the quantity labeled “Seq SS” shows the sum of squares obtained by fitting x1 alone (5885.9) and the sum of squares obtained by fitting x2 after x1. Notationally, these are referred to above as SSR 11 0 0 2 and SSR 12 0 1,0 2 . Since the partial F-test in the above example involves a single variable, it is equivalent to the t-test. To see this, recall from Example 12-5 that the t-test on H0: 2 0 resulted in the test statistic t0 4.4767. Furthermore, the square of a t-random variable with degrees of freedom is an F-random variable with one and degrees of freedom, and we note that t 20 (4.4767)2 20.04 f0.. 12-2.3. More About the Extra Sum of Squares Method (CD Only).

(27) c12. .qxd 5/20/02 9:33 M Page 436 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 436. CHAPTER 12 MULTIPLE LINEAR REGRESSION. EXERCISES FOR SECTION 12-2 12-13. Consider the regression model fit to the soil shear strength data in Exercise 12-1. (a) Test for significance of regression using 0.05. What is the P-value for this test? (b) Construct the t-test on each regression coefficient. What are your conclusions, using 0.05? 12-14. Consider the absorption index data in Exercise 12-2. The total sum of squares for y is SST 742.00. (a) Test for significance of regression using 0.01. What is the P-value for this test? (b) Test the hypothesis H0: 1 0 versus H1: 1 0 using 0.01. What is the P-value for this test? What conclusion can you draw about the usefulness of x1 as a regressor in this model? 12-15. Consider the NFL data in Exercise 12-4. (a) Test for significance of regression using 0.05. What is the P-value for this test? (b) Conduct the t-test for each regression coefficient 2, 7, and 8. Using 0.05, what conclusions can you draw about the variables in this model? 12-16. Reconsider the NFL data in Exercise 12-4. (a) Find the amount by which the regressor x8 (opponents’ yards rushing) increases the regression sum of squares. (b) Use the results from part (a) above and Exercise 12-14 to conduct an F-test for H0: 8 0 versus H1: 8 0 using 0.05. What is the P-value for this test? What conclusions can you draw? 12-17. Consider the gasoline mileage data in Exercise 12-5. (a) Test for significance of regression using 0.05. What conclusions can you draw? (b) Find the t-test statistic for both regressors. Using 0.05, what conclusions can you draw? Do both regressors contribute to the model? 12-18. A regression model Y 0 1x1 2 x2 3 x3 has been fit to a sample of n 25 observations. The calculated t-ratios ˆ j se 1ˆ j 2, j 1, 2, 3 are as follows: for 1, t0 4.82, for 2, t0 8.21 and for 3, t0 0.98. (a) Find P-values for each of the t-statistics. (b) Using 0.05, what conclusions can you draw about the regressor x3? Does it seem likely that this regressor contributes significantly to the model? 12-19. Consider the electric power consumption data in Exercise 12-6. (a) Test for significance of regression using 0.05. What is the P-value for this test? (b) Use the t-test to assess the contribution of each regressor to the model. Using 0.05, what conclusions can you draw? 12-20. Consider the bearing wear data in Exercise 12-7 with no interaction.. (a) Test for significance of regression using 0.05. What is the P-value for this test? What are your conclusions? (b) Compute the t-statistics for each regression coefficient. Using 0.05, what conclusions can you draw? (c) Use the extra sum of squares method to investigate the usefulness of adding x2 load to a model that already contains x1 oil viscosity. Use 0.05. 12-21. Reconsider the bearing wear data from Exercises 12-7 and 12-20. (a) Refit the model with an interaction term. Test for significance of regression using 0.05. (b) Use the extra sum of squares method to determine whether the interaction term contributes significantly to the model. Use 0.05. (c) Estimate 2 for the interaction model. Compare this to the estimate of 2 from the model in Exercise 12-20. 12-22. Consider the wire bond pull strength data in Exercise 12-8. (a) Test for significance of regression using 0.05. Find the P-value for this test. What conclusions can you draw? (b) Calculate the t-test statistic for each regression coefficient. Using 0.05, what conclusions can you draw? Do all variables contribute to the model? 12-23. Reconsider the semiconductor data in Exercise 12-9. (a) Test for significance of regression using 0.05. What conclusions can you draw? (b) Calcuate the t-test statistic for each regression coefficient. Using 0.05, what conclusions can you draw? 12-24. Exercise 12-10 presents data on heat treating gears. (a) Test the regression model for significance of regression. Using 0.05, find the P-value for the test and draw conclusions. (b) Evaluate the contribution of each regressor to the model using the t-test with 0.05. (c) Fit a new model to the response PITCH using new regressors x1 SOAKTIME SOAKPCT and x2 DIFFTIME DIFFPCT. (d) Test the model in part (c) for significance of regression using 0.05. Also calculate the t-test for each regressor and draw conclusions. (e) Estimate 2 for the model from part (c) and compare this to the estimate of 2 for the model in part (a). Which estimate is smaller? Does this offer any insight regarding which model might be preferable? 12-25. Data on National Hockey League team performance was presented in Exercise 12-11. (a) Test the model from this exercise for significance of regression using 0.05. What conclusions can you draw?.

(28) c12. .qxd 5/20/02 9:33 M Page 437 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 437. 12-3 CONFIDENCE INTERVALS IN MULTIPLE LINEAR REGRESSION. play goals. Does this seem to be a logical choice of regressors, considering your answer to part (b)? Test this new model for significance of regression and evaluate the contribution of each regressor to the model using the t-test. Use 0.05.. (b) Use the t-test to evaluate the contribution of each regressor to the model. Does it seem that all regressors are necessary? Use 0.05. (c) Fit a regression model relating the number of games won to the number of points scored and the number of power. 12-3 CONFIDENCE INTERVALS IN MULTIPLE LINEAR REGRESSION 12-3.1. Confidence Intervals on Individual Regression Coefficients In multiple regression models, it is often useful to construct confidence interval estimates for the regression coefficients 5 j 6. The development of a procedure for obtaining these confidence intervals requires that the errors 5i 6 are normally and independently distributed with mean zero and variance 2. This is the same assumption required in hypothesis testing. Therefore, the observations {Yi} are normally and independently distributed with mean 0 k g j1 j xij and variance 2. Since the least squares estimator ˆ is a linear combination of the observations, it follows that ˆ is normally distributed with mean vector and covariance matrix 2 1X¿X2 1. Then each of the statistics T. ˆ j j 2 ˆ 2Cjj. j 0, 1, p , k. (12-33). has a t distribution with n p degrees of freedom, where Cjj is the jjth element of the 1X¿X2 1 matrix, and ˆ 2 is the estimate of the error variance, obtained from Equation 12-16. This leads to the following 100(1 )% confidence interval for the regression coefficient j, j 0, 1, p , k. Definition A 100(1 ) % confidence interval on the regression coefficient j, j 0, 1, p , k in the multiple linear regression model is given by ˆ j t2,np 2 ˆ 2Cjj j ˆ j t2,np 2 ˆ 2Cjj. (12-34). Because 2 ˆ 2Cjj is the standard error of the regression coefficient ˆ j, we would also write the CI formula as ˆ j t2,np se1ˆ j 2 j ˆ j t2,np se1ˆ j 2. EXAMPLE 12-6. We will construct a 95% confidence interval on the parameter 1 in the wire bond pull strength problem. The point estimate of 1 is ˆ 1 2.74427 and the diagonal element of 1X¿X2 1 corresponding to 1 is C11 0.001671. The estimate of 2 is ˆ 2 5.2352, and t0.025,22 2.074. Therefore, the 95% CI on 1 is computed from Equation 12-34 as 2.74427 12.0742 215.235221.0016712 1 2.74427 12.0742 215.235221.0016712 which reduces to 2.55029 1 2.93825.

(29) c12. .qxd 5/20/02 9:33 M Page 438 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 438. CHAPTER 12 MULTIPLE LINEAR REGRESSION. 12-3.2. Confidence Interval on the Mean Response We may also obtain a confidence interval on the mean response at a particular point, say, x01, x02, p , x0k. To estimate the mean response at this point, define the vector 1 x01 x0 Ex02U o x0k The mean response at this point is E1Y 0 x0 2 Y 0 x0 x¿0, which is estimated by ˆ ˆ Y 0 x0 x¿0. (12-35). This estimator is unbiased, since E1x¿0ˆ 2 x¿0 E1Y 0 x0 2 Y 0 x0 and the variance of ˆ Y 0 x0 is V1ˆ Y 0 x0 2 2x¿0 1X¿X2 1x0. (12-36). A 100(1 ) % CI on Y 0 x0 can be constructed from the statistic ˆ Y 0 x0 Y 0 x0. 2 ˆ 2 x¿0 1X¿X21 x0. (12-37). ˛. Definition For the multiple linear regression model, a 100(1 )% confidence interval on the mean response at the point x01, x02, . . . , x0k is ˆ Y 0 x0 t2,np 2 ˆ 2 x¿0 1X¿X2 1 x0 ˛. Y 0 x0 ˆ Y 0 x0 t2,np 2 ˆ 2 x¿0 1¿21 x0 ˛. (12-38). Equation 12-38 is a CI about the regression plane (or hyperplane). It is the multiple regression generalization of Equation 11-31. EXAMPLE 12-7. The engineer in Example 12-1 would like to construct a 95% CI on the mean pull strength for a wire bond with wire length x1 8 and die height x2 275. Therefore, 1 x0 £ 8 § 275.

(30) c12. .qxd 5/20/02 9:33 M Page 439 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 439. 12-4 PREDICTION OF NEW OBSERVATIONS. The estimated mean response at this point is found from Equation 12-35 as ˆ Y |x0 x0¿ ˆ 31. 8. 2.26379 2754 £ 2.74427 § 27.66 0.01253. The variance of ˆ Y 0 x0 is estimated by ˆ 2x0¿ 1¿2 1x0. .214653 .007491 .000340 1 5.2352 31 8 2754 £ .007491 .001671 .000019 § £ 8 § .000340 .000019 .0000015 275 5.2352 10.044442 0.23266. Therefore, a 95% CI on the mean pull strength at this point is found from Equation 12-38 as 27.66 2.074 10.23266 Y 0 x0 27.66 2.074 10.23266 which reduces to 26.66 Y |x0 28.66 Some computer software packages will provide estimates of the mean for a point of interest x0 and the associated CI. Table 12-4 shows the Minitab output for Example 12-7. Both the estimate of the mean and the 95% CI are provided.. 12-4. PREDICTION OF NEW OBSERVATIONS A regression model can be used to predict new or future observations on the response variable Y corresponding to particular values of the independent variables, say, x01, x02, p , x0k. If x¿0 31, x01, x02, p , x0k 4 , a point estimate of the future observation Y0 at the point x01, x02, p , x0k is yˆ 0 x¿0 ˆ ˛. (12-39). A 100(1 )% prediction interval for this future observation is yˆ 0 t2,np 2 ˆ 2 11 x¿0 1¿2 1 x0 2 ˛. Y0 yˆ 0 t2,np 2 ˆ 2 11 x¿0 1¿2 1 x0 2 ˛. (12-40). This prediction interval is a generalization of the prediction interval given in Equation 11-33 for a future observation in simple linear regression. If you compare the prediction interval Equation 12-40 with the expression for the confidence interval on the mean, Equation 12-38, you will observe that the prediction interval is always wider than the confidence interval. The confidence interval expresses the error in estimating the mean of a distribution, while the prediction interval expresses the error in predicting a future observation from the distribution at.

(31) .qxd 5/20/02 9:33 M Page 440 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH 1 14 FIN L:Quark Files:. 440. CHAPTER 12 MULTIPLE LINEAR REGRESSION. x2. Original range for x2. c12. Joint region of original area. x01. x01. x1 Original range for x1. Figure 12-5 An example of extrapolation in multiple regression.. the point x0. This must include the error in estimating the mean at that point, as well as the inherent variability in the random variable Y at the same value x x0. In predicting new observations and in estimating the mean response at a given point x01, x02, . . . , x0k, we must be careful about extrapolating beyond the region containing the original observations. It is very possible that a model that fits well in the region of the original data will no longer fit well outside of that region. In multiple regression it is often easy to inadvertently extrapolate, since the levels of the variables (xi1, xi2, . . . , xik), i 1, 2, . . . , n, jointly define the region containing the data. As an example, consider Fig. 12-5, which illustrates the region containing the observations for a two-variable regression model. Note that the point (x01, x02) lies within the ranges of both regressor variables x1 and x2, but it is outside the region that is actually spanned by the original observations. Thus, either predicting the value of a new observation or estimating the mean response at this point is an extrapolation of the original regression model. EXAMPLE 12-8. Suppose that the engineer in Example 12-1 wishes to construct a 95% prediction interval on the wire bond pull strength when the wire length is x1 8 and the die height is x2 275. Note ˆ 27.66. that x¿0 [1 8 275], and the point estimate of the pull strength is yˆ 0 x¿0 1 Also, in Example 12-7 we calculated x¿0 1¿2 x0 0.04444. Therefore, from Equation 12-40 we have 27.66 2.074 25.235211 0.044442 Y0 27.66 2.074 25.235211 0.044442 ˛. and the 95% prediction interval is 22.81 Y0 32.51 Notice that the prediction interval is wider than the confidence interval on the mean response at the same point, calculated in Example 12-7. The Minitab output in Table 12-4 also displays this prediction interval..