Praise for the Third Edition

"...this is an excellent book which could easily be used as a course text..."
—International Statistical Institute

The Fourth Edition of Applied Linear Regression provides a thorough update of the basic theory and methodology of linear regression modeling. Demonstrating the practical applications of linear regression analysis techniques, the Fourth Edition uses interesting, real-world exercises and examples.

Stressing central concepts such as model building, understanding parameters, assessing fit and reliability, and drawing conclusions, the new edition illustrates how to develop estimation, confidence, and testing procedures primarily through the use of least squares regression. While maintaining the accessible appeal of each previous edition,Applied Linear Regression, Fourth Edition features:

• Graphical methods stressed in the initial exploratory phase, analysis phase, and summarization phase of an analysis
• In-depth coverage of parameter estimates in both simple and complex models, transformations, and regression diagnostics
• Newly added material on topics including testing, ANOVA, and variance assumptions
• Updated methodology, such as bootstrapping, cross-validation binomial and Poisson regression, and modern model selection methods

Applied Linear Regression, Fourth Edition is an excellent textbook for upper-undergraduate and graduate-level students, as well as an appropriate reference guide for practitioners and applied statisticians in engineering, business administration, economics, and the social sciences.

SANFORD WEISBERG, PhD, is Professor of Statistics and Director of the Statistical Consulting Service in the School of Statistics at the University of Minnesota. He is also a coauthor of Applied Regression Including Computing and Graphics and An Introduction to Regression Graphics, both published by Wiley.

1 Scatterplots 1

1.1 Scatterplots 2

1.2 Mean Functions 9

1.3 Variance Functions 12

1.4 Summary Graph 12

1.5 Tools for Looking at Scatterplots 13

1.6 Scatterplot Matrices 15

1.7 Problems 17

2 Simple Linear Regression 21

2.1 Ordinary Least Squares Estimation 22

2.2 Least Squares Criterion 24

2.3 Estimating the Variance 𝜎2 26

2.4 Properties of Least Squares Estimates 27

2.5 Estimated Variances 28

2.6 Confidence Intervals and 𝑡-Tests 29

2.7 The Coefficient of Determination, 𝑅2 33

2.8 The Residuals 35

2.9 Problems 37

3 Multiple Regression 49

3.1 Adding a Regressor to a Simple Linear Regression Model 49

3.2 The Multiple Linear Regression Model 53

3.3 Predictors and Regressors 53

3.4 Ordinary Least Squares 57

3.5 Predictions, Fitted Values and Linear Combinations 65

3.6 Problems 66

4 Interpretation of Main Effects 71

4.1 Understanding Parameter Estimates 71

4.2 Dropping Regressors 81

4.3 Experimentation Versus Observation 84

4.4 Sampling from a Normal Population 86

4.5 More on 𝑅2 88

4.6 Problems 90

5 Complex Regressors 95

5.1 Factors 95

5.2 Many Factors 105

5.3 Polynomial Regression 106

5.4 Splines 109

5.5 Principal Components 112

5.6 Missing Data 115

5.7 Problems 118

6 Testing and Analysis of Variance 129

6.1 𝐹-tests 130

6.2 The Analysis of Variance 134

6.3 Comparisons of Means 138

6.4 Power and Non-null Distributions 138

6.5 Wald Tests 140

6.6 Interpreting Tests 142

6.7 Problems 145

7 Variances 151

7.1 Weighted Least Squares 151

7.2 Misspecified Variances 157

7.3 General Correlation Structures 162

7.4 Mixed Models 163

7.5 Variance Stabilizing Transformations 165

7.6 The Delta Method 166

7.7 The Bootstrap 168

7.8 Problems 173

8 Transformations 179

8.1 Transformation Basics 179

8.2 A General Approach to Transformations 185

8.3 Transforming the Response 190

8.4 Transformations of Nonpositive Variables 192

8.5 Additive Models 192

8.6 Problems 193

9 Regression Diagnostics 199

9.1 The Residuals 199

9.2 Testing for Curvature 206

9.3 Nonconstant Variance 208

9.4 Outliers 208

9.5 Influence of Cases 212

9.6 Normality Assumption 218

9.7 Problems 220

10 Variable Selection 227

10.1 Variable Selection and Parameter Assessment 228

10.2 Variable Selection for Discovery 230

10.3 Model Selection for Prediction 238

10.4 Problems 241

11 Nonlinear Regression 245

11.1 Estimation for Nonlinear Mean Functions 246

11.2 Inference Assuming Large Samples 249

11.3 Starting Values 249

11.4 Bootstrap Inference 255

11.5 Further Reading 257

11.6 Problems 258

12 Binomial and Poisson Regression 263

12.1 Distributions for Counted Data 263

12.2 Regression Models For Counts 265

12.3 Poisson Regression 271

12.4 Transferring What You Know about Linear Models 276

12.5 Generalized Linear Models 278

12.6 Problems 278

A Appendix 283

A.1 Website 283

A.2 Means, Variances, Covariances and Correlations 283

A.3 Least Squares for Simple Regression 286

A.4 Means and Variances of Least Squares Estimates 286

A.5 Estimating E(𝑌 |𝑋) using a Smoother 288

A.6 A Brief Introduction to Matrices and Vectors 290

A.7 Random Vectors 295

A.8 Least Squares Using Matrices 295

A.9 The QR factorization 299

A.10 Spectral Decomposition 300

A.11 Maximum Likelihood Estimates 300

A.12 The Box–Cox Method for Transformations 302

A.13 Case Deletion in Linear Regression 305

Bibliography 321

Index 322