Praise for the First Edition

“This book will serve to greatly complement the growing number of texts dealing with mixed models, and I highly recommend including it in one’s personal library.”

—Journal of the American Statistical Association

 Mixed modeling is a crucial area of statistics, enabling the analysis of clustered and longitudinal data. Mixed Models: Theory and Applications with R, Second Edition fills a gap in existing literature between mathematical and applied statistical books by presenting a powerful examination of mixed model theory and application with special attention given to the implementation in R.

The new edition provides in-depth mathematical coverage of mixed models’ statistical properties and numerical algorithms, as well as nontraditional applications, such as regrowth curves, shapes, and images. The book features the latest topics in statistics including modeling of complex clustered or longitudinal data, modeling data with multiple sources of variation, modeling biological variety and heterogeneity, Healthy Akaike Information Criterion (HAIC), parameter multidimensionality, and statistics of image processing.

Mixed Models: Theory and Applications with R, Second Edition features unique applications of mixed model methodology, as well as:

• Comprehensive theoretical discussions illustrated by examples and figures
• Over 300 exercises, end-of-section problems, updated data sets, and R subroutines
• Problems and extended projects requiring simulations in R intended to reinforce material
• Summaries of major results and general points of discussion at the end of each chapter
• Open problems in mixed modeling methodology, which can be used as the basis for research or PhD dissertations

Ideal for graduate-level courses in mixed statistical modeling, the book is also an excellent reference for professionals in a range of fields, including cancer research, computer science, and engineering.

EUGENE DEMIDENKO, PhD, is Professor of Biostatistics and Epidemiology at the Geisel School of Medicine and Department of Mathematics at Dartmouth College. Dr. Demidenko carries out collaborative work at the Thayer School of Engineering, Dartmouth College, including nanocancer therapy and electrical impedance tomography for breast cancer detection. Dr. Demidenko is recipient of several awards from the American Statistical Association and has been an invited lecturer at several institutes and academies around the world.

Preface xvii

Preface to the Second Edition xix

R software and functions xx

Data Sets xxii

Open Problems in Mixed Models xxiii

1 Introduction: Why Mixed Models? 1

1.1 Mixed effects for clustered data 2

1.2 ANOVA, variance components, and the mixed model 4

1.3 Other special cases of the mixed effects model 6

1.4 A compromise between Bayesian and frequentist approaches 7

1.5 Penalized likelihood and mixed effects 9

1.6 Healthy Akaike information criterion 11

1.7 Penalized smoothing 13

1.8 Penalized polynomial fitting 16

1.9 Restraining parameters, or what to eat 18

1.10 Ill-posed problems, Tikhonov regularization, and mixed effects 20

1.11 Computerized tomography and linear image reconstruction 23

1.12 GLMM for PET 26

1.13 Maple shape leaf analysis 29

1.14 DNA Western blot analysis 31

1.15 Where does the wind blow? 33

1.16 Software and books36

1.17 Summary points 37

2 MLE for LME Model 41

2.1 Example: Weight versus height 42

2.2 The model and log-likelihood functions 45

2.3 Balanced random-coefficient model 60

2.4 LME model with random intercepts 64

2.5 Criterion for the MLE existence 72

2.6 Criterion for positive definiteness of matrix D74

2.7 Preestimation bounds for variance parameters 77

2.8 Maximization algorithms79

2.9 Derivatives of the log-likelihood function 81

2.10 Newton—Raphson algorithm 83

2.11 Fisher scoring algorithm85

2.12 EM algorithm 88

2.13 Starting point 93

2.14 Algorithms for restricted MLE 96

2.15 Optimization on nonnegative definite matrices 97

2.16 lmeFS and lme in R 108

2.17 Appendix: Proof of the MLE existence 112

2.18 Summary points 115

3 Statistical Properties of the LME Model 119

3.1 Introduction 119

3.2 Identifiability of the LMEmodel 119

3.3 Information matrix for variance parameters 122

3.4 Profile-likelihood confidence intervals 133

3.5 Statistical testing of the presence of random effects 135

3.6 Statistical properties of MLE 139

3.7 Estimation of random effects 148

3.8 Hypothesis and membership testing 153

3.9 Ignoring random effects 157

3.10 MINQUE for variance parameters 160

3.11 Method of moments 169

3.12 Variance least squares estimator 173

3.13 Projection on D+ space 178

3.14 Comparison of the variance parameter estimation 178

3.15 Asymptotically efficient estimation for β 182

3.16 Summary points 183

4 Growth Curve Model and Generalizations 187

4.1 Linear growth curve model 187

4.2 General linear growth curve model 203

4.3 Linear model with linear covariance structure 221

4.4 Robust linear mixed effects model 235

4.5 Appendix: Derivation of the MM estimator 243

4.6 Summary points 244

5 Meta-analysis Model 247

5.1 Simple meta-analysis model 248

5.2 Meta-analysis model with covariates 275

5.3 Multivariate meta-analysis model 280

5.4 Summary points 291

6 Nonlinear Marginal Model 293

6.1 Fixed matrix of random effects 294

6.2 Varied matrix of random effects 307

6.3 Three types of nonlinear marginal models 318

6.4 Total generalized estimating equations approach 323

6.5 Summary points 330

7 Generalized Linear Mixed Models 333

7.1 Regression models for binary data 334

7.2 Binary model with subject-specific intercept 357

7.3 Logistic regression with random intercept 364

7.4 Probit model with random intercept 384

7.5 Poisson model with random intercept 388

7.6 Random intercept model: overview 403

7.7 Mixed models with multiple random effects 404

7.8 GLMM and simulation methods 413

7.9 GEE for clustered marginal GLM 418

7.10 Criteria for MLE existence for binary model 426

7.11 Summary points 431

8 Nonlinear Mixed Effects Model 435

8.1 Introduction 435

8.2 The model 436

8.3 Example: Height of girls and boys 439

8.4 Maximum likelihood estimation 441

8.5 Two-stage estimator 444

8.6 First-order approximation 450

8.7 Lindstrom—Bates estimator 452

8.8 Likelihood approximations 457

8.9 One-parameter exponential model 460

8.10 Asymptotic equivalence of the TS and LB estimators 467

8.11 Bias-corrected two-stage estimator 469

8.12 Distribution misspecification 471

8.13 Partially nonlinear marginal mixed model 474

8.14 Fixed sample likelihood approach475

8.15 Estimation of random effects and hypothesis testing 478

8.16 Example (continued) 479

8.17 Practical recommendations 481

8.18 Appendix: Proof of theorem on equivalence 482

8.19 Summary points 485

9 Diagnostics and Influence Analysis 489

9.1 Introduction 489

9.2 Influence analysis for linear regression 490

9.3 The idea of infinitesimal influence 493

9.4 Linear regression model 495

9.5 Nonlinear regression model 512

9.6 Logistic regression for binary outcome 517

9.7 Influence of correlation structure 526

9.8 Influence of measurement error 527

9.9 Influence analysis for the LME model 530

9.10 Appendix: MLE derivative with respect to σ2 536

9.11 Summary points 537

10 Tumor Regrowth Curves 541

10.1 Survival curves 543

10.2 Double—exponential regrowth curve 545

10.3 Exponential growth with fixed regrowth time 559

10.4 General regrowth curve 565

10.5 Double—exponential transient regrowth curve 566

10.6 Gompertz transient regrowth curve 573

10.7 Summary points 576

11 Statistical Analysis of Shape 579

11.1 Introduction 579

11.2 Statistical analysis of random triangles 581

11.3 Face recognition 584

11.4 Scale-irrelevant shape model 585

11.5 Gorilla vertebrae analysis 589

11.6 Procrustes estimation of the mean shape 591

11.7 Fourier descriptor analysis 598

11.8 Summary points 607

12 Statistical Image Analysis 609

12.1 Introduction 609

12.2 Testing for uniform lighting 612

12.3 Kolmogorov—Smirnov image comparison 616

12.4 Multinomial statistical model for images 620

12.5 Image entropy 623

12.6 Ensemble of unstructured images 627

12.7 Image alignment and registration 640

12.8 Ensemble of structured images 652

12.9 Modeling spatial correlation 654

12.10 Summary points 660

13 Appendix: Useful Facts and Formulas 663

13.1 Basic facts of asymptotic theory 663

13.2 Some formulas of matrix algebra 670

13.3 Basic facts of optimization theory 674

References 683

Index 713