| Preface | 373 |
|---|
| 8 | 373 |
|---|
| Contents | 12 |
|---|
| 1. Linear Regression | 16 |
|---|
| 1.1 Introduction | 16 |
| 1.2 The Method of Least Squares | 18 |
| 1.2.1 Correlation version | 22 |
| 1.2.2 Large-sample limit | 23 |
| 1.3 The origins of regression | 24 |
| 1.4 Applications of regression | 26 |
| 1.5 The Bivariate Normal Distribution | 29 |
| 1.6 Maximum Likelihood and Least Squares | 36 |
| 1.7 Sums of Squares | 38 |
| 1.8 Two regressors | 41 |
| Exercises | 43 |
| 2. The Analysis of Variance (ANOVA) | 48 |
|---|
| 2.1 The Chi-Square Distribution | 48 |
| 2.2 Change of variable formula and Jacobians | 51 |
| 2.3 The Fisher F-distribution | 52 |
| 2.4 Orthogonality | 53 |
| 2.5 Normal sample mean and sample variance | 54 |
| 2.6 One-Way Analysis of Variance | 57 |
| 2.7 Two-Way ANOVA | No Replications |
| 2.8 Two-Way ANOVA: Replications and Interaction | 67 |
| Exercises | 71 |
| 3. Multiple Regression | 75 |
|---|
| 3.1 The Normal Equations | 75 |
| 3.2 Solution of the Normal Equations | 78 |
| 3.3 Properties of Least-Squares Estimators | 84 |
| 3.4 Sum-of-Squares Decompositions | 87 |
| 3.4.1 Coefficient of determination | 93 |
| 3.5 Chi-Square Decomposition | 94 |
| 3.5.1 Idempotence, Trace and Rank | 95 |
| 3.5.2 Quadratic forms in normal variates | 96 |
| 3.5.3 Sums of Projections | 96 |
| 3.6 Orthogonal Projections and Pythagoras's Theorem | 99 |
| 3.7 Worked examples | 103 |
| Exercises | 108 |
| 4. Further Multilinear Regression | 112 |
|---|
| 4.1 Polynomial Regression | 112 |
| 4.1.1 The Principle of Parsimony | 115 |
| 4.1.2 Orthogonal polynomials | 116 |
| 4.1.3 Packages | 116 |
| 4.2 Analysis of Variance | 117 |
| 4.3 The Multivariate Normal Distribution | 118 |
| 4.4 The Multinormal Density | 124 |
| 4.4.1 Estimation for the multivariate normal | 126 |
| 4.5 Conditioning and Regression | 128 |
| 4.6 Mean-square prediction | 134 |
| 4.7 Generalised least squares and weighted regression | 136 |
| Exercises | 138 |
| 5. Adding additional covariates and the Analysisof Covariance | 141 |
|---|
| 5.1 Introducing further explanatory variables | 141 |
| 5.1.1 Orthogonal parameters | 145 |
| 5.2 ANCOVA | 147 |
| Interactions | 148 |
| 5.2.1 Nested Models | 151 |
| Update | 151 |
| Akaike Information Criterion (AIC) | 152 |
| Step | 152 |
| 5.3 Examples | 152 |
| Exercises | 157 |
| 6. Linear Hypotheses | 161 |
|---|
| 6.1 Minimisation Under Constraints | 161 |
| 6.2 Sum-of-Squares Decomposition and F-Test | 164 |
| 6.3 Applications: Sequential Methods | 169 |
| 6.3.1 Forward selection | 169 |
| 6.3.2 Backward selection | 170 |
| 6.3.3 Stepwise regression | 171 |
| Exercises | 172 |
| 7. Model Checking and Transformation of Data | 175 |
|---|
| 7.1 Deviations from Standard Assumptions | 175 |
| Residual Plots | 175 |
| Scatter Plots | 175 |
| Non-constant Variance | 176 |
| Unaccounted-for Structure | 176 |
| Outliers | 176 |
| Detecting outliers via residual analysis | 177 |
| Influential Data Points | 178 |
| Cook's distance | 179 |
| Non-additive or non-Gaussian errors | 180 |
| Correlated Errors | 180 |
| 7.2 Transformation of Data | 180 |
| Dimensional Analysis | 183 |
| 7.3 Variance-Stabilising Transformations | 183 |
| Taylor's Power Law | 184 |
| Delta Method | 185 |
| 7.4 Multicollinearity | 186 |
| Regression Diagnostics | 189 |
| Exercises | 189 |
| 8. Generalised Linear Models | 193 |
|---|
| 8.1 Introduction | 193 |
| 8.2 Definitions and examples | 195 |
| 8.2.1 Statistical testing and model comparisons | 197 |
| 8.2.2 Analysis of residuals | 199 |
| 8.2.3 Athletics times | 200 |
| 8.3 Binary models | 202 |
| 8.4 Count data, contingency tables and log-linear models | 205 |
| 8.5 Over-dispersion and the Negative Binomial Distribution | 209 |
| 8.5.1 Practical applications: Analysis of over-dispersed models in R | 209 |
| 211 | 209 |
|---|
| Exercises | 212 |
| 9. Other topics | 214 |
|---|
| 9.1 Mixed models | 214 |
| 9.1.1 Mixed models and Generalised Least Squares | 217 |
| 9.2 Non-parametric regression | 222 |
| 9.2.1 Kriging | 224 |
| 9.3 Experimental Design | 226 |
| 9.3.1 Optimality criteria | 226 |
| 9.3.2 Incomplete designs | 227 |
| 9.4 Time series | 230 |
| 9.4.1 Cointegration and spurious regression | 231 |
| 9.5 Survival analysis | 233 |
| 9.5.1 Proportional hazards | 235 |
| 9.6 p | 235 |
| 9.6 p | 235 |
|---|
| 236 | 235 |
|---|
| Solutions | 237 |
|---|
| Dramatis Personae: Who did what when | 279 |
|---|
| Bibliography | 281 |
|---|
| Index | 288 |