Note: sections marked with an asterisk can be skipped on first reading. |
|
Preface |
|
xv | |
Preface to the First Edition |
|
xxi | |
About the Companion Website |
|
xxix | |
1 Introduction |
|
1 | (16) |
|
1.1 Classical and robust approaches to statistics |
|
|
1 | (1) |
|
1.2 Mean and standard deviation |
|
|
2 | (4) |
|
1.3 The "three sigma edit" rule |
|
|
6 | (2) |
|
|
8 | (4) |
|
1.4.1 Straight-line regression |
|
|
8 | (1) |
|
1.4.2 Multiple linear regression |
|
|
9 | (3) |
|
1.5 Correlation coefficients |
|
|
12 | (1) |
|
1.6 Other parametric models |
|
|
13 | (3) |
|
|
16 | (1) |
2 Location and Scale |
|
17 | (34) |
|
|
17 | (2) |
|
2.2 Formalizing departures from normality |
|
|
19 | (3) |
|
2.3 M-estimators of location |
|
|
22 | (9) |
|
2.3.1 Generalizing maximum likelihood |
|
|
22 | (3) |
|
2.3.2 The distribution of M-estimators |
|
|
25 | (3) |
|
2.3.3 An intuitive view of M-estimators |
|
|
28 | (1) |
|
2.3.4 Redescending M-estimators |
|
|
29 | (2) |
|
2.4 Trimmed and Winsorized means |
|
|
31 | (2) |
|
2.5 M-estimators of scale |
|
|
33 | (2) |
|
2.6 Dispersion estimators |
|
|
35 | (2) |
|
2.7 M-estimators of location with unknown dispersion |
|
|
37 | (3) |
|
2.7.1 Previous estimation of dispersion |
|
|
38 | (1) |
|
2.7.2 Simultaneous M-estimators of location and dispersion |
|
|
38 | (2) |
|
2.8 Numerical computing of M-estimators |
|
|
40 | (2) |
|
2.8.1 Location with previously-computed dispersion estimation |
|
|
40 | (1) |
|
|
41 | (1) |
|
2.8.3 Simultaneous estimation of location and dispersion |
|
|
42 | (1) |
|
2.9 Robust confidence intervals and tests |
|
|
42 | (3) |
|
2.9.1 Confidence intervals |
|
|
42 | (2) |
|
|
44 | (1) |
|
2.10 Appendix: proofs and complements |
|
|
45 | (3) |
|
|
45 | (1) |
|
2.10.2 Asymptotic normality of M-estimators |
|
|
46 | (1) |
|
|
47 | (1) |
|
|
47 | (1) |
|
2.10.5 Alternative algorithms for M-estimators |
|
|
47 | (1) |
|
2.11 Recommendations and software |
|
|
48 | (1) |
|
|
49 | (2) |
3 Measuring Robustness |
|
51 | (36) |
|
3.1 The influence function |
|
|
55 | (3) |
|
3.1.1 *The convergence of the SC to the IF |
|
|
57 | (1) |
|
|
58 | (4) |
|
3.2.1 Location M-estimators |
|
|
59 | (1) |
|
3.2.2 Scale and dispersion estimators |
|
|
59 | (1) |
|
3.2.3 Location with previously-computed dispersion estimator |
|
|
60 | (1) |
|
3.2.4 Simultaneous estimation |
|
|
61 | (1) |
|
3.2.5 Finite-sample breakdown point |
|
|
61 | (1) |
|
3.3 Maximum asymptotic bias |
|
|
62 | (2) |
|
3.4 Balancing robustness and efficiency |
|
|
64 | (2) |
|
3.5 *"Optimal" robustness |
|
|
66 | (4) |
|
3.5.1 Bias-and variance-optimality of location estimators |
|
|
66 | (1) |
|
3.5.2 Bias optimality of scale and dispersion estimators |
|
|
66 | (1) |
|
3.5.3 The infinitesimal approach |
|
|
67 | (1) |
|
3.5.4 The Hampel approach |
|
|
68 | (2) |
|
3.5.5 Balancing bias and variance: the general problem |
|
|
70 | (1) |
|
3.6 Multidimensional parameters |
|
|
70 | (2) |
|
3.7 *Estimators as functionals |
|
|
72 | (4) |
|
3.8 Appendix: Proofs of results |
|
|
76 | (9) |
|
3.8.1 IF of general M-estimators |
|
|
76 | (1) |
|
3.8.2 Maximum BP of location estimators |
|
|
76 | (1) |
|
3.8.3 BP of location M-estimators |
|
|
77 | (2) |
|
3.8.4 Maximum bias of location M-estimators |
|
|
79 | (1) |
|
3.8.5 The minimax bias property of the median |
|
|
80 | (1) |
|
|
80 | (2) |
|
|
82 | (3) |
|
|
85 | (2) |
4 Linear Regression 1 |
|
87 | (28) |
|
|
87 | (4) |
|
4.2 Review of the least squares method |
|
|
91 | (3) |
|
4.3 Classical methods for outlier detection |
|
|
94 | (3) |
|
4.4 Regression M-estimators |
|
|
97 | (6) |
|
4.4.1 M-estimators with known scale |
|
|
99 | (1) |
|
4.4.2 M-estimators with preliminary scale |
|
|
100 | (2) |
|
4.4.3 Simultaneous estimation of regression and scale |
|
|
102 | (1) |
|
4.5 Numerical computing of monotone M-estimators |
|
|
103 | (1) |
|
|
103 | (1) |
|
4.5.2 M-estimators with smooth ψ-function |
|
|
104 | (1) |
|
4.6 BP of monotone regression estimators |
|
|
104 | (2) |
|
4.7 Robust tests for linear hypothesis |
|
|
106 | (3) |
|
4.7.1 Review of the classical theory |
|
|
106 | (2) |
|
4.7.2 Robust tests using M-estimators |
|
|
108 | (1) |
|
4.8 *Regression quantiles |
|
|
109 | (1) |
|
4.9 Appendix: Proofs and complements |
|
|
110 | (3) |
|
|
110 | (1) |
|
4.9.2 Consistency of estimated slopes under asymmetric errors |
|
|
110 | (1) |
|
4.9.3 Maximum FBP of equivariant estimators |
|
|
111 | (1) |
|
4.9.4 The FBP of monotone M-estimators |
|
|
112 | (1) |
|
4.10 Recommendations and software |
|
|
113 | (1) |
|
|
113 | (2) |
5 Linear Regression 2 |
|
115 | (80) |
|
|
115 | (3) |
|
5.2 The linear model with random predictors |
|
|
118 | (1) |
|
5.3 M-estimators with a bounded ρ-function |
|
|
119 | (5) |
|
5.3.1 Properties of M-estimators with a bounded ρ-function |
|
|
120 | (4) |
|
5.4 Estimators based on a robust residual scale |
|
|
124 | (4) |
|
|
124 | (2) |
|
5.4.2 L-estimators of scale and the LTS estimator |
|
|
126 | (1) |
|
|
127 | (1) |
|
|
128 | (5) |
|
5.6 Robust inference and variable selection for M-estimators |
|
|
133 | (5) |
|
5.6.1 Bootstrap robust confidence intervals and tests |
|
|
134 | (1) |
|
|
135 | (3) |
|
|
138 | (12) |
|
5.7.1 Finding local minima |
|
|
140 | (1) |
|
5.7.2 Starting values: the subsampling algorithm |
|
|
141 | (2) |
|
5.7.3 A strategy for faster subsampling-based algorithms |
|
|
143 | (1) |
|
5.7.4 Starting values: the Pena-Yohai estimator |
|
|
144 | (2) |
|
5.7.5 Starting values with numeric and categorical predictors |
|
|
146 | (3) |
|
5.7.6 Comparing initial estimators |
|
|
149 | (1) |
|
5.8 Balancing asymptotic bias and efficiency |
|
|
150 | (5) |
|
5.8.1 "Optimal" redescending M-estimators |
|
|
153 | (2) |
|
5.9 Improving the efficiency of robust regression estimators |
|
|
155 | (9) |
|
5.9.1 Improving efficiency with one-step reweighting |
|
|
155 | (1) |
|
5.9.2 A fully asymptotically efficient one-step procedure |
|
|
156 | (2) |
|
5.9.3 Improving finite-sample efficiency and robustness |
|
|
158 | (6) |
|
5.9.4 Choosing a regression estimator |
|
|
164 | (1) |
|
5.10 Robust regularized regression |
|
|
164 | (8) |
|
|
165 | (3) |
|
|
168 | (3) |
|
5.10.3 Other regularized estimators |
|
|
171 | (1) |
|
|
172 | (4) |
|
5.11.1 Generalized M-estimators |
|
|
172 | (2) |
|
5.11.2 Projection estimators |
|
|
174 | (1) |
|
5.11.3 Constrained M-estimators |
|
|
175 | (1) |
|
5.11.4 Maximum depth estimators |
|
|
175 | (1) |
|
|
176 | (6) |
|
5.12.1 The exact fit property |
|
|
176 | (1) |
|
5.12.2 Heteroskedastic errors |
|
|
177 | (3) |
|
5.12.3 A robust multiple correlation coefficient |
|
|
180 | (2) |
|
5.13 *Appendix: proofs and complements |
|
|
182 | (9) |
|
5.13.1 The BP of monotone M-estimators with random X |
|
|
182 | (1) |
|
|
183 | (1) |
|
5.13.3 Proof of the exact fit property |
|
|
183 | (1) |
|
5.13.4 The BP of S-estimators |
|
|
184 | (2) |
|
5.13.5 Asymptotic bias of M-estimators |
|
|
186 | (1) |
|
5.13.6 Hampel optimality for GM-estimators |
|
|
187 | (1) |
|
5.13.7 Justification of RFPE* |
|
|
188 | (3) |
|
5.14 Recommendations and software |
|
|
191 | (1) |
|
|
191 | (4) |
6 Multivariate Analysis |
|
195 | (76) |
|
|
195 | (5) |
|
6.2 Breakdown and efficiency of multivariate estimators |
|
|
200 | (2) |
|
|
200 | (1) |
|
6.2.2 The multivariate exact fit property |
|
|
201 | (1) |
|
|
201 | (1) |
|
|
202 | (5) |
|
|
205 | (1) |
|
|
205 | (1) |
|
|
206 | (1) |
|
6.4 Estimators based on a robust scale |
|
|
207 | (8) |
|
6.4.1 The minimum volume ellipsoid estimator |
|
|
208 | (1) |
|
|
208 | (2) |
|
|
210 | (1) |
|
6.4.4 S-estimators for high dimension |
|
|
210 | (4) |
|
|
214 | (1) |
|
6.4.6 One-step reweighting |
|
|
215 | (1) |
|
|
215 | (2) |
|
6.6 The Stahel-Donoho estimator |
|
|
217 | (2) |
|
|
219 | (1) |
|
6.8 Numerical computing of multivariate estimators |
|
|
220 | (4) |
|
6.8.1 Monotone M-estimators |
|
|
220 | (1) |
|
6.8.2 Local solutions for S-estimators |
|
|
221 | (1) |
|
6.8.3 Subsampling for estimators based on a robust scale |
|
|
221 | (2) |
|
|
223 | (1) |
|
6.8.5 Computation of S-estimators |
|
|
223 | (1) |
|
|
223 | (1) |
|
6.8.7 The Stahel-Donoho estimator |
|
|
224 | (1) |
|
6.9 Faster robust scatter matrix estimators |
|
|
224 | (5) |
|
6.9.1 Using pairwise robust covariances |
|
|
224 | (4) |
|
6.9.2 The Pena-Prieto procedure |
|
|
228 | (1) |
|
6.10 Choosing a location/scatter estimator |
|
|
229 | (5) |
|
|
230 | (1) |
|
6.10.2 Behavior under contamination |
|
|
231 | (1) |
|
|
232 | (1) |
|
|
233 | (1) |
|
|
233 | (1) |
|
6.11 Robust principal components |
|
|
234 | (6) |
|
6.11.1 Spherical principal components |
|
|
236 | (1) |
|
6.11.2 Robust PCA based on a robust scale |
|
|
237 | (3) |
|
6.12 Estimation of multivariate scatter and location with missing data |
|
|
240 | (2) |
|
|
240 | (1) |
|
6.12.2 GS estimators for missing data |
|
|
241 | (1) |
|
6.13 Robust estimators under the cellwise contamination model |
|
|
242 | (3) |
|
6.14 Regularized robust estimators of the inverse of the covariance matrix |
|
|
245 | (1) |
|
|
246 | (8) |
|
6.15.1 Robust estimation for MLM |
|
|
248 | (1) |
|
6.15.2 Breakdown point of MLM estimators |
|
|
248 | (2) |
|
6.15.3 S-estimators for MLMs |
|
|
250 | (1) |
|
6.15.4 Composite τ-estimators |
|
|
250 | (4) |
|
6.16 *Other estimators of location and scatter |
|
|
254 | (2) |
|
6.16.1 Projection estimators |
|
|
254 | (1) |
|
6.16.2 Constrained M-estimators |
|
|
255 | (1) |
|
6.16.3 Multivariate depth |
|
|
256 | (1) |
|
6.17 Appendix: proofs and complements |
|
|
256 | (12) |
|
6.17.1 Why affine equivariance? |
|
|
256 | (1) |
|
6.17.2 Consistency of equivariant estimators |
|
|
256 | (1) |
|
6.17.3 The estimating equations of the MLE |
|
|
257 | (1) |
|
6.17.4 Asymptotic BP of monotone M-estimators |
|
|
258 | (2) |
|
6.17.5 The estimating equations for S-estimators |
|
|
260 | (1) |
|
6.17.6 Behavior of S-estimators for high p |
|
|
261 | (1) |
|
6.17.7 Calculating the asymptotic covariance matrix of location M-estimators |
|
|
262 | (1) |
|
6.17.8 The exact fit property |
|
|
263 | (1) |
|
6.17.9 Elliptical distributions |
|
|
264 | (1) |
|
6.17.10 Consistency of Gnanadesikan-Kettenring correlations |
|
|
265 | (1) |
|
6.17.11 Spherical principal components |
|
|
266 | (1) |
|
6.17.12 Fixed point estimating equations and computing algorithm for the GS estimator |
|
|
267 | (1) |
|
6.18 Recommendations and software |
|
|
268 | (1) |
|
|
269 | (2) |
7 Generalized Linear Models |
|
271 | (22) |
|
7.1 Binary response regression |
|
|
271 | (4) |
|
7.2 Robust estimators for the logistic model |
|
|
275 | (6) |
|
|
275 | (1) |
|
7.2.2 Redescending M-estimators |
|
|
276 | (5) |
|
7.3 Generalized linear models |
|
|
281 | (3) |
|
7.3.1 Conditionally unbiased bounded influence estimators |
|
|
283 | (1) |
|
7.4 Transformed M-estimators |
|
|
284 | (5) |
|
7.4.1 Definition of transformed M-estimators |
|
|
284 | (2) |
|
7.4.2 Some examples of variance-stabilizing transformations |
|
|
286 | (1) |
|
7.4.3 Other estimators for GLMs |
|
|
286 | (3) |
|
7.5 Recommendations and software |
|
|
289 | (1) |
|
|
290 | (3) |
8 Time Series |
|
293 | (70) |
|
8.1 Time series outliers and their impact |
|
|
294 | (8) |
|
8.1.1 Simple examples of outliers influence |
|
|
296 | (2) |
|
8.1.2 Probability models for time series outliers |
|
|
298 | (3) |
|
|
301 | (1) |
|
8.2 Classical estimators for AR models |
|
|
302 | (6) |
|
8.2.1 The Durbin-Levinson algorithm |
|
|
305 | (2) |
|
8.2.2 Asymptotic distribution of classical estimators |
|
|
307 | (1) |
|
8.3 Classical estimators for ARMA models |
|
|
308 | (2) |
|
8.4 M-estimators of ARMA models |
|
|
310 | (3) |
|
8.4.1 M-estimators and their asymptotic distribution |
|
|
310 | (1) |
|
8.4.2 The behavior of M-estimators in AR processes with additive outliers |
|
|
311 | (1) |
|
8.4.3 The behavior of LS and M-estimators for ARMA processes with infinite innovation variance |
|
|
312 | (1) |
|
8.5 Generalized M-estimators |
|
|
313 | (2) |
|
8.6 Robust AR estimation using robust filters |
|
|
315 | (6) |
|
8.6.1 Naive minimum robust scale autoregression estimators |
|
|
315 | (1) |
|
8.6.2 The robust filter algorithm |
|
|
316 | (2) |
|
8.6.3 Minimum robust scale estimators based on robust filtering |
|
|
318 | (1) |
|
8.6.4 A robust Durbin-Levinson algorithm |
|
|
319 | (1) |
|
8.6.5 Choice of scale for the robust Durbin-Levinson procedure |
|
|
320 | (1) |
|
8.6.6 Robust identification of AR order |
|
|
320 | (1) |
|
8.7 Robust model identification |
|
|
321 | (3) |
|
8.8 Robust ARMA model estimation using robust filters |
|
|
324 | (5) |
|
8.8.1 τ-estimators of ARMA models |
|
|
324 | (2) |
|
8.8.2 Robust filters for ARMA models |
|
|
326 | (2) |
|
8.8.3 Robustly filtered τ-estimators |
|
|
328 | (1) |
|
8.9 ARIMA and SARIMA models |
|
|
329 | (4) |
|
8.10 Detecting time series outliers and level shifts |
|
|
333 | (7) |
|
8.10.1 Classical detection of time series outliers and level shifts |
|
|
334 | (2) |
|
8.10.2 Robust detection of outliers and level shifts for ARIMA models |
|
|
336 | (2) |
|
8.10.3 REGARIMA models: estimation and outlier detection |
|
|
338 | (2) |
|
8.11 Robustness measures for time series |
|
|
340 | (5) |
|
8.11.1 Influence function |
|
|
340 | (2) |
|
|
342 | (1) |
|
|
343 | (1) |
|
8.11.4 Maximum bias curves for the AR(1) model |
|
|
343 | (2) |
|
8.12 Other approaches for ARMA models |
|
|
345 | (2) |
|
8.12.1 Estimators based on robust autocovariances |
|
|
345 | (1) |
|
8.12.2 Estimators based on memory-m prediction residuals |
|
|
346 | (1) |
|
8.13 High-efficiency robust location estimators |
|
|
347 | (1) |
|
8.14 Robust spectral density estimation |
|
|
348 | (8) |
|
8.14.1 Definition of the spectral density |
|
|
348 | (1) |
|
8.14.2 AR spectral density |
|
|
349 | (1) |
|
8.14.3 Classic spectral density estimation methods |
|
|
349 | (1) |
|
|
350 | (1) |
|
8.14.5 Influence of outliers on spectral density estimators |
|
|
351 | (2) |
|
8.14.6 Robust spectral density estimation |
|
|
353 | (1) |
|
8.14.7 Robust time-average spectral density estimator |
|
|
354 | (2) |
|
8.15 Appendix A: Heuristic derivation of the asymptotic distribution of M-estimators for ARMA models |
|
|
356 | (3) |
|
8.16 Appendix B: Robust filter covariance recursions |
|
|
359 | (1) |
|
8.17 Appendix C: ARMA model state-space representation |
|
|
360 | (1) |
|
8.18 Recommendations and software |
|
|
361 | (1) |
|
|
361 | (2) |
9 Numerical Algorithms |
|
363 | (10) |
|
9.1 Regression M-estimators |
|
|
363 | (3) |
|
9.2 Regression S-estimators |
|
|
366 | (1) |
|
|
366 | (1) |
|
|
367 | (2) |
|
9.4.1 Convergence of the fixed-point algorithm |
|
|
367 | (1) |
|
9.4.2 Algorithms for the non-concave case |
|
|
368 | (1) |
|
9.5 Multivariate M-estimators |
|
|
369 | (1) |
|
9.6 Multivariate S-estimators |
|
|
370 | (3) |
|
9.6.1 S-estimators with monotone weights |
|
|
370 | (1) |
|
|
371 | (1) |
|
9.6.3 S-estimators with non-monotone weights |
|
|
371 | (1) |
|
|
372 | (1) |
10 Asymptotic Theory of M-estimators |
|
373 | (28) |
|
10.1 Existence and uniqueness of solutions |
|
|
374 | (2) |
|
10.1.1 Redescending location estimators |
|
|
375 | (1) |
|
|
376 | (1) |
|
10.3 Asymptotic normality |
|
|
377 | (2) |
|
10.4 Convergence of the SC to the IF |
|
|
379 | (2) |
|
10.5 M-estimators of several parameters |
|
|
381 | (3) |
|
10.6 Location M-estimators with preliminary scale |
|
|
384 | (2) |
|
|
386 | (1) |
|
10.8 Optimality of the MLE |
|
|
386 | (2) |
|
10.9 Regression M-estimators: existence and uniqueness |
|
|
388 | (1) |
|
10.10 Regression M-estimators: asymptotic normality |
|
|
389 | (5) |
|
|
389 | (5) |
|
10.10.2 Asymptotic normality: random X |
|
|
394 | (1) |
|
10.11 Regression M estimators: Fisher-consistency |
|
|
394 | (4) |
|
10.11.1 Redescending estimators |
|
|
394 | (2) |
|
10.11.2 Monotone estimators |
|
|
396 | (2) |
|
10.12 Nonexistence of moments of the sample median |
|
|
398 | (1) |
|
|
399 | (2) |
11 Description of Datasets |
|
401 | (6) |
References |
|
407 | (16) |
Index |
|
423 | |