Preface |
|
ix | |
Symbols and Acronyms |
|
xiii | |
|
1 The Linear Data Fitting Problem |
|
|
1 | (24) |
|
1.1 Parameter estimation, data approximation |
|
|
1 | (3) |
|
1.2 Formulation of the data fitting problem |
|
|
4 | (5) |
|
1.3 Maximum likelihood estimation |
|
|
9 | (4) |
|
1.4 The residuals and their properties |
|
|
13 | (6) |
|
|
19 | (6) |
|
2 The Linear Least Squares Problem |
|
|
25 | (22) |
|
2.1 Linear least squares problem formulation |
|
|
25 | (8) |
|
2.2 The QR factorization and its role |
|
|
33 | (6) |
|
2.3 Permuted QR factorization |
|
|
39 | (8) |
|
3 Analysis of Least Squares Problems |
|
|
47 | (18) |
|
|
47 | (3) |
|
3.2 The singular value decomposition |
|
|
50 | (4) |
|
3.3 Generalized singular value decomposition |
|
|
54 | (1) |
|
3.4 Condition number and column scaling |
|
|
55 | (3) |
|
3.5 Perturbation analysis |
|
|
58 | (7) |
|
4 Direct Methods for Full-Rank Problems |
|
|
65 | (26) |
|
|
65 | (3) |
|
|
68 | (2) |
|
|
70 | (10) |
|
4.4 Modifying least squares problems |
|
|
80 | (5) |
|
|
85 | (3) |
|
4.6 Stability and condition number estimation |
|
|
88 | (1) |
|
4.7 Comparison of the methods |
|
|
89 | (2) |
|
5 Direct Methods for Rank-Deficient Problems |
|
|
91 | (14) |
|
|
92 | (1) |
|
5.2 Peters-Wilkinson LU factorization |
|
|
93 | (1) |
|
5.3 QR factorization with column permutations |
|
|
94 | (4) |
|
5.4 UTV and VSV decompositions |
|
|
98 | (1) |
|
|
99 | (2) |
|
|
101 | (4) |
|
6 Methods for Large-Scale Problems |
|
|
105 | (16) |
|
6.1 Iterative versus direct methods |
|
|
105 | (2) |
|
6.2 Classical stationary methods |
|
|
107 | (1) |
|
6.3 Non-stationary methods, Krylov methods |
|
|
108 | (6) |
|
6.4 Practicalities: preconditioning and stopping criteria |
|
|
114 | (3) |
|
|
117 | (4) |
|
7 Additional Topics in Least Squares |
|
|
121 | (26) |
|
7.1 Constrained linear least squares problems |
|
|
121 | (10) |
|
7.2 Missing data problems |
|
|
131 | (5) |
|
7.3 Total least squares (TLS) |
|
|
136 | (7) |
|
|
143 | (1) |
|
|
144 | (3) |
|
8 Nonlinear Least Squares Problems |
|
|
147 | (16) |
|
|
147 | (3) |
|
8.2 Unconstrained problems |
|
|
150 | (6) |
|
8.3 Optimality conditions for constrained problems |
|
|
156 | (2) |
|
8.4 Separable nonlinear least squares problems |
|
|
158 | (2) |
|
8.5 Multiobjective optimization |
|
|
160 | (3) |
|
9 Algorithms for Solving Nonlinear LSQ Problems |
|
|
163 | (28) |
|
|
164 | (2) |
|
9.2 The Gauss-Newton method |
|
|
166 | (4) |
|
9.3 The Levenberg-Marquardt method |
|
|
170 | (6) |
|
9.4 Additional considerations and software |
|
|
176 | (2) |
|
9.5 Iteratively reweighted LSQ algorithms for robust data fitting problems |
|
|
178 | (3) |
|
9.6 Variable projection algorithm |
|
|
181 | (5) |
|
9.7 Block methods for large-scale problems |
|
|
186 | (5) |
|
10 Ill-Conditioned Problems |
|
|
191 | (12) |
|
|
191 | (1) |
|
10.2 Regularization methods |
|
|
192 | (3) |
|
10.3 Parameter selection techniques |
|
|
195 | (3) |
|
10.4 Extensions of Tikhonov regularization |
|
|
198 | (3) |
|
10.5 Ill-conditioned NLLSQ problems |
|
|
201 | (2) |
|
11 Linear Least Squares Applications |
|
|
203 | (28) |
|
11.1 Splines in approximation |
|
|
203 | (9) |
|
11.2 Global temperatures data fitting |
|
|
212 | (9) |
|
11.3 Geological surface modeling |
|
|
221 | (10) |
|
12 Nonlinear Least Squares Applications |
|
|
231 | (32) |
|
12.1 Neural networks training |
|
|
231 | (7) |
|
12.2 Response surfaces, surrogates or proxies |
|
|
238 | (3) |
|
12.3 Optimal design of a supersonic aircraft |
|
|
241 | (7) |
|
|
248 | (3) |
|
12.5 Piezoelectric crystal identification |
|
|
251 | (7) |
|
12.6 Travel time inversion of seismic data |
|
|
258 | (5) |
|
Appendix A Sensitivity Analysis |
|
|
263 | (4) |
|
A.1 Floating-point arithmetic |
|
|
263 | (1) |
|
A.2 Stability, conditioning and accuracy |
|
|
264 | (3) |
|
Appendix B Linear Algebra Background |
|
|
267 | (4) |
|
|
267 | (1) |
|
|
268 | (1) |
|
|
269 | (1) |
|
B.4 Some additional matrix properties |
|
|
270 | (1) |
|
Appendix C Advanced Calculus Background |
|
|
271 | (4) |
|
|
271 | (1) |
|
C.2 Multivariate calculus |
|
|
272 | (3) |
|
|
275 | (6) |
|
|
275 | (5) |
|
|
280 | (1) |
References |
|
281 | (20) |
Index |
|
301 | |