Preface |
|
xiii | |
|
|
|
1 Basic properties of vectors and matrices |
|
|
3 | (28) |
|
|
3 | (1) |
|
|
3 | (1) |
|
3 Matrices: addition and multiplication |
|
|
4 | (2) |
|
4 The transpose of a matrix |
|
|
6 | (1) |
|
|
6 | (1) |
|
6 Linear forms and quadratic forms |
|
|
7 | (2) |
|
|
9 | (1) |
|
|
10 | (1) |
|
|
10 | (1) |
|
|
11 | (1) |
|
|
12 | (2) |
|
|
14 | (1) |
|
13 Eigenvalues and eigenvectors |
|
|
14 | (3) |
|
14 Schur's decomposition theorem |
|
|
17 | (1) |
|
15 The Jordan decomposition |
|
|
18 | (2) |
|
16 The singular-value decomposition |
|
|
20 | (1) |
|
17 Further results concerning eigenvalues |
|
|
20 | (3) |
|
18 Positive (semi)definite matrices |
|
|
23 | (2) |
|
19 Three further results for positive definite matrices |
|
|
25 | (1) |
|
|
26 | (1) |
|
21 Symmetric matrix functions |
|
|
27 | (1) |
|
|
28 | (2) |
|
|
30 | (1) |
|
2 Kronecker products, vec operator, and Moore-Penrose inverse |
|
|
31 | (16) |
|
|
31 | (1) |
|
|
31 | (2) |
|
3 Eigenvalues of a Kronecker product |
|
|
33 | (1) |
|
|
34 | (2) |
|
5 The Moore-Penrose (MP) inverse |
|
|
36 | (1) |
|
6 Existence and uniqueness of the MP inverse |
|
|
37 | (1) |
|
7 Some properties of the MP inverse |
|
|
38 | (1) |
|
|
39 | (2) |
|
9 The solution of linear equation systems |
|
|
41 | (2) |
|
|
43 | (2) |
|
|
45 | (2) |
|
3 Miscellaneous matrix results |
|
|
47 | (26) |
|
|
47 | (1) |
|
|
47 | (2) |
|
|
49 | (2) |
|
|
51 | (1) |
|
5 The matrix equation AX = 0 |
|
|
51 | (1) |
|
|
52 | (2) |
|
7 The commutation matrix Kmn |
|
|
54 | (2) |
|
8 The duplication matrix Dn |
|
|
56 | (2) |
|
9 Relationship between Dn+1 and Dn, I |
|
|
58 | (1) |
|
10 Relationship between Dn+1 and Dn, II |
|
|
59 | (1) |
|
11 Conditions for a quadratic form to be positive (negative) subject to linear constraints |
|
|
60 | (3) |
|
12 Necessary and sufficient conditions for r(A : B) = r(A) + r(B) |
|
|
63 | (2) |
|
13 The bordered Gramian matrix |
|
|
65 | (2) |
|
14 The equations X1A + X2B' = G1, X1B = G2 |
|
|
67 | (2) |
|
|
69 | (1) |
|
|
70 | (3) |
|
Part Two Differentials: the theory |
|
|
|
4 Mathematical preliminaries |
|
|
73 | (14) |
|
|
73 | (1) |
|
2 Interior points and accumulation points |
|
|
73 | (2) |
|
|
75 | (2) |
|
4 The Bolzano-Weierstrass theorem |
|
|
77 | (1) |
|
|
78 | (1) |
|
6 The limit of a function |
|
|
79 | (1) |
|
7 Continuous functions and compactness |
|
|
80 | (1) |
|
|
81 | (2) |
|
9 Convex and concave functions |
|
|
83 | (3) |
|
|
86 | (1) |
|
5 Differentials and differentiability |
|
|
87 | (24) |
|
|
87 | (1) |
|
|
88 | (2) |
|
3 Differentiability and linear approximation |
|
|
90 | (1) |
|
4 The differential of a vector function |
|
|
91 | (2) |
|
5 Uniqueness of the differential |
|
|
93 | (1) |
|
6 Continuity of differentiate functions |
|
|
94 | (1) |
|
|
95 | (1) |
|
8 The first identification theorem |
|
|
96 | (1) |
|
9 Existence of the differential, I |
|
|
97 | (2) |
|
10 Existence of the differential, II |
|
|
99 | (1) |
|
11 Continuous differentiability |
|
|
100 | (1) |
|
|
100 | (2) |
|
|
102 | (1) |
|
14 The mean-value theorem for real-valued functions |
|
|
103 | (1) |
|
15 Differentiable matrix functions |
|
|
104 | (2) |
|
16 Some remarks on notation |
|
|
106 | (2) |
|
17 Complex differentiation |
|
|
108 | (2) |
|
|
110 | (1) |
|
|
110 | (1) |
|
6 The second differential |
|
|
111 | (18) |
|
|
111 | (1) |
|
2 Second-order partial derivatives |
|
|
111 | (1) |
|
|
112 | (1) |
|
4 Twice differentiability and second-order approximation, I |
|
|
113 | (1) |
|
5 Definition of twice differentiability |
|
|
114 | (1) |
|
6 The second differential |
|
|
115 | (2) |
|
7 Symmetry of the Hessian matrix |
|
|
117 | (2) |
|
8 The second identification theorem |
|
|
119 | (1) |
|
9 Twice differentiability and second-order approximation, II |
|
|
119 | (2) |
|
10 Chain rule for Hessian matrices |
|
|
121 | (2) |
|
11 The analog for second differentials |
|
|
123 | (1) |
|
12 Taylor's theorem for real-valued functions |
|
|
124 | (1) |
|
13 Higher-order differentials |
|
|
125 | (1) |
|
14 Real analytic functions |
|
|
125 | (1) |
|
15 Twice differentiable matrix functions |
|
|
126 | (1) |
|
|
127 | (2) |
|
|
129 | (34) |
|
|
129 | (1) |
|
2 Unconstrained optimization |
|
|
130 | (1) |
|
3 The existence of absolute extrema |
|
|
131 | (1) |
|
4 Necessary conditions for a local minimum |
|
|
132 | (2) |
|
5 Sufficient conditions for a local minimum: first-derivative test |
|
|
134 | (2) |
|
6 Sufficient conditions for a local minimum: second-derivative test |
|
|
136 | (2) |
|
7 Characterization of differentiable convex functions |
|
|
138 | (3) |
|
8 Characterization of twice differentiable convex functions |
|
|
141 | (1) |
|
9 Sufficient conditions for an absolute minimum |
|
|
142 | (1) |
|
10 Monotonic transformations |
|
|
143 | (1) |
|
11 Optimization subject to constraints |
|
|
144 | (1) |
|
12 Necessary conditions for a local minimum under constraints |
|
|
145 | (4) |
|
13 Sufficient conditions for a local minimum under constraints |
|
|
149 | (5) |
|
14 Sufficient conditions for an absolute minimum under constraints |
|
|
154 | (1) |
|
15 A note on constraints in matrix form |
|
|
155 | (1) |
|
16 Economic interpretation of Lagrange multipliers |
|
|
155 | (2) |
|
Appendix: the implicit function theorem |
|
|
157 | (2) |
|
|
159 | (4) |
|
Part Three Differentials: the practice |
|
|
|
8 Some important differentials |
|
|
163 | (28) |
|
|
163 | (1) |
|
2 Fundamental rules of differential calculus |
|
|
163 | (2) |
|
3 The differential of a determinant |
|
|
165 | (3) |
|
4 The differential of an inverse |
|
|
168 | (1) |
|
5 Differential of the Moore-Penrose inverse |
|
|
169 | (3) |
|
6 The differential of the adjoint matrix |
|
|
172 | (2) |
|
7 On differentiating eigenvalues and eigenvectors |
|
|
174 | (2) |
|
8 The continuity of eigenprojections |
|
|
176 | (4) |
|
9 The differential of eigenvalues and eigenvectors: symmetric case |
|
|
180 | (3) |
|
10 Two alternative expressions for dλ |
|
|
183 | (2) |
|
11 Second differential of the eigenvalue function |
|
|
185 | (1) |
|
|
186 | (3) |
|
|
189 | (2) |
|
9 First-order differentials and Jacobian matrices |
|
|
191 | (20) |
|
|
191 | (1) |
|
|
192 | (1) |
|
|
192 | (2) |
|
|
194 | (2) |
|
5 Identification of Jacobian matrices |
|
|
196 | (1) |
|
6 The first identification table |
|
|
197 | (1) |
|
7 Partitioning of the derivative |
|
|
197 | (1) |
|
8 Scalar functions of a scalar |
|
|
198 | (1) |
|
9 Scalar functions of a vector |
|
|
198 | (1) |
|
10 Scalar functions of a matrix, I: trace |
|
|
199 | (2) |
|
11 Scalar functions of a matrix, II: determinant |
|
|
201 | (1) |
|
12 Scalar functions of a matrix, III: eigenvalue |
|
|
202 | (1) |
|
13 Two examples of vector functions |
|
|
203 | (1) |
|
|
204 | (2) |
|
|
206 | (2) |
|
|
208 | (1) |
|
17 Jacobians of transformations |
|
|
209 | (1) |
|
|
210 | (1) |
|
10 Second-order differentials and Hessian matrices |
|
|
211 | (14) |
|
|
211 | (1) |
|
2 The second identification table |
|
|
211 | (1) |
|
3 Linear and quadratic forms |
|
|
212 | (1) |
|
|
213 | (1) |
|
5 The determinant function |
|
|
214 | (1) |
|
6 The eigenvalue function |
|
|
215 | (1) |
|
|
215 | (2) |
|
|
217 | (1) |
|
9 The eigenvector function |
|
|
218 | (1) |
|
10 Hessian of matrix functions, I |
|
|
219 | (1) |
|
11 Hessian of matrix functions, II |
|
|
219 | (1) |
|
|
220 | (5) |
|
|
|
|
225 | (48) |
|
|
225 | (1) |
|
2 The Cauchy-Schwarz inequality |
|
|
226 | (1) |
|
3 Matrix analogs of the Cauchy-Schwarz inequality |
|
|
227 | (1) |
|
4 The theorem of the arithmetic and geometric means |
|
|
228 | (2) |
|
|
230 | (2) |
|
6 Concavity of Ai and convexity of An |
|
|
232 | (1) |
|
7 Variational description of eigenvalues |
|
|
232 | (2) |
|
8 Fischer's min-max theorem |
|
|
234 | (2) |
|
9 Monotonicity of the eigenvalues |
|
|
236 | (1) |
|
10 The Poincare separation theorem |
|
|
236 | (1) |
|
11 Two corollaries of Poincare's theorem |
|
|
237 | (1) |
|
12 Further consequences of the Poincare theorem |
|
|
238 | (1) |
|
13 Multiplicative version |
|
|
239 | (2) |
|
14 The maximum of a bilinear form |
|
|
241 | (1) |
|
|
242 | (1) |
|
16 An interlude: Karamata's inequality |
|
|
242 | (2) |
|
17 Karamata's inequality and eigenvalues |
|
|
244 | (1) |
|
18 An inequality concerning positive semidefinite matrices |
|
|
245 | (1) |
|
19 A representation theorem for (Σ api)1/P |
|
|
246 | (1) |
|
20 A representation theorem for (trAp)1/p |
|
|
247 | (1) |
|
|
248 | (2) |
|
|
250 | (1) |
|
23 Minkowski's inequality |
|
|
251 | (2) |
|
24 Quasilinear representation of |A|1/n |
|
|
253 | (2) |
|
25 Minkowski's determinant theorem |
|
|
255 | (1) |
|
26 Weighted means of order p |
|
|
256 | (2) |
|
27 Schlomilch's inequality |
|
|
258 | (1) |
|
28 Curvature properties of Mp(x, a) |
|
|
259 | (1) |
|
|
260 | (1) |
|
30 Generalized least squares |
|
|
261 | (1) |
|
31 Restricted least squares |
|
|
262 | (2) |
|
32 Restricted least squares: matrix version |
|
|
264 | (1) |
|
|
265 | (4) |
|
|
269 | (4) |
|
Part Five The linear model |
|
|
|
12 Statistical preliminaries |
|
|
273 | (12) |
|
|
273 | (1) |
|
2 The cumulative distribution function |
|
|
273 | (1) |
|
3 The joint density function |
|
|
274 | (1) |
|
|
274 | (1) |
|
5 Variance and covariance |
|
|
275 | (2) |
|
6 Independence of two random variables |
|
|
277 | (2) |
|
7 Independence of n random variables |
|
|
279 | (1) |
|
|
279 | (1) |
|
9 The one-dimensional normal distribution |
|
|
279 | (1) |
|
10 The multivariate normal distribution |
|
|
280 | (2) |
|
|
282 | (1) |
|
|
282 | (1) |
|
|
283 | (2) |
|
13 The linear regression model |
|
|
285 | (36) |
|
|
285 | (1) |
|
2 Affine minimum-trace unbiased estimation |
|
|
286 | (1) |
|
3 The Gauss-Markov theorem |
|
|
287 | (3) |
|
4 The method of least squares |
|
|
290 | (1) |
|
|
291 | (2) |
|
|
293 | (2) |
|
|
295 | (1) |
|
8 Linear constraints: the case M(R') ⊂ M(X') |
|
|
296 | (4) |
|
9 Linear constraints: the general case |
|
|
300 | (2) |
|
10 Linear constraints: the case M(R') ⊂ M(X') = {0} |
|
|
302 | (2) |
|
11 A singular variance matrix: the case M(X) ⊂ M(V) |
|
|
304 | (1) |
|
12 A singular variance matrix: the case r(X'V+X) = r(X) |
|
|
305 | (2) |
|
13 A singular variance matrix: the general case, I |
|
|
307 | (1) |
|
14 Explicit and implicit linear constraints |
|
|
307 | (3) |
|
15 The general linear model, I |
|
|
310 | (1) |
|
16 A singular variance matrix: the general case, II |
|
|
311 | (3) |
|
17 The general linear model, II |
|
|
314 | (1) |
|
18 Generalized least squares |
|
|
315 | (1) |
|
19 Restricted least squares |
|
|
316 | (2) |
|
|
318 | (1) |
|
|
319 | (2) |
|
14 Further topics in the linear model |
|
|
321 | (26) |
|
|
321 | (1) |
|
2 Best quadratic unbiased estimation of σ2 |
|
|
322 | (1) |
|
3 The best quadratic and positive unbiased estimator of σ2 |
|
|
322 | (2) |
|
4 The best quadratic unbiased estimator of σ2 |
|
|
324 | (2) |
|
5 Best quadratic invariant estimation of σ2 |
|
|
326 | (1) |
|
6 The best quadratic and positive invariant estimator of σ2 |
|
|
327 | (2) |
|
7 The best quadratic invariant estimator of σ2 |
|
|
329 | (1) |
|
8 Best quadratic, unbiased estimation: multivariate normal case |
|
|
330 | (2) |
|
9 Bounds for the bias of the least-squares estimator of σ2, I |
|
|
332 | (1) |
|
10 Bounds for the bias of the Least-squares estimator of σ2, II |
|
|
333 | (2) |
|
11 The prediction of disturbances |
|
|
335 | (1) |
|
12 Best linear unbiased predictors with scalar variance matrix |
|
|
336 | (2) |
|
13 Best linear unbiased predictors with fixed variance matrix, I |
|
|
338 | (2) |
|
14 Best linear unbiased predictors with fixed variance matrix, II |
|
|
340 | (1) |
|
15 Local sensitivity of the posterior mean |
|
|
341 | (1) |
|
16 Local sensitivity of the posterior precision |
|
|
342 | (2) |
|
|
344 | (3) |
|
Part Six Applications to maximum likelihood estimation |
|
|
|
15 Maximum likelihood estimation |
|
|
347 | (20) |
|
|
347 | (1) |
|
2 The method of maximum likelihood (ML) |
|
|
347 | (1) |
|
3 ML estimation of the multivariate normal distribution |
|
|
348 | (2) |
|
4 Symmetry: implicit versus explicit treatment |
|
|
350 | (1) |
|
5 The treatment of positive definiteness |
|
|
351 | (1) |
|
|
352 | (2) |
|
7 ML estimation of the multivariate normal distribution: distinct means |
|
|
354 | (1) |
|
8 The multivariate linear regression model |
|
|
354 | (3) |
|
9 The errors-in-variables model |
|
|
357 | (2) |
|
10 The nonlinear regression model with normal errors |
|
|
359 | (2) |
|
11 Special case: functional independence of mean and variance parameters |
|
|
361 | (1) |
|
12 Generalization of Theorem 15.6 |
|
|
362 | (2) |
|
|
364 | (1) |
|
|
365 | (2) |
|
16 Simultaneous equations |
|
|
367 | (22) |
|
|
367 | (1) |
|
2 The simultaneous equations model |
|
|
367 | (2) |
|
3 The identification problem |
|
|
369 | (2) |
|
4 Identification with linear constraints on B and V only |
|
|
371 | (1) |
|
5 Identification with linear constraints on B, T, and D |
|
|
371 | (2) |
|
|
373 | (1) |
|
7 FIML: the information matrix (general case) |
|
|
374 | (2) |
|
8 FIML: asymptotic variance matrix (special case) |
|
|
376 | (2) |
|
9 LIML: first-order conditions |
|
|
378 | (3) |
|
10 LIML: information matrix |
|
|
381 | (2) |
|
11 LIML: asymptotic variance matrix |
|
|
383 | (5) |
|
|
388 | (1) |
|
17 Topics in psychometrics |
|
|
389 | (34) |
|
|
389 | (1) |
|
2 Population principal components |
|
|
390 | (1) |
|
3 Optimality of principal components |
|
|
391 | (1) |
|
|
392 | (1) |
|
5 Sample principal components |
|
|
393 | (2) |
|
6 Optimality of sample principal components |
|
|
395 | (1) |
|
7 One-mode component analysis |
|
|
395 | (3) |
|
8 One-mode component analysis and sample principal components |
|
|
398 | (1) |
|
9 Two-mode component analysis |
|
|
399 | (1) |
|
10 Multimode component analysis |
|
|
400 | (4) |
|
|
404 | (3) |
|
|
407 | (1) |
|
13 A Newton-Raphson routine |
|
|
408 | (4) |
|
14 Kaiser's varimax method |
|
|
412 | (2) |
|
15 Canonical correlations and variates in the population |
|
|
414 | (3) |
|
16 Correspondence analysis |
|
|
417 | (1) |
|
17 Linear discriminant analysis |
|
|
418 | (1) |
|
|
419 | (4) |
|
|
|
18 Matrix calculus: the essentials |
|
|
423 | (26) |
|
|
423 | (1) |
|
|
424 | (2) |
|
|
426 | (3) |
|
|
429 | (2) |
|
|
431 | (1) |
|
|
432 | (2) |
|
7 Interlude on linear and quadratic forms |
|
|
434 | (1) |
|
8 The second differential |
|
|
434 | (2) |
|
9 Chain rule for second differentials |
|
|
436 | (2) |
|
|
438 | (1) |
|
11 The Kronecker product and vec operator |
|
|
439 | (2) |
|
|
441 | (1) |
|
13 The commutation matrix |
|
|
442 | (1) |
|
14 From second differential to Hessian |
|
|
443 | (1) |
|
15 Symmetry and the duplication matrix |
|
|
444 | (1) |
|
|
445 | (3) |
|
|
448 | (1) |
Bibliography |
|
449 | (18) |
Index of symbols |
|
467 | (4) |
Subject index |
|
471 | |