|
|
xiii | |
Preface |
|
xix | |
Acknowledgments |
|
xxi | |
Introduction |
|
xxiii | |
|
PART I FOUNDATIONS OF OPTIMIZATION AND ALGORITHMS |
|
|
|
1 A Brief History of Optimization |
|
|
3 | (12) |
|
|
4 | (2) |
|
|
6 | (1) |
|
1.3 Heuristics and Metaheuristics |
|
|
7 | (8) |
|
|
10 | (5) |
|
2 Engineering Optimization |
|
|
15 | (14) |
|
|
15 | (2) |
|
|
17 | (2) |
|
2.3 Optimization Algorithms |
|
|
19 | (3) |
|
|
22 | (1) |
|
|
22 | (2) |
|
|
24 | (1) |
|
2.7 No Free Lunch Theorems |
|
|
25 | (4) |
|
|
27 | (2) |
|
3 Mathematical Foundations |
|
|
29 | (32) |
|
3.1 Upper and Lower Bounds |
|
|
29 | (2) |
|
|
31 | (4) |
|
|
35 | (5) |
|
3.3.1 Continuity and Smoothness |
|
|
35 | (1) |
|
|
36 | (2) |
|
3.3.3 Optimality Criteria |
|
|
38 | (2) |
|
3.4 Vector and Matrix Norms |
|
|
40 | (3) |
|
3.5 Eigenvalues and Definiteness |
|
|
43 | (5) |
|
|
43 | (3) |
|
|
46 | (2) |
|
3.6 Linear and Affine Functions |
|
|
48 | (3) |
|
|
48 | (1) |
|
|
49 | (1) |
|
|
49 | (2) |
|
3.7 Gradient and Hessian Matrices |
|
|
51 | (2) |
|
|
51 | (1) |
|
|
51 | (1) |
|
3.7.3 Function approximations |
|
|
52 | (1) |
|
3.7.4 Optimality of multivariate functions |
|
|
52 | (1) |
|
|
53 | (8) |
|
|
53 | (2) |
|
|
55 | (3) |
|
|
58 | (3) |
|
4 Classic Optimization Methods I |
|
|
61 | (24) |
|
4.1 Unconstrained Optimization |
|
|
61 | (1) |
|
4.2 Gradient-Based Methods |
|
|
62 | (6) |
|
|
62 | (1) |
|
4.2.2 Steepest Descent Method |
|
|
63 | (2) |
|
|
65 | (1) |
|
4.2.4 Conjugate Gradient Method |
|
|
66 | (2) |
|
4.3 Constrained Optimization |
|
|
68 | (1) |
|
|
68 | (2) |
|
|
70 | (6) |
|
|
70 | (2) |
|
|
72 | (4) |
|
4.6 Nonlinear Optimization |
|
|
76 | (1) |
|
|
76 | (1) |
|
|
76 | (4) |
|
4.9 Karush-Kuhn-Tucker Conditions |
|
|
80 | (5) |
|
|
83 | (2) |
|
5 Classic Optimization Methods II |
|
|
85 | (10) |
|
|
85 | (1) |
|
|
86 | (2) |
|
|
86 | (1) |
|
5.2.2 Nelder-Mead Downhill Simplex |
|
|
86 | (2) |
|
|
88 | (3) |
|
5.4 Sequential Quadratic Programming |
|
|
91 | (4) |
|
5.4.1 Quadratic Programming |
|
|
91 | (1) |
|
5.4.2 Sequential Quadratic Programming |
|
|
91 | (2) |
|
|
93 | (2) |
|
|
95 | (16) |
|
|
95 | (2) |
|
6.2 Convex Optimization Examples |
|
|
97 | (2) |
|
6.3 Equality Constrained Optimization |
|
|
99 | (2) |
|
|
101 | (3) |
|
6.5 Interior-Point Methods |
|
|
104 | (1) |
|
6.6 Stochastic and Robust Optimization |
|
|
105 | (6) |
|
|
107 | (4) |
|
|
111 | (22) |
|
7.1 Euler-Lagrange Equation |
|
|
111 | (9) |
|
|
111 | (3) |
|
7.1.2 Euler-Lagrange Equation |
|
|
114 | (6) |
|
7.2 Variations with Constraints |
|
|
120 | (4) |
|
7.3 Variations for Multiple Variables |
|
|
124 | (1) |
|
|
125 | (8) |
|
|
126 | (1) |
|
7.4.2 Pontryagin's Principle |
|
|
127 | (2) |
|
|
129 | (1) |
|
7.4.4 Stochastic Optimal Control |
|
|
130 | (1) |
|
|
131 | (2) |
|
8 Random Number Generators |
|
|
133 | (10) |
|
8.1 Linear Congruential Algorithms |
|
|
133 | (1) |
|
|
134 | (2) |
|
|
136 | (4) |
|
8.4 Metropolis Algorithms |
|
|
140 | (3) |
|
|
141 | (2) |
|
|
143 | (10) |
|
|
143 | (3) |
|
9.2 Monte Carlo Integration |
|
|
146 | (3) |
|
9.3 Importance of Sampling |
|
|
149 | (4) |
|
|
151 | (2) |
|
10 Random Walk and Markov Chain |
|
|
153 | (20) |
|
|
153 | (2) |
|
|
155 | (4) |
|
|
156 | (2) |
|
10.2.2 Random Walk in Higher Dimensions |
|
|
158 | (1) |
|
|
159 | (2) |
|
|
161 | (1) |
|
10.5 Markov Chain Monte Carlo |
|
|
161 | (6) |
|
10.5.1 Metropolis-Hastings Algorithms |
|
|
164 | (2) |
|
|
166 | (1) |
|
10.6 Markov Chain and Optimisation |
|
|
167 | (6) |
|
|
169 | (4) |
|
PART II METAHEURISTIC ALGORITHMS |
|
|
|
|
173 | (8) |
|
|
173 | (1) |
|
|
174 | (3) |
|
|
174 | (2) |
|
11.2.2 Choice of Parameters |
|
|
176 | (1) |
|
|
177 | (4) |
|
|
179 | (2) |
|
|
181 | (8) |
|
12.1 Annealing and Probability |
|
|
181 | (1) |
|
12.2 Choice of Parameters |
|
|
182 | (2) |
|
|
184 | (1) |
|
|
184 | (5) |
|
|
186 | (3) |
|
|
189 | (8) |
|
|
189 | (1) |
|
13.2 Ant Colony Optimization |
|
|
190 | (2) |
|
13.3 Double Bridge Problem |
|
|
192 | (1) |
|
13.4 Virtual Ant Algorithm |
|
|
193 | (4) |
|
|
195 | (2) |
|
|
197 | (6) |
|
14.1 Behavior of Honey Bees |
|
|
197 | (1) |
|
|
198 | (3) |
|
14.2.1 Honey Bee Algorithm |
|
|
198 | (2) |
|
14.2.2 Virtual Bee Algorithm |
|
|
200 | (1) |
|
14.2.3 Artificial Bee Colony Optimization |
|
|
201 | (1) |
|
|
201 | (2) |
|
|
202 | (1) |
|
15 Particle Swarm Optimization |
|
|
203 | (10) |
|
|
203 | (1) |
|
|
204 | (1) |
|
|
205 | (2) |
|
|
207 | (2) |
|
15.4.1 Multimodal Functions |
|
|
207 | (1) |
|
|
208 | (1) |
|
|
209 | (4) |
|
|
210 | (3) |
|
|
213 | (8) |
|
16.1 Music-Based Algorithms |
|
|
213 | (2) |
|
|
215 | (2) |
|
|
217 | (4) |
|
|
218 | (3) |
|
|
221 | (12) |
|
17.1 Behaviour of Fireflies |
|
|
221 | (1) |
|
17.2 Firefly-Inspired Algorithm |
|
|
222 | (4) |
|
|
222 | (1) |
|
17.2.2 Light Intensity and Attractiveness |
|
|
222 | (3) |
|
17.2.3 Scaling and Global Optima |
|
|
225 | (1) |
|
|
225 | (1) |
|
|
226 | (7) |
|
17.3.1 Multiple Global Optima |
|
|
226 | (1) |
|
17.3.2 Multimodal Functions |
|
|
227 | (1) |
|
|
228 | (1) |
|
|
229 | (4) |
|
|
|
18 Multiobjective Optimization |
|
|
233 | (14) |
|
|
233 | (4) |
|
|
237 | (2) |
|
|
239 | (2) |
|
18.4 Metaheuristic Search |
|
|
241 | (1) |
|
|
242 | (5) |
|
|
244 | (3) |
|
19 Engineering Applications |
|
|
247 | (14) |
|
|
247 | (1) |
|
|
248 | (1) |
|
|
249 | (3) |
|
19.4 Optimization of Eigenvalues and Frequencies |
|
|
252 | (4) |
|
19.5 Inverse Finite Element Analysis |
|
|
256 | (5) |
|
|
258 | (3) |
|
|
261 | (72) |
|
Appendix A Test Problems in Optimization |
|
|
261 | (6) |
|
Appendix B Matlab® Programs |
|
|
267 | (16) |
|
|
267 | (3) |
|
|
270 | (2) |
|
B.3 Particle Swarm Optimization |
|
|
272 | (1) |
|
|
273 | (2) |
|
|
275 | (3) |
|
B.6 Large Sparse Linear Systems |
|
|
278 | (1) |
|
B.7 Nonlinear Optimization |
|
|
279 | (1) |
|
|
279 | (2) |
|
|
281 | (2) |
|
|
283 | (22) |
|
Appendix D Problem Solutions |
|
|
305 | (28) |
References |
|
333 | (10) |
Index |
|
343 | |