Abbreviations |
|
xi | |
Preface |
|
xiii | |
Notation |
|
xvii | |
Acknowledgements |
|
xix | |
1 Introduction |
|
|
1.1 The importance of optimization |
|
|
1 | |
|
1.2 Inspiration from biological phenomena |
|
|
2 | |
|
1.3 Optimization of a simple behaviour for an autonomous robot |
|
|
5 | |
2 Classical optimization |
|
|
|
9 | |
|
2.1.1 Local and global optima |
|
|
9 | |
|
2.1.2 Objective functions |
|
|
10 | |
|
|
11 | |
|
2.2 Taxonomy of optimization problems |
|
|
11 | |
|
2.3 Continuous optimization |
|
|
12 | |
|
2.3.1 Properties of local optima |
|
|
12 | |
|
2.3.2 Global optima of convex functions |
|
|
14 | |
|
2.3.2.1 Convex sets and functions |
|
|
14 | |
|
2.3.2.2 Optima of convex functions |
|
|
16 | |
|
2.4 Algorithms for continuous optimization |
|
|
16 | |
|
2.4.1 Unconstrained optimization |
|
|
17 | |
|
|
17 | |
|
|
19 | |
|
|
21 | |
|
2.4.2 Constrained optimization |
|
|
24 | |
|
2.4.2.1 The method of Lagrange multipliers |
|
|
25 | |
|
2.4.2.2 An analytical method for optimization under inequality constraints |
|
|
29 | |
|
|
30 | |
|
2.5 Limitations of classical optimization |
|
|
33 | |
|
|
34 | |
3 Evolutionary algorithms |
|
|
3.1 Biological background |
|
|
35 | |
|
|
40 | |
|
3.2.1 Components of genetic algorithms |
|
|
46 | |
|
|
46 | |
|
|
48 | |
|
|
52 | |
|
|
53 | |
|
|
55 | |
|
|
55 | |
|
3.2.1.7 A standard genetic algorithm |
|
|
55 | |
|
3.2.1.8 Parameter selection |
|
|
56 | |
|
3.2.2 Properties of genetic algorithms |
|
|
59 | |
|
3.2.2.1 The schema theorem |
|
|
59 | |
|
|
60 | |
|
3.2.2.3 Premature convergence |
|
|
67 | |
|
3.3 Linear genetic programming |
|
|
72 | |
|
3.3.1 Registers and instructions |
|
|
73 | |
|
|
74 | |
|
3.3.3 Evolutionary operators in LGP |
|
|
75 | |
|
3.4 Interactive evolutionary computation |
|
|
78 | |
|
3.5 Biological vs. artificial evolution |
|
|
82 | |
|
|
83 | |
|
3.6.1 Optimization of truck braking systems |
|
|
83 | |
|
3.6.2 Determination of orbits of interacting galaxies |
|
|
86 | |
|
3.6.3 Prediction of cancer survival |
|
|
92 | |
|
|
96 | |
4 Ant colony optimization |
|
|
4.1 Biological background |
|
|
100 | |
|
|
104 | |
|
|
105 | |
|
|
109 | |
|
|
111 | |
|
4.3.1 Single-machine scheduling |
|
|
112 | |
|
4.3.2 Co-operative transport using autonomous robots |
|
|
114 | |
|
|
116 | |
5 Particle swarm optimization |
|
|
5.1 Biological background |
|
|
117 | |
|
5.1.1 A model of swarming |
|
|
118 | |
|
|
120 | |
|
|
124 | |
|
5.3.1 Best-in-current-swarm vs. best-ever |
|
|
125 | |
|
5.3.2 Neighbourhood topologies |
|
|
125 | |
|
5.3.3 Maintaining coherence |
|
|
126 | |
|
|
127 | |
|
|
128 | |
|
|
129 | |
|
5.4.1 Variable truncation |
|
|
129 | |
|
|
130 | |
|
|
130 | |
|
5.5.1 Optimization of neural networks |
|
|
131 | |
|
5.5.1.1 Prediction of pollutant levels |
|
|
133 | |
|
5.5.1.2 Prediction of elephant migration patterns |
|
|
134 | |
|
5.5.2 Optimization of cancer chemotherapy |
|
|
136 | |
|
|
137 | |
6 Performance comparison |
|
|
6.1 Unconstrained function optimization |
|
|
140 | |
|
6.2 Constrained function optimization |
|
|
143 | |
|
6.3 Optimization of feedforward neural networks |
|
|
145 | |
|
6.4 The travelling salesman problem |
|
|
146 | |
A Neural networks |
|
|
A.1 Biological background |
|
|
151 | |
|
A.1.1 Neurons and synapses |
|
|
151 | |
|
A.1.2 Biological neural networks |
|
|
152 | |
|
|
153 | |
|
|
154 | |
|
A.1.3.2 Habituation and sensitization |
|
|
154 | |
|
A.2 Artificial neural networks |
|
|
156 | |
|
|
158 | |
|
A.2.2 Feedforward neural networks and backpropagation |
|
|
159 | |
|
|
159 | |
|
A.2.2.2 Limitations of single-layer networks |
|
|
161 | |
|
|
161 | |
|
A.2.3 Recurrent neural networks |
|
|
169 | |
|
|
171 | |
|
|
172 | |
B Analysis of optimization algorithms |
|
|
B.1 Classical optimization |
|
|
173 | |
|
B.1.1 Global minima of convex functions |
|
|
173 | |
|
B.1.2 Properties of the gradient |
|
|
174 | |
|
|
174 | |
|
|
174 | |
|
B.2.2 The genetic algorithm as a Markov process |
|
|
176 | |
|
B.2.2.1 Number of populations of a given size |
|
|
176 | |
|
B.2.3 Infinite population models |
|
|
177 | |
|
B.2.3.1 Representing the crossover operator |
|
|
177 | |
|
B.2.3.2 Initial distribution of chromosomes |
|
|
178 | |
|
B.2.3.3 Elementary properties of binomial coefficients |
|
|
178 | |
|
B.2.3.4 The mutation operator for functions of unitation |
|
|
179 | |
|
B.2.3.5 Selection and mutations for the Onemax problem |
|
|
180 | |
|
B.2.4 Expected runtime of a simple GA |
|
|
181 | |
|
B.2.5 Estimating optimal mutation rates |
|
|
182 | |
|
B.3 Ant colony optimization |
|
|
183 | |
|
B.3.1 Pheromone limits in MMAS |
|
|
183 | |
|
|
184 | |
|
B.3.3 Runtime analysis for a simple ACO algorithm |
|
|
184 | |
|
B.4 Particle swarm optimization |
|
|
188 | |
|
B.4.1 Particle trajectories in PSO |
|
|
188 | |
C Data analysis |
|
|
C.1 Hypothesis evaluation |
|
|
193 | |
|
|
200 | |
D Benchmark functions |
|
|
D.1 The Goldstein—Price function |
|
|
206 | |
|
D.2 The Rosenbrock function |
|
|
206 | |
|
D.3 The Sine square function |
|
|
207 | |
|
D.4 The Colville function |
|
|
208 | |
|
D.5 A multidimensional benchmark function |
|
|
208 | |
Answers to selected exercises |
|
209 | |
Bibliography |
|
211 | |
Index |
|
215 | |