|
1 Introduction to Computational Intelligence |
|
|
1 | (8) |
|
|
1 | (1) |
|
1.2 Computational Intelligence |
|
|
2 | (2) |
|
1.3 About the Second Edition of This Book |
|
|
4 | (5) |
|
|
5 | (4) |
|
|
|
2 Introduction to Neural Networks |
|
|
9 | (6) |
|
|
9 | (2) |
|
2.2 Biological Background |
|
|
11 | (4) |
|
|
13 | (2) |
|
|
15 | (22) |
|
3.1 Definition and Examples |
|
|
15 | (2) |
|
3.2 Geometric Interpretation |
|
|
17 | (2) |
|
|
19 | (1) |
|
3.4 Networks of Threshold Logic Units |
|
|
20 | (3) |
|
3.5 Training the Parameters |
|
|
23 | (10) |
|
|
33 | (1) |
|
|
33 | (4) |
|
|
34 | (3) |
|
4 General Neural Networks |
|
|
37 | (10) |
|
4.1 Structure of Neural Networks |
|
|
37 | (3) |
|
4.2 Operation of Neural Networks |
|
|
40 | (4) |
|
4.3 Training Neural Networks |
|
|
44 | (3) |
|
|
47 | (46) |
|
5.1 Definition and Examples |
|
|
47 | (6) |
|
5.2 Function Approximation |
|
|
53 | (6) |
|
|
59 | (2) |
|
|
61 | (4) |
|
5.5 Error Backpropagation |
|
|
65 | (3) |
|
5.6 Gradient Descent Examples |
|
|
68 | (4) |
|
5.7 Variants of Gradient Descent |
|
|
72 | (5) |
|
|
72 | (1) |
|
5.7.2 Lifting the Derivative of the Activation Function |
|
|
73 | (1) |
|
|
73 | (1) |
|
5.7.4 Self-Adaptive Error Backpropagation |
|
|
74 | (1) |
|
5.7.5 Resilient Error Backpropagation |
|
|
75 | (1) |
|
|
75 | (2) |
|
|
77 | (1) |
|
5.8 Examples for Some Variants |
|
|
77 | (2) |
|
5.9 Number of Hidden Neurons |
|
|
79 | (3) |
|
|
82 | (7) |
|
5.11 Sensitivity Analysis |
|
|
89 | (4) |
|
|
91 | (2) |
|
6 Radial Basis Function Networks |
|
|
93 | (20) |
|
6.1 Definition and Examples |
|
|
93 | (5) |
|
6.2 Function Approximation |
|
|
98 | (3) |
|
6.3 Initializing the Parameters |
|
|
101 | (6) |
|
6.4 Training the Parameters |
|
|
107 | (4) |
|
|
111 | (2) |
|
|
112 | (1) |
|
|
113 | (18) |
|
7.1 Definition and Examples |
|
|
113 | (3) |
|
7.2 Learning Vector Quantization |
|
|
116 | (7) |
|
7.3 Neighborhood of the Output Neurons |
|
|
123 | (1) |
|
7.3 Neighborhood of the Output Neurons |
|
|
123 | (8) |
|
|
128 | (3) |
|
|
131 | (28) |
|
8.1 Definition and Examples |
|
|
131 | (4) |
|
8.2 Convergence of the Computations |
|
|
135 | (4) |
|
|
139 | (5) |
|
8.4 Solving Optimization Problems |
|
|
144 | (6) |
|
|
150 | (1) |
|
|
151 | (8) |
|
|
156 | (3) |
|
|
159 | (14) |
|
|
159 | (5) |
|
9.2 Representing Differential Equations |
|
|
164 | (2) |
|
9.3 Vectorial Neural Networks |
|
|
166 | (3) |
|
9.4 Error Backpropagation in Time |
|
|
169 | (4) |
|
|
171 | (2) |
|
10 Mathematical Remarks for Neural Networks |
|
|
173 | (10) |
|
10.1 Equations for Straight Lines |
|
|
173 | (2) |
|
|
175 | (4) |
|
10.3 Activation Transformation |
|
|
179 | (4) |
|
|
180 | (3) |
|
Part II Evolutionary Algorithms |
|
|
|
11 Introduction to Evolutionary Algorithms |
|
|
183 | (30) |
|
|
183 | (1) |
|
11.2 Biological Evolution |
|
|
184 | (5) |
|
|
189 | (8) |
|
11.3.1 Optimization Problems |
|
|
189 | (3) |
|
11.3.2 Basic Notions and Concepts |
|
|
192 | (3) |
|
11.3.3 Building Blocks of an Evolutionary Algorithm |
|
|
195 | (2) |
|
11.4 The n-Queens Problem |
|
|
197 | (5) |
|
11.5 Related Optimization Techniques |
|
|
202 | (6) |
|
11.5.1 Gradient Ascent or Descent |
|
|
203 | (2) |
|
|
205 | (1) |
|
11.5.3 Simulated Annealing |
|
|
206 | (1) |
|
11.5.4 Threshold Accepting |
|
|
206 | (1) |
|
11.5.5 Great Deluge Algorithm |
|
|
207 | (1) |
|
11.5.6 Record-to-Record Travel |
|
|
207 | (1) |
|
11.6 The Traveling Salesman Problem |
|
|
208 | (5) |
|
|
211 | (2) |
|
12 Elements of Evolutionary Algorithms |
|
|
213 | (32) |
|
12.1 Encoding of Solution Candidates |
|
|
213 | (7) |
|
|
214 | (2) |
|
|
216 | (2) |
|
12.1.3 Closedness of the Search Space |
|
|
218 | (2) |
|
12.2 Fitness and Selection |
|
|
220 | (12) |
|
12.2.1 Fitness Proportionate Selection |
|
|
221 | (1) |
|
12.2.2 The Dominance Problem |
|
|
222 | (1) |
|
12.2.3 Vanishing Selective Pressure |
|
|
223 | (2) |
|
12.2.4 Adapting the Fitness Function |
|
|
225 | (2) |
|
12.2.5 The Variance Problem |
|
|
227 | (1) |
|
12.2.6 Rank-Based Selection |
|
|
228 | (1) |
|
12.2.7 Tournament Selection |
|
|
229 | (1) |
|
|
230 | (1) |
|
|
231 | (1) |
|
12.2.10 Characterization of Selection Methods |
|
|
231 | (1) |
|
|
232 | (13) |
|
12.3.1 Mutation Operators |
|
|
233 | (3) |
|
12.3.2 Crossover Operators |
|
|
236 | (4) |
|
12.3.3 Multi-parent Operators |
|
|
240 | (1) |
|
12.3.4 Characteristics of Recombination Operators |
|
|
241 | (1) |
|
12.3.5 Interpolating and Extrapolating Recombination |
|
|
242 | (1) |
|
|
243 | (2) |
|
13 Fundamental Evolutionary Algorithms |
|
|
245 | (54) |
|
|
245 | (12) |
|
13.1.1 The Schema Theorem |
|
|
247 | (7) |
|
13.1.2 The Two-Armed Bandit Argument |
|
|
254 | (2) |
|
13.1.3 The Principle of Minimal Alphabets |
|
|
256 | (1) |
|
13.2 Evolution Strategies |
|
|
257 | (11) |
|
|
258 | (1) |
|
13.2.2 Global Variance Adaptation |
|
|
259 | (2) |
|
13.2.3 Local Variance Adaptation |
|
|
261 | (1) |
|
|
262 | (5) |
|
13.2.5 Recombination Operators |
|
|
267 | (1) |
|
|
268 | (12) |
|
|
271 | (2) |
|
|
273 | (2) |
|
13.3.3 Application Examples |
|
|
275 | (4) |
|
13.3.4 The Problem of Introns |
|
|
279 | (1) |
|
|
280 | (1) |
|
13.4 Multi-criteria Optimization |
|
|
280 | (7) |
|
13.4.1 Weighted Combination of Criteria |
|
|
281 | (1) |
|
13.4.2 Pareto-Optimal Solutions |
|
|
281 | (2) |
|
13.4.3 Finding Pareto-Frontiers with Evolutionary Algorithms |
|
|
283 | (4) |
|
13.5 Special Applications and Techniques |
|
|
287 | (12) |
|
13.5.1 Behavioral Simulation |
|
|
288 | (6) |
|
|
294 | (2) |
|
|
296 | (3) |
|
14 Computational Swarm Intelligence |
|
|
299 | (30) |
|
|
299 | (1) |
|
14.2 Basic Principles of Computational Swarm Intelligence |
|
|
300 | (5) |
|
14.2.1 Swarms in Known Environments |
|
|
303 | (1) |
|
14.2.2 Swarms in Unknown Environments |
|
|
304 | (1) |
|
14.3 Particle Swarm Optimization |
|
|
305 | (4) |
|
14.3.1 Influence of the Parameters |
|
|
307 | (1) |
|
|
308 | (1) |
|
|
308 | (1) |
|
14.4 Multi-objective Particle Swarm Optimization |
|
|
309 | (7) |
|
14.4.1 Leader Selection Mechanism |
|
|
310 | (2) |
|
|
312 | (4) |
|
14.5 Many-Objective Particle Swarm Optimization |
|
|
316 | (1) |
|
14.5.1 Ranking Non-dominated Solutions |
|
|
316 | (1) |
|
14.5.2 Distance Based Ranking |
|
|
317 | (1) |
|
14.6 Ant Colony Optimization |
|
|
317 | (12) |
|
|
324 | (5) |
|
|
|
15 Introduction to Fuzzy Sets and Fuzzy Logic |
|
|
329 | (32) |
|
15.1 Natural Languages and Formal Models |
|
|
329 | (1) |
|
|
330 | (2) |
|
15.3 Interpretation of Fuzzy Sets |
|
|
332 | (3) |
|
15.3.1 Gradual Membership is Different from Probability |
|
|
332 | (1) |
|
15.3.2 Fuzzy Sets for Modeling Similarity |
|
|
333 | (1) |
|
15.3.3 Fuzzy Sets for Modeling Preference |
|
|
334 | (1) |
|
15.3.4 Fuzzy Sets for Modeling Possibility |
|
|
334 | (1) |
|
15.3.5 Consistent Interpretations of Fuzzy Sets in Applications |
|
|
334 | (1) |
|
15.4 Representation of Fuzzy Sets |
|
|
335 | (5) |
|
15.4.1 Definition Based on Functions |
|
|
336 | (2) |
|
|
338 | (2) |
|
|
340 | (10) |
|
15.5.1 Propositions and Truth Values |
|
|
340 | (3) |
|
15.5.2 t-Norms and t-Conorms |
|
|
343 | (5) |
|
15.5.3 Aggregation Functions |
|
|
348 | (1) |
|
15.5.4 Basic Assumptions and Problems |
|
|
349 | (1) |
|
15.6 Operations on Fuzzy Sets |
|
|
350 | (5) |
|
|
350 | (2) |
|
|
352 | (1) |
|
|
353 | (1) |
|
15.6.4 Linguistic Modifiers |
|
|
354 | (1) |
|
15.7 Extensions of Fuzzy Set Theory |
|
|
355 | (6) |
|
|
358 | (3) |
|
16 The Extension Principle |
|
|
361 | (8) |
|
16.1 Mappings of Fuzzy Sets |
|
|
361 | (2) |
|
16.2 Mapping of Level Sets |
|
|
363 | (1) |
|
16.3 Cartesian Product and Cylindrical Extension |
|
|
364 | (1) |
|
16.4 Extension Principle for Multivariate Mappings |
|
|
365 | (4) |
|
|
367 | (2) |
|
|
369 | (14) |
|
|
369 | (2) |
|
17.2 Application of Relations and Deduction |
|
|
371 | (2) |
|
17.3 Chains of Deductions |
|
|
373 | (2) |
|
17.4 Simple Fuzzy Relations |
|
|
375 | (4) |
|
17.5 Composition of Fuzzy Relations |
|
|
379 | (2) |
|
17.6 Fuzzy Relational Equations |
|
|
381 | (2) |
|
|
382 | (1) |
|
|
383 | (12) |
|
|
383 | (1) |
|
18.2 Fuzzy Sets and Extensional Hulls |
|
|
384 | (2) |
|
|
386 | (3) |
|
18.4 Fuzzy Sets and Similarity Relations |
|
|
389 | (6) |
|
|
393 | (2) |
|
|
395 | (36) |
|
|
395 | (10) |
|
19.1.1 Remarks on Fuzzy Controller Design |
|
|
399 | (3) |
|
19.1.2 Defuzzification Methods |
|
|
402 | (3) |
|
19.2 Takagi--Sugeno--Kang Controllers |
|
|
405 | (2) |
|
19.3 Mamdani Controller and Similarity Relations |
|
|
407 | (3) |
|
19.3.1 Interpretation of a Controller |
|
|
407 | (2) |
|
19.3.2 Construction of a Controller |
|
|
409 | (1) |
|
19.4 Logic-Based Controllers |
|
|
410 | (2) |
|
19.5 Control Based on Fuzzy Relational Equations |
|
|
412 | (1) |
|
19.6 Hybrid Systems to Tune Fuzzy Controllers |
|
|
413 | (18) |
|
19.6.1 Neuro-Fuzzy Control |
|
|
414 | (9) |
|
19.6.2 Evolutionary Fuzzy Control |
|
|
423 | (5) |
|
|
428 | (3) |
|
|
431 | (28) |
|
20.1 Fuzzy Methods in Data Analysis |
|
|
431 | (1) |
|
|
432 | (13) |
|
|
433 | (1) |
|
20.2.2 Presuppositions and Notation |
|
|
433 | (1) |
|
20.2.3 Classical c-Means Clustering |
|
|
434 | (2) |
|
20.2.4 Fuzzification by Membership Transformation |
|
|
436 | (3) |
|
20.2.5 Fuzzification by Membership Regularization |
|
|
439 | (5) |
|
|
444 | (1) |
|
20.3 Analysis of Imprecise Data Using Random Sets |
|
|
445 | (2) |
|
20.4 Possibility Theory and Generalized Measures |
|
|
447 | (3) |
|
20.5 Fuzzy Random Variables |
|
|
450 | (9) |
|
|
453 | (6) |
|
Part IV Bayes and Markov Networks |
|
|
|
21 Introduction to Bayes Networks |
|
|
459 | (6) |
|
22 Elements of Probability and Graph Theory |
|
|
465 | (28) |
|
|
465 | (9) |
|
22.1.1 Random Variables and Random Vectors |
|
|
468 | (4) |
|
|
472 | (2) |
|
|
474 | (19) |
|
|
474 | (8) |
|
|
482 | (3) |
|
|
485 | (6) |
|
|
491 | (2) |
|
|
493 | (14) |
|
|
504 | (3) |
|
|
507 | (14) |
|
|
511 | (7) |
|
|
511 | (1) |
|
|
512 | (1) |
|
|
512 | (5) |
|
|
517 | (1) |
|
24.2 Other Propagation Algorithms |
|
|
518 | (3) |
|
|
519 | (2) |
|
25 Learning Graphical Models |
|
|
521 | (10) |
|
|
530 | (1) |
|
|
531 | (10) |
|
|
531 | (2) |
|
|
533 | (2) |
|
26.3 A Real-World Application |
|
|
535 | (6) |
|
26.3.1 Knowledge Representation |
|
|
536 | (2) |
|
|
538 | (3) |
|
|
541 | (12) |
|
|
541 | (2) |
|
|
543 | (3) |
|
27.3 Policies and Strategies |
|
|
546 | (1) |
|
27.4 Finding Optimal Strategies |
|
|
546 | (2) |
|
|
548 | (5) |
|
|
551 | (2) |
Index |
|
553 | |