|
1 Automated Machine Learning and Bayesian Optimization |
|
|
1 | (18) |
|
1.1 Automated Machine Learning |
|
|
1 | (7) |
|
|
1 | (2) |
|
|
3 | (1) |
|
1.1.3 Hyperparameter Optimization |
|
|
4 | (2) |
|
1.1.4 Combined Algorithm Selection and Hyperparameter Optimization |
|
|
6 | (1) |
|
1.1.5 Why Hyperparameter Optimization Is Important? |
|
|
6 | (2) |
|
1.2 The Basic Structure of Bayesian Optimization |
|
|
8 | (6) |
|
1.2.1 Sequential Model-Based Optimization |
|
|
8 | (2) |
|
|
10 | (2) |
|
1.2.3 Acquisition Function |
|
|
12 | (2) |
|
1.3 Automated Machine Learning for Predictive Analytics |
|
|
14 | (3) |
|
|
17 | (2) |
|
2 From Global Optimization to Optimal Learning |
|
|
19 | (18) |
|
2.1 A Priori Analysis of Global Optimization Strategies |
|
|
20 | (1) |
|
2.2 Lipschitz Global Optimization (LGO) |
|
|
21 | (3) |
|
|
24 | (4) |
|
2.3.1 General Properties of Uniform Sampling |
|
|
26 | (1) |
|
|
26 | (1) |
|
|
26 | (2) |
|
2.4 Bandits, Active Learning and Bayesian Optimization |
|
|
28 | (4) |
|
|
32 | (5) |
|
|
37 | (20) |
|
|
37 | (10) |
|
3.1.1 Gaussian Processes Regression |
|
|
37 | (3) |
|
3.1.2 Kernel: The Data Geometry of Bayesian Optimization |
|
|
40 | (3) |
|
3.1.3 Embedding Derivative Observations in the Gaussian Process |
|
|
43 | (3) |
|
3.1.4 Numerical Instability |
|
|
46 | (1) |
|
|
47 | (3) |
|
|
50 | (5) |
|
|
51 | (2) |
|
3.3.2 Neural Networks: Feedforward, Deep and Bayesian |
|
|
53 | (2) |
|
|
55 | (2) |
|
4 The Acquisition Function |
|
|
57 | (16) |
|
4.1 Traditional Acquisition Functions |
|
|
57 | (5) |
|
4.1.1 Probability of Improvement |
|
|
57 | (1) |
|
4.1.2 Expected Improvement |
|
|
58 | (2) |
|
4.1.3 Upper/Lower Confidence Bound |
|
|
60 | (2) |
|
4.2 New Acquisition Functions |
|
|
62 | (8) |
|
4.2.1 Scaled Expected Improvement |
|
|
62 | (1) |
|
4.2.2 Portfolio Allocation |
|
|
62 | (1) |
|
|
63 | (2) |
|
4.2.4 Entropy-Based Acquisition Functions |
|
|
65 | (1) |
|
|
66 | (2) |
|
|
68 | (1) |
|
|
69 | (1) |
|
4.3 Optimizing the Acquisition Function |
|
|
70 | (1) |
|
|
71 | (2) |
|
5 Exotic Bayesian Optimization |
|
|
73 | (24) |
|
5.1 Constrained Global Optimization |
|
|
73 | (3) |
|
5.2 Support Vector Machine---Constrained Bayesian Optimization |
|
|
76 | (10) |
|
5.3 Safe Bayesian Optimization |
|
|
86 | (4) |
|
5.4 Parallel Bayesian Optimization |
|
|
90 | (1) |
|
5.5 Multi-objective Bayesian Optimization |
|
|
91 | (2) |
|
5.6 Multi-source and Multi-fidelity Bayesian Optimization |
|
|
93 | (1) |
|
|
94 | (3) |
|
|
97 | (14) |
|
|
97 | (4) |
|
6.2 Bayesian Optimization as a Service |
|
|
101 | (1) |
|
6.3 Bayesian Optimization-Based Services for Hyperparameters Optimization |
|
|
102 | (1) |
|
6.4 Test Functions and Generators |
|
|
103 | (4) |
|
6.4.1 Survey and Site/Repository of Test Functions |
|
|
103 | (4) |
|
6.4.2 Test Functions Generators |
|
|
107 | (1) |
|
6.5 Non-Bayesian Global Optimization Software |
|
|
107 | (1) |
|
|
108 | (3) |
|
|
111 | |
|
7.1 Overview of Applications |
|
|
111 | (5) |
|
|
116 | (6) |
|
7.2.1 Leakage Localization |
|
|
116 | (2) |
|
7.2.2 Pump Scheduling Optimization |
|
|
118 | (4) |
|
|
122 | |