Muutke küpsiste eelistusi

Bayesian Optimization [Kõva köide]

5.00/5 (2 hinnangut Goodreads-ist)
(Washington University in St Louis)
  • Formaat: Hardback, 358 pages, kõrgus x laius x paksus: 261x209x22 mm, kaal: 1020 g, Worked examples or Exercises
  • Ilmumisaeg: 09-Feb-2023
  • Kirjastus: Cambridge University Press
  • ISBN-10: 110842578X
  • ISBN-13: 9781108425780
Teised raamatud teemal:
Bayesian OptimizationSuurem pilt
  • Formaat: Hardback, 358 pages, kõrgus x laius x paksus: 261x209x22 mm, kaal: 1020 g, Worked examples or Exercises
  • Ilmumisaeg: 09-Feb-2023
  • Kirjastus: Cambridge University Press
  • ISBN-10: 110842578X
  • ISBN-13: 9781108425780
Teised raamatud teemal:
Püsilink: https://www.kriso.ee/db/9781108425780.html
Märksõnad:
Bayesian optimization is a methodology for optimizing expensive objective functions that has proven success in the sciences, engineering, and beyond. This timely text provides a self-contained and comprehensive introduction to the subject, starting from scratch and carefully developing all the key ideas along the way. This bottom-up approach illuminates unifying themes in the design of Bayesian optimization algorithms and builds a solid theoretical foundation for approaching novel situations. The core of the book is divided into three main parts, covering theoretical and practical aspects of Gaussian process modeling, the Bayesian approach to sequential decision making, and the realization and computation of practical and effective optimization policies. Following this foundational material, the book provides an overview of theoretical convergence results, a survey of notable extensions, a comprehensive history of Bayesian optimization, and an extensive annotated bibliography of applications.

Muu info

A comprehensive introduction to Bayesian optimization that starts from scratch and carefully develops all the key ideas along the way.
Preface ix
Notation xiii
1 Introduction
1(14)
1.1 Formalization of Optimization
2(3)
1.2 The Bayesian Approach
5(10)
2 Gaussian Processes
15(30)
2.1 Definition and Basic Properties
16(2)
2.2 Inference with Exact and Noisy Observations
18(8)
2.3 Overview of Remainder of
Chapter
26(1)
2.4 Joint Gaussian Processes
26(2)
2.5 Continuity
28(2)
2.6 Differentiability
30(3)
2.7 Existence and Uniqueness of Global Maxima
33(2)
2.8 Inference with Non-Gaussian Observations and Constraints
35(6)
2.9 Summary of Major Ideas
41(4)
3 Modeling With Gaussian Processes
45(22)
3.1 The Prior Mean Function
46(3)
3.2 The Prior Covariance Function
49(2)
3.3 Notable Covariance Functions
51(3)
3.4 Modifying and Combining Covariance Functions
54(7)
3.5 Modeling Functions on High-Dimensional Domains
61(3)
3.6 Summary of Major Ideas
64(3)
4 Model Assessment, Selection, And Averaging
67(20)
4.1 Models and Model Structures
68(2)
4.2 Bayesian Inference over Parametric Model Spaces
70(3)
4.3 Model Selection via Posterior Maximization
73(1)
4.4 Model Averaging
74(4)
4.5 Multiple Model Structures
78(3)
4.6 Automating Model Structure Search
81(3)
4.7 Summary of Major Ideas
84(3)
5 Decision Theory For Optimization
87(22)
5.1 Introduction to Bayesian Decision Theory
89(2)
5.2 Sequential Decisions with a Fixed Budget
91(8)
5.3 Cost and Approximation of the Optimal Policy
99(4)
5.4 Cost-Aware Optimization and Termination as a Decision
103(3)
3.5 Summary of Major Ideas
106(3)
6 Utility Functions For Optimization
109(14)
6.1 Expected Utility of Terminal Recommendation
109(5)
6.2 Cumulative Reward
114(1)
6.3 Information Gain
115(1)
6.4 Dependence on Model of Objective Function
116(1)
6.5 Comparison of Utility Functions
117(2)
6.6 Summary of Major Ideas
119(4)
7 Common Bayesian Optimization Policies
123(34)
7.1 Example Optimization Scenario
124(1)
7.2 Decision-Theoretic Policies
124(3)
7.3 Expected Improvement
127(2)
7.4 Knowledge Gradient
129(2)
7.5 Probability of Improvement
131(4)
7.6 Mutual Information and Entropy Search
135(6)
7.7 Multi-Armed Bandits and Optimization
141(4)
7.8 Maximizing a Statistical Upper Bound
145(3)
7.9 Thompson Sampling
148(2)
7.10 Other Ideas in Policy Construction
150(6)
7.11 Summary of Major Ideas
156(1)
8 Computing Policies With Gaussian Processes
157(44)
8.1 Notation for Objective Function Model
157(1)
8.2 Expected Improvement
158(9)
8.3 Probability of Improvement
167(3)
8.4 Upper Confidence Bound
170(1)
8.5 Approximate Computation for One-Step Lookahead
171(1)
8.6 Knowledge Gradient
172(4)
8.7 Thompson Sampling
176(4)
8.8 Mutual Information with x*
180(7)
8.9 Mutual Information with f*
187(5)
8.10 Averaging over a Space of Gaussian Processes
192(4)
8.11 Alternative Models: Bayesian Neural Networks, etc.
196(4)
8.12 Summary of Major Ideas
200(1)
9 Implementation
201(12)
9.1 Gaussian Process Inference, Scaling, and Approximation
201(6)
9.2 Optimizing Acquisition Functions
207(3)
9.3 Starting and Stopping Optimization
210(2)
9.4 Summary of Major Ideas
212(1)
10 Theoretical Analysis
213(32)
10.1 Regret
213(2)
10.2 Useful Function Spaces for Studying Convergence
215(5)
10.3 Relevant Properties of Covariance Functions
220(4)
10.4 Bayesian Regret with Observation Noise
224(8)
10.5 Worst-Case Regret with Observation Noise
232(5)
10.6 The Exact Observation Case
237(4)
10.7 The Effect of Unknown Hyperparameters
241(2)
10.8 Summary of Major Ideas
243(2)
11 Extensions And Related Settings
245(42)
11.1 Unknown Observation Costs
245(4)
11.2 Constrained Optimization and Unknown Constraints
249(3)
11.3 Synchronous Batch Observations
252(10)
11.4 Asynchronous Observation with Pending Experiments
262(1)
11.5 Multifidelity Optimization
263(3)
11.6 Multitask Optimization
266(3)
11.7 Multiobjective Optimization
269(7)
11.8 Gradient Observations
276(1)
11.9 Stochastic and Robust Optimization
277(4)
11.10 Incremental Optimization of Sequential Procedures
281(1)
11.11 Non-Gaussian Observation Models and Active Search
282(3)
11.12 Local Optimization
285(2)
12 A Brief History Of Bayesian Optimization
287(8)
12.1 Historical Precursors and Optimal Design
287(1)
12.2 Sequential Analysis and Bayesian Experimental Design
287(2)
12.3 The Rise of Bayesian Optimization
289(1)
12.4 Later Rediscovery and Development
290(2)
12.5 Multi-Armed Bandits to Infinite-Armed Bandits
292(2)
12.6 What's Next?
294(1)
A The Gaussian Distribution 295(6)
B Methods For Approximate Bayesian Inference 301(6)
C Gradients 307(6)
D Annotated Bibliography Of Applications 313(18)
References 331(22)
Index 353
Roman Garnett is Associate Professor at Washington University in St. Louis. He has been a leader in the Bayesian optimization community since 2011, when he co-founded a long-running workshop on the subject at the NeurIPS conference. His research focus is developing Bayesian methods including Bayesian optimization for automating scientific discovery, an effort supported by an NSF CAREER award.