Preface |
|
xiii | |
Notation |
|
xv | |
|
|
1 | (18) |
|
1.1 Sparsity: Occam's razor of modern signal processing? |
|
|
1 | (1) |
|
1.2 Sparse stochastic models: the step beyond Gaussianity |
|
|
2 | (3) |
|
1.3 From splines to stochastic processes, or when Schoenberg meets Levy |
|
|
5 | (11) |
|
1.3.1 Splines and Legos revisited |
|
|
5 | (3) |
|
1.3.2 Higher-degree polynomial splines |
|
|
8 | (1) |
|
1.3.3 Random splines, innovations, and Levy processes |
|
|
9 | (3) |
|
1.3.4 Wavelet analysis of Levy processes and M-term approximations |
|
|
12 | (3) |
|
1.3.5 Levy's wavelet-based synthesis of Brownian motion |
|
|
15 | (1) |
|
1.4 Historical notes: Paul Levy and his legacy |
|
|
16 | (3) |
|
|
19 | (6) |
|
2.1 On the implications of the innovation model |
|
|
20 | (2) |
|
2.1.1 Linear combination of sampled values |
|
|
20 | (1) |
|
|
21 | (1) |
|
2.2 Organization of the book |
|
|
22 | (3) |
|
3 Mathematical context and background |
|
|
25 | (32) |
|
3.1 Some classes of function spaces |
|
|
25 | (7) |
|
3.1.1 About the notation: mathematics vs. engineering |
|
|
28 | (1) |
|
|
28 | (1) |
|
|
29 | (3) |
|
3.2 Dual spaces and adjoint operators |
|
|
32 | (3) |
|
3.2.1 The dual of Lp spaces |
|
|
33 | (1) |
|
3.2.2 The duals of D and S |
|
|
33 | (1) |
|
3.2.3 Distinction between Hermitian and duality products |
|
|
34 | (1) |
|
3.3 Generalized functions |
|
|
35 | (8) |
|
3.3.1 Intuition and definition |
|
|
35 | (1) |
|
3.3.2 Operations on generalized functions |
|
|
36 | (1) |
|
3.3.3 The Fourier transform of generalized functions |
|
|
37 | (1) |
|
|
38 | (1) |
|
3.3.5 Linear shift-invariant operators and convolutions |
|
|
39 | (1) |
|
3.3.6 Convolution operators on Lp(Rd) |
|
|
40 | (3) |
|
|
43 | (4) |
|
3.4.1 Probability measures |
|
|
43 | (1) |
|
3.4.2 Joint probabilities and independence |
|
|
44 | (1) |
|
3.4.3 Characteristic functions in finite dimensions |
|
|
45 | (1) |
|
3.4.4 Characteristic functional in infinite dimensions |
|
|
46 | (1) |
|
3.5 Generalized random processes and fields |
|
|
47 | (7) |
|
3.5.1 Generalized random processes as collections of random variables |
|
|
47 | (2) |
|
3.5.2 Generalized random processes as random generalized functions |
|
|
49 | (1) |
|
3.5.3 Determination of statistics from the characteristic functional |
|
|
49 | (2) |
|
3.5.4 Operations on generalized stochastic processes |
|
|
51 | (1) |
|
3.5.5 Innovation processes |
|
|
52 | (1) |
|
3.5.6 Example: filtered white Gaussian noise |
|
|
53 | (1) |
|
3.6 Bibliographical pointers and historical notes |
|
|
54 | (3) |
|
4 Continuous-domain innovation models |
|
|
57 | (32) |
|
4.1 Introduction: from Gaussian to sparse probability distributions |
|
|
58 | (1) |
|
4.2 Levy exponents and infinitely divisible distributions |
|
|
59 | (12) |
|
4.2.1 Canonical Levy--Khintchine representation |
|
|
60 | (4) |
|
4.2.2 Deciphering the Levy--Khintchine formula |
|
|
64 | (4) |
|
4.2.3 Gaussian vs. sparse categorization |
|
|
68 | (1) |
|
4.2.4 Proofs of Theorems 4.1 and 4.2 |
|
|
69 | (2) |
|
4.3 Finite-dimensional innovation model |
|
|
71 | (2) |
|
4.4 White Levy noises or innovations |
|
|
73 | (11) |
|
4.4.1 Specification of white noise in Schwartz' space S' |
|
|
73 | (3) |
|
4.4.2 Impulsive Poisson noise |
|
|
76 | (2) |
|
4.4.3 Properties of white noise |
|
|
78 | (6) |
|
4.5 Generalized stochastic processes and linear models |
|
|
84 | (3) |
|
|
84 | (1) |
|
4.5.2 Existence and characterization of the solution |
|
|
84 | (3) |
|
4.6 Bibliographical notes |
|
|
87 | (2) |
|
5 Operators and their inverses |
|
|
89 | (24) |
|
5.1 Introductory example: first-order differential equation |
|
|
90 | (2) |
|
5.2 Shift-invariant inverse operators |
|
|
92 | (3) |
|
5.3 Stable differential systems in 1-D |
|
|
95 | (2) |
|
5.3.1 First-order differential operators with stable inverses |
|
|
96 | (1) |
|
5.3.2 Higher-order differential operators with stable inverses |
|
|
96 | (1) |
|
5.4 Unstable Nth-order differential systems |
|
|
97 | (7) |
|
5.4.1 First-order differential operators with unstable shift-invariant inverses |
|
|
97 | (4) |
|
5.4.2 Higher-order differential operators with unstable shift-invariant inverses |
|
|
101 | (1) |
|
5.4.3 Generalized boundary conditions |
|
|
102 | (2) |
|
5.3 Fractional-order operators |
|
|
104 | (5) |
|
5.5.1 Fractional derivatives in one dimension |
|
|
104 | (3) |
|
5.5.2 Fractional Laplacians |
|
|
107 | (1) |
|
|
108 | (1) |
|
5.6 Discrete convolution operators |
|
|
109 | (2) |
|
5.7 Bibliographical notes |
|
|
111 | (2) |
|
|
113 | (37) |
|
6.1 From Legos to wavelets |
|
|
113 | (5) |
|
6.2 Basic concepts and definitions |
|
|
118 | (6) |
|
6.2.1 Spline-admissible operators |
|
|
118 | (2) |
|
6.2.2 Splines and operators |
|
|
120 | (1) |
|
|
121 | (3) |
|
6.2.4 Admissible wavelets |
|
|
124 | (1) |
|
6.3 First-order exponential B-splines and wavelets |
|
|
124 | (3) |
|
6.3.1 B-spline construction |
|
|
125 | (1) |
|
6.3.2 Interpolator in augmented-order spline space |
|
|
126 | (1) |
|
6.3.3 Differential wavelets |
|
|
126 | (1) |
|
6.4 Generalized B-spline basis |
|
|
127 | (15) |
|
6.4.1 B-spline properties |
|
|
128 | (8) |
|
6.4.2 B-spline factorization |
|
|
136 | (1) |
|
6.4.3 Polynomial B-splines |
|
|
137 | (1) |
|
6.4.4 Exponential B-splines |
|
|
138 | (1) |
|
6.4.5 Fractional B-splines |
|
|
139 | (2) |
|
6.4.6 Additional brands of univariate B-splines |
|
|
141 | (1) |
|
6.4.7 Multidimensional B-splines |
|
|
141 | (1) |
|
6.5 Generalized operator-like wavelets |
|
|
142 | (5) |
|
6.5.1 Multiresolution analysis of L2 (Rd) |
|
|
142 | (1) |
|
6.5.2 Multiresolution B-splines and the two-scale relation |
|
|
143 | (1) |
|
6.5.3 Construction of an operator-like wavelet basis |
|
|
144 | (3) |
|
6.6 Bibliographical notes |
|
|
147 | (3) |
|
7 Sparse stochastic processes |
|
|
150 | (41) |
|
7.1 Introductory example: non-Gaussian AR(1) processes |
|
|
150 | (2) |
|
7.2 General abstract characterization |
|
|
152 | (6) |
|
7.3 Non-Gaussian stationary processes |
|
|
158 | (5) |
|
7.3.1 Autocorrelation function and power spectrum |
|
|
159 | (1) |
|
7.3.2 Generalized increment process |
|
|
160 | (1) |
|
7.3.3 Generalized stationary Gaussian processes |
|
|
161 | (1) |
|
|
162 | (1) |
|
7.4 Levy processes and their higher-order extensions |
|
|
163 | (13) |
|
|
163 | (3) |
|
7.4.2 Higher-order extensions of Levy processes |
|
|
166 | (1) |
|
7.4.3 Non-stationary Levy correlations |
|
|
167 | (2) |
|
7.4.4 Removal of long-range dependencies |
|
|
169 | (3) |
|
7.4.5 Examples of sparse processes |
|
|
172 | (3) |
|
|
175 | (1) |
|
7.5 Self-similar processes |
|
|
176 | (11) |
|
7.5.1 Stable fractal processes |
|
|
177 | (3) |
|
7.5.2 Fractional Brownian motion through the looking-glass |
|
|
180 | (5) |
|
7.5.3 Scale-invariant Poisson processes |
|
|
185 | (2) |
|
7.6 Bibliographical notes |
|
|
187 | (4) |
|
|
191 | (32) |
|
8.1 Decoupling of Levy processes: finite differences vs. wavelets |
|
|
191 | (3) |
|
8.2 Extended theoretical framework |
|
|
194 | (3) |
|
8.2.1 Discretization mechanism: sampling vs. projections |
|
|
194 | (1) |
|
8.2.2 Analysis of white noise with non-smooth functions |
|
|
195 | (2) |
|
8.3 Generalized increments for the decoupling of sample values |
|
|
197 | (8) |
|
8.3.1 First-order statistical characterization |
|
|
199 | (1) |
|
8.3.2 Higher-order statistical dependencies |
|
|
200 | (1) |
|
8.3.3 Generalized increments and stochastic difference equations |
|
|
201 | (1) |
|
8.3.4 Discrete whitening filter |
|
|
202 | (1) |
|
8.3.5 Robust localization |
|
|
202 | (3) |
|
|
205 | (5) |
|
8.4.1 Wavelet-domain statistics |
|
|
206 | (2) |
|
8.4.2 Higher-order wavelet dependencies and cumulants |
|
|
208 | (2) |
|
8.5 Optimal representation of Levy and AR(1) processes |
|
|
210 | (12) |
|
8.5.1 Generalized increments and first-order linear prediction |
|
|
211 | (1) |
|
8.5.2 Vector-matrix formulation |
|
|
212 | (1) |
|
8.5.3 Transform-domain statistics |
|
|
212 | (4) |
|
8.5.4 Comparison of orthogonal transforms |
|
|
216 | (6) |
|
8.6 Bibliographical notes |
|
|
222 | (1) |
|
9 Infinite divisibility and transform-domain statistics |
|
|
223 | (25) |
|
9.1 Composition of id laws, spectral mixing, and analysis of white noise |
|
|
224 | (6) |
|
9.2 Class C and unimodality |
|
|
230 | (2) |
|
9.3 Self-decomposable distributions |
|
|
232 | (2) |
|
|
234 | (1) |
|
|
235 | (2) |
|
9.6 Levy exponents and cumulants |
|
|
237 | (2) |
|
|
239 | (3) |
|
|
241 | (1) |
|
|
241 | (1) |
|
9.7.3 Compound-Poisson case |
|
|
241 | (1) |
|
9.7.4 General iterated-convolution interpretation |
|
|
241 | (1) |
|
|
242 | (5) |
|
9.8.1 Scale evolution of the pdf |
|
|
243 | (1) |
|
9.8.2 Scale evolution of the moments |
|
|
244 | (2) |
|
9.8.3 Asymptotic convergence to a Gaussian/stable distribution |
|
|
246 | (1) |
|
9.9 Notes and pointers to the literature |
|
|
247 | (1) |
|
10 Recovery of sparse signals |
|
|
248 | (42) |
|
10.1 Discretization of linear inverse problems |
|
|
249 | (6) |
|
10.1.1 Shift-invariant reconstruction subspace |
|
|
249 | (3) |
|
10.1.2 Finite-dimensional formulation |
|
|
252 | (3) |
|
10.2 MAP estimation and regularization |
|
|
255 | (8) |
|
10.2.1 Potential function |
|
|
256 | (2) |
|
10.2.2 LMMSE/Gaussian solution |
|
|
258 | (1) |
|
10.2.3 Proximal operators |
|
|
259 | (2) |
|
|
261 | (2) |
|
10.3 MAP reconstruction of biomedical images |
|
|
263 | (14) |
|
10.3.1 Scale-invariant image model and common numerical setup |
|
|
264 | (1) |
|
10.3.2 Deconvolution of fluorescence micrographs |
|
|
265 | (4) |
|
10.3.3 Magnetic resonance imaging |
|
|
269 | (3) |
|
|
272 | (4) |
|
|
276 | (1) |
|
10.4 The quest for the minimum-error solution |
|
|
277 | (9) |
|
10.4.1 MMSE estimators for first-order processes |
|
|
278 | (1) |
|
10.4.2 Direct solution by belief propagation |
|
|
279 | (4) |
|
10.4.3 MMSE vs. MAP denoising of Levy processes |
|
|
283 | (3) |
|
10.5 Bibliographical notes |
|
|
286 | (4) |
|
11 Wavelet-domain methods |
|
|
290 | (36) |
|
11.1 Discretization of inverse problems in a wavelet basis |
|
|
291 | (3) |
|
11.1.1 Specification of wavelet-domain MAP estimator |
|
|
292 | (1) |
|
11.1.2 Evolution of the potential function across scales |
|
|
293 | (1) |
|
11.2 Wavelet-based methods for solving linear inverse problems |
|
|
294 | (6) |
|
|
295 | (1) |
|
11.2.2 Iterative shrinkage/thresholding algorithm |
|
|
296 | (1) |
|
11.2.3 Fast iterative shrinkage/thresholding algorithm |
|
|
297 | (1) |
|
11.2.4 Discussion of wavelet-based image reconstruction |
|
|
298 | (2) |
|
11.3 Study of wavelet-domain shrinkage estimators |
|
|
300 | (13) |
|
11.3.1 Pointwise MAP estimators for AWGN |
|
|
301 | (1) |
|
11.3.2 Pointwise MMSE estimators for AWGN |
|
|
301 | (2) |
|
11.3.3 Comparison of shrinkage functions: MAP vs. MMSE |
|
|
303 | (9) |
|
11.3.4 Conclusion on simple wavelet-domain shrinkage estimators |
|
|
312 | (1) |
|
11.4 Improved denoising by consistent cycle spinning |
|
|
313 | (11) |
|
11.4.1 First-order wavelets: design and implementation |
|
|
313 | (2) |
|
11.4.2 From wavelet bases to tight wavelet frames |
|
|
315 | (3) |
|
11.4.3 Iterative MAP denoising |
|
|
318 | (2) |
|
11.4.4 Iterative MMSE denoising |
|
|
320 | (4) |
|
11.5 Bibliographical notes |
|
|
324 | (2) |
|
|
326 | (2) |
|
Appendix A Singular integrals |
|
|
328 | (8) |
|
A.1 Regularization of singular integrals by analytic continuation |
|
|
329 | (2) |
|
A.2 Fourier transform of homogeneous distributions |
|
|
331 | (1) |
|
A.3 Hadamard's finite part |
|
|
332 | (2) |
|
A.4 Some convolution integrals with singular kernels |
|
|
334 | (2) |
|
Appendix B Positive definiteness |
|
|
336 | (8) |
|
B.1 Positive definiteness and Bochner's theorem |
|
|
336 | (3) |
|
B.2 Conditionally positive--definite functions |
|
|
339 | (3) |
|
B.3 Levy--Khintchine formula from the point of view of generalized functions |
|
|
342 | (2) |
|
Appendix C Special functions |
|
|
344 | (3) |
|
C.1 Modified Bessel functions |
|
|
344 | (1) |
|
|
344 | (2) |
|
C.3 Symmetric-alpha-stable distributions |
|
|
346 | (1) |
References |
|
347 | (16) |
Index |
|
363 | |