Contributors |
|
xiii | |
About the Editors |
|
xv | |
Section Editors |
|
xvii | |
Introduction |
|
xxi | |
|
SECTION 1 VISUAL INFORMATION PROCESSING |
|
|
|
Chapter 1 Multiview Video: Acquisition, Processing, Compression, and Virtual View Rendering |
|
|
3 | (72) |
|
|
|
|
|
3 | (9) |
|
1.1.1 Multiview Video and 3D Graphic Representation Formats for VR |
|
|
4 | (1) |
|
1.1.2 Super-Multiview Video for 3D Light Field Displays |
|
|
5 | (1) |
|
1.1.3 DIBR Smooth View Interpolation |
|
|
5 | (2) |
|
1.1.4 Basic Principles of DIBR |
|
|
7 | (2) |
|
1.1.5 DIBR vs. Point Clouds |
|
|
9 | (3) |
|
1.1.6 DIBR, Multiview Video, and MPEG Standardization |
|
|
12 | (1) |
|
1.2 Multiview Video Acquisition |
|
|
12 | (17) |
|
1.2.1 Multiview Fundamentals |
|
|
12 | (4) |
|
1.2.2 Depth in Stereo and Multiview Video |
|
|
16 | (4) |
|
|
20 | (3) |
|
1.2.4 Acquisition System Examples |
|
|
23 | (6) |
|
1.3 Multiview Video Preprocessing |
|
|
29 | (9) |
|
1.3.1 Geometrical Parameters |
|
|
29 | (6) |
|
|
35 | (3) |
|
|
38 | (10) |
|
1.4.1 Local Stereo Matching |
|
|
39 | (2) |
|
1.4.2 Global Stereo Matching |
|
|
41 | (3) |
|
1.4.3 Multicamera Depth Estimation |
|
|
44 | (4) |
|
1.5 View Synthesis and Virtual Navigation |
|
|
48 | (6) |
|
|
49 | (2) |
|
|
51 | (1) |
|
|
52 | (1) |
|
1.5.4 View Synthesis Reference Software |
|
|
53 | (1) |
|
|
54 | (11) |
|
|
54 | (4) |
|
1.6.2 Monoscopic Video Coding and Simulcast Coding of Multiview Video |
|
|
58 | (4) |
|
1.6.3 Multiview Video Coding |
|
|
62 | (1) |
|
|
63 | (2) |
|
|
65 | (10) |
|
|
67 | (1) |
|
|
67 | (1) |
|
|
68 | (6) |
|
|
74 | (1) |
|
Chapter 2 Plenoptic Imaging: Representation and Processing |
|
|
75 | (38) |
|
|
|
|
|
75 | (3) |
|
2.2 Light Representation: The Plenoptic Function Paradigm |
|
|
78 | (4) |
|
2.3 Empowering the Plenoptic Function: Example Use Cases |
|
|
82 | (6) |
|
2.3.1 Light Field Communication |
|
|
83 | (1) |
|
2.3.2 Light Field Editing |
|
|
84 | (1) |
|
|
85 | (1) |
|
2.3.4 Interactive All-Reality |
|
|
86 | (2) |
|
2.4 Plenoptic Acquisition and Representation Models |
|
|
88 | (7) |
|
|
89 | (2) |
|
|
91 | (3) |
|
|
94 | (1) |
|
2.5 Plenoptic Data Coding |
|
|
95 | (3) |
|
2.6 Plenoptic Data Rendering |
|
|
98 | (3) |
|
2.6.1 Rendering Textured Meshes and Point Clouds |
|
|
98 | (1) |
|
2.6.2 Interpolating a Light Field in a Microlens and/or Discrete Camera Array |
|
|
98 | (1) |
|
2.6.3 View Synthesis in MVV Plus Depth |
|
|
99 | (1) |
|
2.6.4 Refocusing With Microlens Light Field |
|
|
99 | (2) |
|
2.7 Plenoptic Representations Relationships |
|
|
101 | (1) |
|
2.8 Related Standardization Initiatives |
|
|
102 | (3) |
|
|
103 | (1) |
|
|
104 | (1) |
|
2.9 Future Trends and Challenges |
|
|
105 | (8) |
|
|
107 | (1) |
|
|
107 | (1) |
|
|
108 | (3) |
|
|
111 | (2) |
|
Chapter 3 Visual Attention, Visual Salience, and Perceived Interest in Multimedia Applications |
|
|
113 | (50) |
|
|
|
|
113 | (2) |
|
3.1.1 Visual Attention in the Field of Multimedia: A Rising Story |
|
|
113 | (1) |
|
3.1.2 From Vision Science to Engineering: Concepts Mash Up and Confusion |
|
|
114 | (1) |
|
3.2 Classification of Attention Mechanisms |
|
|
115 | (5) |
|
3.2.1 Overt and Covert Attention |
|
|
115 | (1) |
|
3.2.2 Types of Overt Visual Attention Mechanisms |
|
|
116 | (4) |
|
3.3 Computational Models of Visual Attention |
|
|
120 | (10) |
|
3.3.1 Top-Down Computational Attention Models |
|
|
121 | (2) |
|
3.3.2 Information-Theory and Decision-Theory Models |
|
|
123 | (2) |
|
3.3.3 Spatio-Temporal Computational Models |
|
|
125 | (1) |
|
3.3.4 Graph-Based Methods |
|
|
126 | (2) |
|
3.3.5 Scan-Path (Saccadic) Models |
|
|
128 | (2) |
|
3.4 Acquiring Ground Truth Visual Attention Data for Model Verification |
|
|
130 | (17) |
|
|
130 | (8) |
|
3.4.2 Processing the Eye-Tracking Data |
|
|
138 | (3) |
|
3.4.3 Testing the Computational Models |
|
|
141 | (6) |
|
3.5 Applications of Visual Attention |
|
|
147 | (16) |
|
|
147 | (2) |
|
3.5.2 Visual Attention in Multimedia Delivery |
|
|
149 | (1) |
|
3.5.3 Applications in Medicine |
|
|
150 | (1) |
|
3.5.4 Visual Attention and Immersive Media: A Rising Love Story |
|
|
151 | (1) |
|
|
152 | (11) |
|
Chapter 4 Emerging Science of QoE in Multimedia Applications Concepts, Experimental Guidelines, and Validation of Models |
|
|
163 | (50) |
|
|
|
4.1 QoE Definition and Influencing Factors |
|
|
164 | (2) |
|
4.1.1 Factors Influencing QoE |
|
|
164 | (2) |
|
|
166 | (25) |
|
4.2.1 Including System Influence Factors in QoE Measurement |
|
|
168 | (1) |
|
4.2.2 Including Context Influence Factors in QoE Measurement |
|
|
169 | (1) |
|
4.2.3 Including Human Influence Factors in QoE Measurement |
|
|
169 | (1) |
|
4.2.4 Multidimensional Perceptual Scales for QoE Measurement |
|
|
170 | (2) |
|
4.2.5 Direct Scaling Methods |
|
|
172 | (5) |
|
4.2.6 Processing of Results of Direct Scaling Methods |
|
|
177 | (1) |
|
4.2.7 Indirect Scaling Methods |
|
|
178 | (4) |
|
4.2.8 Processing of Results of Indirect Screening Methods |
|
|
182 | (5) |
|
4.2.9 Influence Factor's Significance Calculation |
|
|
187 | (4) |
|
4.3 Performance Evaluation of Objective QoE Estimators |
|
|
191 | (14) |
|
4.3.1 Pearson's Linear Correlation Coefficient |
|
|
192 | (1) |
|
4.3.2 Root-Mean-Squared Error |
|
|
193 | (1) |
|
4.3.3 Epsilon-Insensitive Root-Mean-Squared Error |
|
|
194 | (1) |
|
|
194 | (1) |
|
4.3.5 Spearman's Rank Order Correlation Coefficient |
|
|
195 | (1) |
|
4.3.6 Kendall's Rank Order Correlation Coefficient |
|
|
195 | (1) |
|
4.3.7 Resolving Power Measures |
|
|
196 | (4) |
|
4.3.8 ROC-Based Performance Evaluation |
|
|
200 | (3) |
|
4.3.9 Compensation for Multiple Comparisons |
|
|
203 | (2) |
|
|
205 | (8) |
|
|
205 | (8) |
|
SECTION 2 COMPUTATIONAL IMAGING AND 3D ANALYSIS |
|
|
|
Chapter 5 Computational Photography |
|
|
213 | (24) |
|
|
Aswin C. Sankaranarayanan |
|
|
|
|
213 | (1) |
|
5.2 Breaking Precepts Underlying Photography |
|
|
214 | (5) |
|
5.2.1 Sensor Resolution ≠ Image Resolution |
|
|
214 | (2) |
|
5.2.2 Space-Time Bandwidth Product Can Be Greater Than the ADC Rate |
|
|
216 | (1) |
|
5.2.3 Depth of Field Can Be Changed Independent of Exposure Time |
|
|
217 | (2) |
|
5.3 Cameras With Novel Form Factors and Capabilities |
|
|
219 | (6) |
|
|
219 | (3) |
|
|
222 | (2) |
|
5.3.3 Subdiffraction Limited Microscopy |
|
|
224 | (1) |
|
5.4 Solving Inverse Problems |
|
|
225 | (7) |
|
5.4.1 Time-of-Flight-Based Range Imaging |
|
|
225 | (4) |
|
5.4.2 Direct-Global Separation |
|
|
229 | (2) |
|
|
231 | (1) |
|
|
232 | (5) |
|
|
233 | (4) |
|
Chapter 6 Face Detection With a 3D Model |
|
|
237 | (24) |
|
|
|
|
|
237 | (4) |
|
|
239 | (2) |
|
6.2 Face Detection Using a 3D Model |
|
|
241 | (6) |
|
|
242 | (1) |
|
6.2.2 Inference Algorithm |
|
|
242 | (1) |
|
6.2.3 Detecting Face Keypoints |
|
|
243 | (1) |
|
6.2.4 Generating 3D Pose Candidates |
|
|
244 | (1) |
|
6.2.5 Generating Face Candidates |
|
|
245 | (1) |
|
6.2.6 Scoring the Face Candidates |
|
|
245 | (2) |
|
6.2.7 Nonmaximal Suppression |
|
|
247 | (1) |
|
6.3 Parameter Sensitive Model |
|
|
247 | (3) |
|
6.3.1 Training the Parameter Sensitive Model |
|
|
248 | (2) |
|
|
250 | (1) |
|
6.4.1 Fitting a Rigid Projection Transformation |
|
|
250 | (1) |
|
6.4.2 Learning a 3D Model From 2D Annotations |
|
|
250 | (1) |
|
|
251 | (6) |
|
6.5.1 Evaluation of Face Candidates |
|
|
252 | (2) |
|
6.5.2 Face Detection Results |
|
|
254 | (3) |
|
6.6 Conclusions and Future Trends |
|
|
257 | (4) |
|
|
258 | (3) |
|
Chapter 7 A Survey on Nonrigid 3D Shape Analysis |
|
|
261 | (44) |
|
|
|
261 | (2) |
|
|
263 | (4) |
|
|
263 | (1) |
|
7.2.2 Invariance Requirements |
|
|
264 | (2) |
|
7.2.3 Problem Statement and Taxonomy |
|
|
266 | (1) |
|
7.3 Shape Spaces and Metrics |
|
|
267 | (11) |
|
7.3.1 Kendall's Shape Space |
|
|
268 | (3) |
|
7.3.2 Metrics That Capture Physical Deformations |
|
|
271 | (4) |
|
7.3.3 Transformation-Based Representations |
|
|
275 | (3) |
|
7.4 Registration and Geodesies |
|
|
278 | (8) |
|
|
278 | (4) |
|
|
282 | (4) |
|
7.5 Statistical Analysis Under Elastic Metrics |
|
|
286 | (3) |
|
7.5.1 Statistical Analysis Using Non-Euclidean Metrics |
|
|
286 | (2) |
|
7.5.2 Statistical Analysis by SRNF Inversion |
|
|
288 | (1) |
|
7.6 Examples and Applications |
|
|
289 | (8) |
|
7.6.1 Registration and Geodesic Deformations |
|
|
289 | (2) |
|
7.6.2 Elastic Coregistration of 3D Shapes |
|
|
291 | (1) |
|
|
292 | (2) |
|
7.6.4 Random 3D Model Synthesis |
|
|
294 | (1) |
|
|
294 | (3) |
|
7.7 Summary and Perspectives |
|
|
297 | (8) |
|
|
299 | (1) |
|
|
299 | (6) |
|
Chapter 8 Markov Models and MCMC Algorithms in Image Processing |
|
|
305 | (42) |
|
|
8.1 Introduction: The Probabilistic Approach in Image Analysis |
|
|
305 | (1) |
|
8.2 Lattice-based Models and the Bayesian Paradigm |
|
|
306 | (9) |
|
|
306 | (2) |
|
|
308 | (3) |
|
8.2.3 Parameter Estimation |
|
|
311 | (4) |
|
8.3 Some Inverse Problems |
|
|
315 | (7) |
|
8.3.1 Denoising and Deconvolution: The Restoration Problem |
|
|
315 | (1) |
|
8.3.2 Segmentation Problem |
|
|
316 | (4) |
|
|
320 | (2) |
|
8.4 Spatial Point Processes |
|
|
322 | (8) |
|
|
323 | (2) |
|
|
325 | (5) |
|
8.5 Multiple Objects Detection |
|
|
330 | (12) |
|
8.5.1 Population Evaluation |
|
|
330 | (4) |
|
8.5.2 Road Network Detection |
|
|
334 | (8) |
|
|
342 | (5) |
|
|
342 | (2) |
|
|
344 | (3) |
|
SECTION 3 IMAGE AND VIDEO-BASED ANALYTICS |
|
|
|
Chapter 9 Scalable Image Informatics |
|
|
347 | (18) |
|
|
|
|
|
|
347 | (2) |
|
|
347 | (2) |
|
|
349 | (3) |
|
|
351 | (1) |
|
9.2.2 Versioning, Provenance, and Queries |
|
|
351 | (1) |
|
|
352 | (4) |
|
9.3.1 Uniform Metadata Representation and Query Orchestration: Data Service |
|
|
353 | (1) |
|
9.3.2 Scalability of Micro-Services and Analysis |
|
|
353 | (1) |
|
9.3.3 Analysis Extensions: Module Service |
|
|
354 | (1) |
|
9.3.4 Uniform Representation of Heterogeneous Storage Subsystems: Blob Service |
|
|
355 | (1) |
|
9.3.5 Uniform Access and Operations Over Data Files: Image Service and Table Service |
|
|
355 | (1) |
|
|
356 | (2) |
|
9.4.1 Python and Matlab Scripting |
|
|
357 | (1) |
|
|
357 | (1) |
|
9.4.3 Complex Module Execution Descriptors |
|
|
357 | (1) |
|
9.5 Building on the Concepts: Sparse Images |
|
|
358 | (1) |
|
9.6 Feature Services and Machine Leaning |
|
|
359 | (2) |
|
|
359 | (1) |
|
9.6.2 Connoisseur Service for Deep Learning |
|
|
359 | (2) |
|
9.6.3 Connoisseur Module for Domain Experts |
|
|
361 | (1) |
|
9.7 Application Example: Annotation and Classification of Underwater Images |
|
|
361 | (2) |
|
|
363 | (2) |
|
|
364 | (1) |
|
|
364 | (1) |
|
Chapter 10 Person Re-identification |
|
|
365 | (30) |
|
|
|
|
365 | (2) |
|
10.2 The Re-identification Problem: Scenarios, Taxonomies, and Related Work |
|
|
367 | (4) |
|
10.2.1 The Scenarios and Taxonomy |
|
|
367 | (1) |
|
|
368 | (2) |
|
10.2.3 Feature Extraction |
|
|
370 | (1) |
|
|
371 | (1) |
|
10.3 Experimental Evaluation of Re-id Datasets and Their Characteristics |
|
|
371 | (6) |
|
|
377 | (8) |
|
10.4.1 Object Segmentation |
|
|
378 | (1) |
|
10.4.2 Symmetry-Based Silhouette Partition |
|
|
379 | (2) |
|
10.4.3 Symmetry-Driven Accumulation of Local Features |
|
|
381 | (3) |
|
10.4.4 The Matching Phase |
|
|
384 | (1) |
|
|
385 | (5) |
|
10.5.1 Mahalanobis Metric Learning |
|
|
386 | (1) |
|
10.5.2 Large Margin Nearest Neighbor |
|
|
387 | (1) |
|
10.5.3 Efficient Impostor-Based Metric Learning |
|
|
388 | (1) |
|
|
389 | (1) |
|
10.6 Conclusions and New Challenges |
|
|
390 | (5) |
|
|
390 | (5) |
|
Chapter 11 Social Network Inference in Videos |
|
|
395 | (30) |
|
|
|
|
395 | (3) |
|
|
398 | (2) |
|
11.3 Video Shot Segmentation |
|
|
400 | (1) |
|
|
401 | (1) |
|
11.5 Learning to Group Actors |
|
|
402 | (8) |
|
|
404 | (3) |
|
|
407 | (2) |
|
|
409 | (1) |
|
11.6 Inferring Social Communities |
|
|
410 | (2) |
|
11.6.1 Social Network Graph |
|
|
410 | (1) |
|
11.6.2 Actor Interaction Model |
|
|
411 | (1) |
|
11.7 Social Network Analysis |
|
|
412 | (1) |
|
11.7.1 Assignment to Communities |
|
|
412 | (1) |
|
11.7.2 Estimating Community Leader |
|
|
413 | (1) |
|
|
413 | (5) |
|
|
414 | (1) |
|
11.8.2 Audiovisual Alignment |
|
|
414 | (1) |
|
|
415 | (1) |
|
11.8.4 Community Assignment |
|
|
415 | (3) |
|
|
418 | (1) |
|
|
418 | (1) |
|
|
418 | (3) |
|
|
421 | (4) |
|
|
422 | (2) |
|
|
424 | (1) |
Index |
|
425 | |