About the Author |
|
xiii | |
|
|
xv | |
Series Editor's Foreword |
|
xvii | |
Preface |
|
xix | |
|
|
xxi | |
|
1 Senses, Perception, and Natural Human-Interfaces for Interactive Displays |
|
|
1 | (26) |
|
|
|
1 | (3) |
|
1.2 Human Senses and Perception |
|
|
4 | (5) |
|
1.3 Human Interface Technologies |
|
|
9 | (11) |
|
1.3.1 Legacy Input Devices |
|
|
9 | (2) |
|
1.3.2 Touch-based Interactions |
|
|
11 | (2) |
|
1.3.3 Voice-based Interactions |
|
|
13 | (2) |
|
1.3.4 Vision-based Interactions |
|
|
15 | (3) |
|
1.3.5 Multimodal Interactions |
|
|
18 | (2) |
|
1.4 Towards "True" 3D Interactive Displays |
|
|
20 | (3) |
|
|
23 | (4) |
|
|
24 | (3) |
|
|
27 | (80) |
|
|
|
27 | (1) |
|
2.2 Introduction to Touch Technologies |
|
|
28 | (7) |
|
|
30 | (1) |
|
2.2.2 Classifying Touch Technologies by Size and Application |
|
|
30 | (2) |
|
2.2.3 Classifying Touch Technologies by Materials and Structure |
|
|
32 | (1) |
|
2.2.4 Classifying Touch Technologies by the Physical Quantity Being Measured |
|
|
33 | (1) |
|
2.2.5 Classifying Touch Technologies by Their Sensing Capabilities |
|
|
33 | (1) |
|
2.2.6 The Future of Touch Technologies |
|
|
34 | (1) |
|
2.3 History of Touch Technologies |
|
|
35 | (1) |
|
2.4 Capacitive Touch Technologies |
|
|
35 | (16) |
|
2.4.1 Projected Capacitive (P-Cap) |
|
|
35 | (12) |
|
|
47 | (4) |
|
2.5 Resistive Touch Technologies |
|
|
51 | (10) |
|
|
51 | (6) |
|
2.5.2 Digital Multi-touch Resistive (DMR) |
|
|
57 | (2) |
|
2.5.3 Analog Multi-touch Resistive (AMR) |
|
|
59 | (2) |
|
2.6 Acoustic Touch Technologies |
|
|
61 | (7) |
|
2.6.1 Surface Acoustic Wave (SAW) |
|
|
61 | (3) |
|
2.6.2 Acoustic Pulse Recognition (APR) |
|
|
64 | (3) |
|
2.6.3 Dispersive Signal Technology (DST) |
|
|
67 | (1) |
|
2.7 Optical Touch Technologies |
|
|
68 | (18) |
|
2.7.1 Traditional Infrared |
|
|
68 | (5) |
|
2.7.2 Multi-touch Infrared |
|
|
73 | (3) |
|
2.7.3 Camera-based Optical |
|
|
76 | (5) |
|
2.7.4 In-glass Optical (Planar Scatter Detection -- PSD) |
|
|
81 | (1) |
|
2.7.5 Vision-based Optical |
|
|
82 | (4) |
|
2.8 Embedded Touch Technologies |
|
|
86 | (10) |
|
2.8.1 On-cell Mutual-capacitive |
|
|
89 | (1) |
|
2.5.2 Hybrid In-cell/On-cell Mutual-capacitive |
|
|
90 | (1) |
|
2.8.3 In-cell Mutual-capacitive |
|
|
91 | (2) |
|
2.8.4 in-cell Light Sensing |
|
|
93 | (3) |
|
2.9 Other Touch Technologies |
|
|
96 | (2) |
|
|
96 | (2) |
|
2.9.2 Combinations of Touch Technologies |
|
|
98 | (1) |
|
|
98 | (2) |
|
|
100 | (7) |
|
|
101 | (6) |
|
3 Voice in the User Interface |
|
|
107 | (58) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
107 | (3) |
|
|
110 | (9) |
|
|
110 | (2) |
|
3.2.2 Acoustic Model and Front-end |
|
|
112 | (1) |
|
3.2.3 Aligning Speech to HMMs |
|
|
113 | (1) |
|
|
114 | (1) |
|
3.2.5 Search: Solving Crosswords at 1000 Words a Second |
|
|
115 | (1) |
|
3.2.6 Training Acoustic and Language Models |
|
|
116 | (1) |
|
3.2.7 Adapting Acoustic and Language Models for Speaker Dependent Recognition |
|
|
116 | (1) |
|
3.2.8 Alternatives to the "Canonical" System |
|
|
117 | (1) |
|
|
117 | (2) |
|
3.3 Deep Neural Networks for Voice Recognition |
|
|
119 | (3) |
|
3.4 Hardware Optimization |
|
|
122 | (1) |
|
3.4.1 Lower Power Wake-up Computation |
|
|
122 | (1) |
|
3.4.2 Hardware Optimization for Specific Computations |
|
|
123 | (1) |
|
3.5 Signal Enhancement Techniques for Robust Voice Recognition |
|
|
123 | (5) |
|
3.5.1 Robust Voice Recognition |
|
|
124 | (1) |
|
3.5.2 Single-channel Noise Suppression |
|
|
124 | (1) |
|
3.5.3 Multi-channel Noise Suppression |
|
|
125 | (1) |
|
|
125 | (2) |
|
3.5.5 Acoustic Echo Cancellation |
|
|
127 | (1) |
|
|
127 | (1) |
|
|
128 | (2) |
|
|
128 | (1) |
|
3.6.2 Existing Challenges to Voice Biometrics |
|
|
129 | (1) |
|
3.6.3 New Areas of Research in Voice Biometrics |
|
|
130 | (1) |
|
|
130 | (4) |
|
3.8 Natural Language Understanding |
|
|
134 | (7) |
|
3.8.1 Mixed Initiative Conversations |
|
|
135 | (2) |
|
3.8.2 Limitations of Slot and Filler Technology |
|
|
137 | (4) |
|
3.9 Multi-turn Dialog Management |
|
|
141 | (3) |
|
3.10 Planning and Reasoning |
|
|
144 | (7) |
|
3.10.1 Technical Challenges |
|
|
144 | (2) |
|
3.10.2 Semantic Analysis and Discourse Representation |
|
|
146 | (1) |
|
|
147 | (1) |
|
3.10.4 Dialog Management as Collaboration |
|
|
148 | (1) |
|
3.10.5 Planning and Re-planning |
|
|
149 | (1) |
|
3.10.6 Knowledge Representation and Reasoning |
|
|
149 | (1) |
|
|
150 | (1) |
|
3.10.8 Suggested Readings |
|
|
151 | (1) |
|
|
151 | (3) |
|
|
152 | (1) |
|
3.11.2 Find Relevant Information |
|
|
152 | (1) |
|
3.11.3 Answers and Evidence |
|
|
153 | (1) |
|
3.11.4 Presenting the Answer |
|
|
153 | (1) |
|
3.12 Distributed Voice Interface Architecture |
|
|
154 | (3) |
|
3.12.1 Distributed User Interfaces |
|
|
154 | (1) |
|
3.12.2 Distributed Speech and Language Technology |
|
|
155 | (2) |
|
|
157 | (8) |
|
|
158 | (1) |
|
|
158 | (7) |
|
4 Visual Sensing and Gesture Interactions |
|
|
165 | (16) |
|
|
|
165 | (2) |
|
4.2 Imaging Technologies: 2D and 3D |
|
|
167 | (3) |
|
4.3 Interacting with Gestures |
|
|
170 | (7) |
|
|
177 | (4) |
|
|
178 | (3) |
|
5 Real-Time 3D Sensing With Structured Light Techniques |
|
|
181 | (34) |
|
|
|
|
|
181 | (2) |
|
5.2 Structured Pattern Codifications |
|
|
183 | (8) |
|
5.2.1 2D Pseudo-random Codifications |
|
|
183 | (1) |
|
5.2.2 Binary Structured Codifications |
|
|
184 | (3) |
|
5.2.3 N-ary Codifications |
|
|
187 | (1) |
|
5.2.4 Continuous Sinusoidal Phase Codifications |
|
|
187 | (4) |
|
5.3 Structured Light System Calibration |
|
|
191 | (2) |
|
5.4 Examples of 3D Sensing with DFP Techniques |
|
|
193 | (2) |
|
5.5 Real-Time 3D Sensing Techniques |
|
|
195 | (6) |
|
5.5.1 Fundamentals of Digital-light-processing (DLP) Technology |
|
|
196 | (2) |
|
5.5.2 Real-Time 3D Data Acquisition |
|
|
198 | (1) |
|
5.5.3 Real-Time 3D Data Processing and Visualization |
|
|
199 | (1) |
|
5.5.4 Example of Real-Time 3D Sensing |
|
|
200 | (1) |
|
5.6 Real-Time 3D Sensing for Human Computer Interaction Applications |
|
|
201 | (3) |
|
5.6.1 Real-Time 3D Facial Expression Capture and its HCI Implications |
|
|
201 | (1) |
|
5.6.2 Real-Time 3D Body Part Gesture Capture and its HCI Implications |
|
|
202 | (2) |
|
5.6.3 Concluding Human Computer Interaction Implications |
|
|
204 | (1) |
|
5.7 Some Recent Advancements |
|
|
204 | (4) |
|
5.7.1 Real-Time 3D Sensing and Natural 2D Color Texture Capture |
|
|
204 | (2) |
|
5.7.2 Superfast 3D Sensing |
|
|
206 | (2) |
|
|
208 | (7) |
|
|
209 | (1) |
|
|
209 | (6) |
|
6 Real-Time Stereo 3D Imaging Techniques |
|
|
215 | (18) |
|
|
|
215 | (1) |
|
|
216 | (3) |
|
6.3 Structure of Stereo Correspondence Algorithms |
|
|
219 | (3) |
|
6.3.1 Matching Cost Computation |
|
|
220 | (1) |
|
6.3.2 Matching Cost Aggregation |
|
|
221 | (1) |
|
6.4 Categorization of Characteristics |
|
|
222 | (3) |
|
6.4.1 Depth Estimation Density |
|
|
222 | (2) |
|
6.4.2 Optimization Strategy |
|
|
224 | (1) |
|
6.5 Categorization of Implementation Platform |
|
|
225 | (4) |
|
|
225 | (1) |
|
6.5.2 GPU-accelerated Methods |
|
|
226 | (1) |
|
6.5.3 Hardware Implementations (FPGAs, ASICs) |
|
|
227 | (2) |
|
|
229 | (4) |
|
|
229 | (4) |
|
7 Time-of-Flight 3D-Imaging Techniques |
|
|
233 | (18) |
|
|
|
233 | (1) |
|
7.2 Time-of-Flight 3D Sensing |
|
|
233 | (2) |
|
7.3 Pulsed Time-of-Flight Method |
|
|
235 | (1) |
|
7.4 Continuous Time-of-Flight Method |
|
|
236 | (1) |
|
|
236 | (3) |
|
|
239 | (1) |
|
7.7 Limitations and Improvements |
|
|
240 | (4) |
|
|
240 | (1) |
|
|
241 | (1) |
|
|
242 | (1) |
|
7.7.4 Multi-path and Scattering |
|
|
243 | (1) |
|
7.7.5 Power Budget and Optimization |
|
|
243 | (1) |
|
7.8 Time-of-Flight Camera Components |
|
|
244 | (1) |
|
|
244 | (3) |
|
|
244 | (1) |
|
|
245 | (2) |
|
7.10 Current State of the Art |
|
|
247 | (1) |
|
|
247 | (4) |
|
|
248 | (3) |
|
|
251 | (34) |
|
|
8.1 Introduction and Motivation |
|
|
251 | (2) |
|
|
253 | (3) |
|
|
256 | (4) |
|
8.3.1 Types of Eye Trackers |
|
|
256 | (1) |
|
8.3.2 Corneal Reflection Method |
|
|
257 | (3) |
|
8.4 Objections and Obstacles |
|
|
260 | (3) |
|
|
260 | (1) |
|
|
261 | (1) |
|
|
261 | (1) |
|
|
261 | (1) |
|
8.4.5 Midas Touch Problem |
|
|
262 | (1) |
|
8.5 Eye Gaze Interaction Research |
|
|
263 | (1) |
|
|
264 | (6) |
|
8.6.1 Solving the Midas Touch Problem |
|
|
264 | (1) |
|
8.6.2 Solving the Accuracy Issue |
|
|
265 | (1) |
|
8.6.3 Comparison of Mouse and Gaze Pointing |
|
|
266 | (1) |
|
8.6.4 Mouse and Gaze Coordination |
|
|
267 | (2) |
|
8.6.5 Gaze Pointing Feedback |
|
|
269 | (1) |
|
|
270 | (5) |
|
8.7.1 The Concept of Gaze Gestures |
|
|
270 | (1) |
|
8.7.2 Gesture Detection Algorithm |
|
|
270 | (1) |
|
8.7.3 Human Ability to Perform Gaze Gestures |
|
|
271 | (1) |
|
8.7.4 Gaze Gesture Alphabets |
|
|
272 | (1) |
|
8.7.5 Gesture Separation from Natural Eye Movement |
|
|
273 | (1) |
|
8.7.6 Applications for Gaze Gestures |
|
|
274 | (1) |
|
|
275 | (5) |
|
8.8.1 Activity Recognition |
|
|
275 | (2) |
|
|
277 | (2) |
|
8.8.3 Attention Detection |
|
|
279 | (1) |
|
|
280 | (1) |
|
|
280 | (5) |
|
|
281 | (4) |
|
9 Multimodal Input for Perceptual User Interfaces |
|
|
285 | (28) |
|
|
|
|
|
285 | (1) |
|
9.2 Multimodal Interaction Types |
|
|
286 | (1) |
|
9.3 Multimodal Interfaces |
|
|
287 | (16) |
|
|
287 | (7) |
|
|
294 | (5) |
|
9.3.3 Eye Tracking and Gaze |
|
|
299 | (1) |
|
|
300 | (1) |
|
9.3.5 Brain-computer Input |
|
|
301 | (2) |
|
9.4 Multimodal Integration Strategies |
|
|
303 | (2) |
|
9.4.1 Frame-based Integration |
|
|
304 | (1) |
|
9.4.2 Unification-based Integration |
|
|
304 | (1) |
|
9.4.3 Procedural Integration |
|
|
305 | (1) |
|
9.4.4 Symbolic/Statistical Integration |
|
|
305 | (1) |
|
9.5 Usability Issues with Multimodal Interaction |
|
|
305 | (2) |
|
|
307 | (6) |
|
|
308 | (5) |
|
10 Multimodal Interaction in Biometrics: Technological and Usability Challenges |
|
|
313 | (30) |
|
|
|
|
|
313 | (7) |
|
10.1.1 Motivations for Identity Assurance |
|
|
314 | (1) |
|
|
314 | (1) |
|
10.1.3 Application Characteristics of Multimodal Biometrics |
|
|
314 | (2) |
|
10.1.4 2D and 3D Face Recognition |
|
|
316 | (1) |
|
10.1.5 A Multimodal Case Study |
|
|
317 | (1) |
|
10.1.6 Adaptation to Blind Subjects |
|
|
318 | (2) |
|
10.1.7 Chapter Organization |
|
|
320 | (1) |
|
10.2 Anatomy of the Mobile Biometry Platform |
|
|
320 | (8) |
|
|
320 | (3) |
|
|
323 | (2) |
|
|
325 | (1) |
|
|
326 | (1) |
|
10.2.5 Mobile Platform Implementation |
|
|
326 | (1) |
|
10.2.6 MoBio Database and Protocol |
|
|
327 | (1) |
|
10.3 Case Study: Usability Study for the Visually Impaired |
|
|
328 | (10) |
|
10.3.1 Impact of Head Pose Variations on Performance |
|
|
329 | (1) |
|
10.3.2 User Interaction Module: Head Pose Quality Assessment |
|
|
329 | (4) |
|
10.3.3 User-Interaction Module: Audio Feedback Mechanism |
|
|
333 | (3) |
|
10.3.4 Usability Testing with the Visually Impaired |
|
|
336 | (2) |
|
10.4 Discussions and Conclusions |
|
|
338 | (5) |
|
|
339 | (1) |
|
|
339 | (4) |
|
11 Towards "True" 3D Interactive Displays |
|
|
343 | (32) |
|
|
|
|
|
343 | (3) |
|
11.2 The Origins of Biological Vision |
|
|
346 | (6) |
|
|
352 | (7) |
|
11.4 Towards "True" 3D Visual Displays |
|
|
359 | (9) |
|
11.5 Interacting with Visual Content on a 3D Display |
|
|
368 | (3) |
|
|
371 | (4) |
|
|
371 | (4) |
Index |
|
375 | |