The advent of increasingly large consumer collections of audio (e.g., iTunes), imagery (e.g., Flickr), and video (e.g., YouTube) is driving a need not only for multimedia retrieval but also information extraction from and across media. Furthermore, industrial and government collections fuel requirements for stock media access, media preservation, broadcast news retrieval, identity management, and video surveillance. While significant advances have been made in language processing for information extraction from unstructured multilingual text and extraction of objects from imagery and video, these advances have been explored in largely independent research communities who have addressed extracting information from single media (e.g., text, imagery, audio). And yet users need to search for concepts across individual media, author multimedia artifacts, and perform multimedia analysis in many domains.
This collection is intended to serve several purposes, including reporting the current state of the art, stimulating novel research, and encouraging cross-fertilization of distinct research disciplines. The collection and integration of a common base of intellectual material will provide an invaluable service from which to teach a future generation of cross disciplinary media scientists and engineers.
FOREWORD ix Alan F. Smeaton PREFACE xiii Mark T. Maybury
ACKNOWLEDGMENTS xv CONTRIBUTORS xvii 1 INTRODUCTION 1 Mark T. Maybury 2
MULTIMEDIA INFORMATION EXTRACTION: HISTORY AND STATE OF THE ART 13 Mark T.
Maybury SECTION 1 IMAGE EXTRACTION 41 3 VISUAL FEATURE LOCALIZATION FOR
DETECTING UNIQUE OBJECTS IN IMAGES 45 Madirakshi Das, Alexander C. Loui, and
Andrew C. Blose 4 ENTROPY-BASED ANALYSIS OF VISUAL AND GEOLOCATION CONCEPTS
IN IMAGES 63 Keiji Yanai, Hidetoshi Kawakubo, and Kobus Barnard 5 THE
MEANING OF 3D SHAPE AND SOME TECHNIQUES TO EXTRACT IT 81 Sven Havemann,
Torsten Ullrich, and Dieter W. Fellner 6 A DATA-DRIVEN MEANINGFUL
REPRESENTATION OF EMOTIONAL FACIAL EXPRESSIONS 99 Nicolas Stoiber, Gaspard
Breton, and Renaud Seguier SECTION 2 VIDEO EXTRACTION 113 7 VISUAL
SEMANTICS FOR REDUCING FALSE POSITIVES IN VIDEO SEARCH 119 Rohini K. Srihari
and Adrian Novischi 8 AUTOMATED ANALYSIS OF IDEOLOGICAL BIAS IN VIDEO 129
Wei-Hao Lin and Alexander G. Hauptmann 9 MULTIMEDIA INFORMATION EXTRACTION
IN A LIVE MULTILINGUAL NEWS MONITORING SYSTEM 145 David D. Palmer, Marc B.
Reichman, and Noah White 10 SEMANTIC MULTIMEDIA EXTRACTION USING AUDIO AND
VIDEO 159 Evelyne Tzoukermann, Geetu Ambwani, Amit Bagga, Leslie Chipman,
Anthony R. Davis, Ryan Farrell, David Houghton, Oliver Jojic, Jan Neumann,
Robert Rubinoff, Bageshree Shevade, and Hongzhong Zhou 11 ANALYSIS OF
MULTIMODAL NATURAL LANGUAGE CONTENT IN BROADCAST VIDEO 175 Prem Natarajan,
Ehry MacRostie, Rohit Prasad, and Jonathan Watson 12 WEB-BASED MULTIMEDIA
INFORMATION EXTRACTION BASED ON SOCIAL REDUNDANCY 185 Jose San Pedro, Stefan
Siersdorfer, Vaiva Kalnikaite, and Steve Whittaker 13 INFORMATION FUSION
AND ANOMALY DETECTION WITH UNCALIBRATED CAMERAS IN VIDEO SURVEILLANCE 201
Erhan Baki Ermis, Venkatesh Saligrama, and Pierre-Marc Jodoin SECTION 3
AUDIO, GRAPHICS, AND BEHAVIOR EXTRACTION 217 14 AUTOMATIC DETECTION,
INDEXING, AND RETRIEVAL OF MULTIPLE ATTRIBUTES FROM CROSS-LINGUAL MULTIMEDIA
DATA 221 Qian Hu, Fred J. Goodman, Stanley M. Boykin, Randall K. Fish,
Warren R. Greiff, Stephen R. Jones, and Stephen R. Moore 15 INFORMATION
GRAPHICS IN MULTIMODAL DOCUMENTS 235 Sandra Carberry, Stephanie Elzer,
Richard Burns, Peng Wu, Daniel Chester, and Seniz Demir 16 EXTRACTING
INFORMATION FROM HUMAN BEHAVIOR 253 Fabio Pianesi, Bruno Lepri, Nadia Mana,
Alessandro Cappelletti, and Massimo Zancanaro SECTION 4 AFFECT EXTRACTION
FROM AUDIO AND IMAGERY 269 17 RETRIEVAL OF PARALINGUISTIC INFORMATION IN
BROADCASTS 273 Bjorn Schuller, Martin Wollmer, Florian Eyben, and Gerhard
Rigoll 18 AUDIENCE REACTIONS FOR INFORMATION EXTRACTION ABOUT PERSUASIVE
LANGUAGE IN POLITICAL COMMUNICATION 289 Marco Guerini, Carlo Strapparava,
and Oliviero Stock 19 THE NEED FOR AFFECTIVE METADATA IN CONTENT-BASED
RECOMMENDER SYSTEMS FOR IMAGES 305 Marko TkalEiE, Jurij TasiE, and Andrej
Kosir 20 AFFECT-BASED INDEXING FOR MULTIMEDIA DATA 321 Gareth J. F. Jones
and Ching Hau Chan SECTION 5 MULTIMEDIA ANNOTATION AND AUTHORING 347 21
MULTIMEDIA ANNOTATION, QUERYING, AND ANALYSIS IN ANVIL 351 Michael Kipp 22
TOWARD FORMALIZATION OF DISPLAY GRAMMAR FOR INTERACTIVE MEDIA PRODUCTION WITH
MULTIMEDIA INFORMATION EXTRACTION 369 Robin Bargar 23 MEDIA AUTHORING WITH
ONTOLOGICAL REASONING: USE CASE FOR MULTIMEDIA INFORMATION EXTRACTION 385
Insook Choi 24 ANNOTATING SIGNIFICANT RELATIONS ON MULTIMEDIA WEB DOCUMENTS
401 Matusala Addisu, Danilo Avola, Paola Bianchi, Paolo Bottoni, Stefano
Levialdi, and Emanuele Panizzi ABBREVIATIONS AND ACRONYMS 419 REFERENCES
425 INDEX 461
MARK T. MAYBURY, PhD , is an Executive Director at MITRE, a federally funded research and development center. In 2010, Dr. Maybury took a leave of absence from MITRE when appointed to the role of Chief Scientist of the United States Air Force. He is a former member of the board of directors of the Object Management Group and the Air Force Scientific Advisory Board. An IEEE Fellow, Dr. Maybury is a member of the ACM Intelligent User Interface Steering Committee and has served on the Advanced Visual Interfaces Program Committee for over ten years. He holds several patents and has edited, coedited, or coauthored a number of books on information retrieval and related topics.
