Video Content Analysis Using Multimodal Information

With the fast growth ofmultimedia information content-based video anal­ ysis indexing and representation have attracted increasing attention in re­ cent years. Many applications have emerged in these areas such as video­ on-demand distributed multimedia systems digital video libraries distance learning/education entertainment surveillance and geographical information systems. The need for content-based video indexing and retrieval was also rec­ ognized by ISOIMPEG and a new international standard called \Multimedia Content Description Interface\ (or in short MPEG-7)was initialized in 1998 and finalized in September 2001. In this context a systematic and thorough review ofexisting approaches as well as the state-of-the-art techniques in video content analysis indexing and representation areas are investigated and studied in this book. In addition we will specifically elaborate on a system which analyzes indexes and abstracts movie contents based on the integration ofmultiple media modalities. Content ofeach part ofthis book is briefly previewed below. In the first part we segment a video sequence into a set ofcascaded shots where a shot consistsofone or more continuouslyrecorded image frames. Both raw and compressedvideo data will beinvestigated. Moreover consideringthat there are always non-story units in real TV programs such as commercials a novel commercial break detection/extraction scheme is developed which ex­ ploits both audio and visual cues to achieve robust results. Specifically we first employ visual cues such as the video data statistics the camera cut fre­ quency and the existenceofdelimiting black frames between commercials and programs to obtain coarse-level detection results.