Indexing for topics in videos using foils

Abstract

A long-standing goal of distance learning has been to provide a quality of learning comparable to the face-to-face environment of a traditional classroom for teaching or training. One of the fundamental problems in achieving this goal is providing effective ways of high-level semantic querying such as for the retrieval of relevant learning material relating to a topic of discussion. In this paper we present a method of identifying video segments relating to a topic of discussion by indexing videos using the image and text content of foils. Specifically, we present a novel method of locating and recognizing foil images in video using the color and spatial layout geometry of their regions. We then search the audio associated with video based on the text content of the foil to identify related video segments in which concepts represented on a foil are heard. Finally, we combine the results of foil image and text search of video exploiting their time co-occurrence. The resulting identification of topics is evaluated in the domain of classroom lectures and talks.