- Slide: Single key frame; every image from the lecture video's collection of images from any given class is considered a Slide. This includes images captured directly from the video as well as images added manually, such as higher-quality electronic slides.
- Topic: Collection of all Slides that contain similar content. In particular for Slides of media type Board and Sheet, this refers to images that capture the same written (or graphic) material. The physical boundaries for this material are: 1. Board: physically separated blackboard/whiteboard panels, and 2. Sheet: a sheet of paper or facing pages of a book.
- Section: Collection of Slides of one particular media type that are a subset of one Topic AND that occur consecutively (temporal) in the ordered series of Slides.
- Media Types / Categories:
2.2 Content Interaction: Media Types
The standard media- and recording-enabled classroom is very limited in physical points of interest for video recording. Most of the time, the camera focuses on the instructor or the lecture material. Lecture material tends to be communicated via few types of media, including the traditional blackboard, a whiteboard, sheets of written paper, transparencies, or printed material whose content is visually projected, electronic slides, and other digital content. At other times, the camera captures the instructor addressing the students verbally without the use of material, and rarely, the camera captures students in the classroom during a discussion or a question.
In our design for the Lecture Browser, we explicitely differentiate between various types of media. The difference between media types is important for two several reasons:
- The type of media captured by the camera (i.e. used by the instructor) should have significance: depending on the course, lecture content disseminated through written media versus digital media versus verbal discussion may fall into a certain expected categories. For example, an instructor making use of electronic slides for content dissemination may occasionally work through an example on the blackboard. Here, the differentiation in media type suggests a specific role that the underlying video content assumes: lecture content versus example problem. For a second example, an instructor making use of one of the media types for content dissemination may occasionally address the students without material. Here, the difference in media type suggests dissemination of lecture material versus discussion about homework or other logistics.
- For a student participating in the live course and using the video as review material, the type of media may elicit a useful memory of the discussed content. For example, when searching for specific content, the student may recall that this content was projected from the computer. The concept of relating information to location (media type) was already employed by the Romans in the "Forum Romanum Method":
"Roman senators used to carefully prepare an impressive structure of their speech and a well-considered sequence of their arguments. They debated without written notes, and thus it was important to not forget any detail.
In preparing their arguments, the Roman speakers walked through the streets and visited numerous places in the city, while fixing keywords to columns, archways, or fountains. During their speech to the senate they would imagine themselves re-visiting the same trail, and re-visiting all of the landmarks and famous buildings, only to remember the original sequence of arguments they planned to bring forth in their speech." (Bruno Klumpp, 1998, 2001)
- For certain media types that disseminate material (blackboard, projected writing, etc.), it is possible to cluster key frames with similar content. Once classified, key frames of these media types are analyzed according to their characteristics.
We have identified five predominant media types:
- Board: Black board, white board
- Class: View of the students during a discussion, etc.
- Computer: Material presented with the computer, e.g. electronic slides, web sites, etc.
- Podium: View of the instructor addressing the class without using any material/tools, e.g. in a discussion with the class, announcing assignments, etc.
- Sheet: Projected view of material on prepared or in-class generated sheets of paper, content/illustrations from printed media, etc.
Not all media types are used in a typical class. Instructors tend to prefer either the blackboard or projected paper material as their predominant delivery method. Other media are used occasionally.
Note: with the continued development of electronic media, electronic slides have likely surpassed instructor-written classroom content by now. Nevertheless, the research presented here can easily be extended to a more in-depth analysis of electronic slides as the main delivery media in a classroom.
2.3 Content Clusters: Topics
Lecture content develops slowly, resulting in highly redundant Slides with similar content for most parts of the lecture. Especially when the blackboard or projected written material are used, the instructor requires time to write the material while (presumably) explaining it in some detail. While the highly redundant individual Slides remain important information, they should be clustered into sets. These sets serve several purposes:
- 1. Compress the large number of Slides in a lecture video into significantly fewer unique Topics; the ratio of Slides:Topics can be as high as 20:1 on average
- 2. Create a more structured representation of the lecture; the structure easily reveals the beginning and end of individual Topics
2.4 Topics and Sections
A Topic is defined as a set of Slides with the same contents, that is contents, which can be visually superimposed and matched. While Slides with similar content tend to appear consecutively, it is possible, and not uncommon, for Topics to include breaks. A break in a Topic occurs when the camera changes its field of view, records other material, and then returns to the original view. The equivalent in a classroom can occur for many reasons:
- The instructor moves away from one blackboard, which outlines theory, and moves to another blackboard (or other media type) to show an example
- One blackboard outlines the topics of the day, which are used as a reference (table of contents) throughout the lecture
- The instructor uses hand-written sheets of paper, and refers back to an earlier sheet to make a point
Consecutive Slides within a Topic are termed Sections, and axiomatically, each Topic contains at least one Section.
Even though non-consecutive Slides with similar content are assigned to one Topic, the gaps within overlap temporally with content from other Topics. The Lecture Browser must be sensitive to these inconsistencies and present the Slides in Topics while preserving the temporal dimension.