Facial Recognition in Movie Trailers

Abstract

Applications for facial recognition have diversified from security into the domain of entertainment videos. By applying this technology, users search for a particular actor and receive a list of videos in which the actor’s face appeared. In order to accomplish this, a fully automatic end-to-end system for video face recognition is required, which leverages information from still images both in the known dictionary and in the video itself.

In the past, application of the popular 1-minimization Sparse Representation-based Classification (SRC) has been prohibitively expensive since it operates on a frame-by-frame basis. Instead of computing SRC on each frame, which takes approximately 45 minutes per track, MSSRC reduces a face track to a single feature vector for 1-minimization (1.5 minutes per track), while obtaining MSSRC recall at 90 percent precision. By using all of the available video data in a face track, this method provides greater accuracy in identifying faces and excels at rejecting unknown identities. When tested, the Movie Trailer Face Dataset outperformed many existing state-of-the art datasets in identifying actors in movie trailers.

Technical Details

This method developed by UCF researchers performs the difficult task of face tracking using high-performance SHORE face detection, which generates tracks using two metrics: spatial and appearance. While the spatial metric compares the current bounding box with the previous, the appearance metric computes a histogram intersection of the local bounding box, which can handle abrupt changes in the scene and the face. Each new face is compared to existing tracks, and if the location and appearance metric is similar, the face is added to the track. Using this method, 113 movie trailers have been processed to form the Movie Trailer Face Dataset, which consists of 3,585 face tracks. On the face tracks, the MSSRC algorithm performs video face recognition using a joint optimization, leveraging all of the available video data and the knowledge that the face track frames belong to the same individual. The MSSRC algorithm increases speed and cuts computational costs by applying 1-minimization on the mean of the face track, which is five times as fast as fast as a frame-by-frame application.

Benefit

Market Application