Real-time Chroma Key Matting Using Image Statistics

Abstract

Researchers at the University of Central Florida have devised an improved system, method, and computer readable media for real-time chroma key matting by using image statistics. Chroma keying is a special effects or post-production technique for layering two images or video streams together based on color hues. This new technology operates in real-time, functions in visually noisy environments, and simplifies calibration. Prior techniques have required low signal noise, significant calibration, and do not operate in real-time. The UCF invention meets the need for an affordable, robust, and fast matting process that works with low-quality video signals, less than perfect lighting environments, and non-uniform matting material.

Technical Details

The UCF approach is different from other technologies in that it incorporates the use of a processor and preferably uses the Graphic Processing Unit (GPU) of the computer, thus freeing up CPU cycles for other real-time operations. An alpha matte is generated from a video signal based on the chroma key information in real-time via pixel shaders. To accomplish this, princial components analysis (PCA) generates a linear transformation matrix where the resulting color triplet's Euclidean distance is directly related to the probability that the color exists in the chroma key spectrum. The result of this process is a trimap of the video signal's opacity. To solve the alpha matte from the trimap, an energy function constrained by the trimap is minimized with gradient descent. This energy function is based on the least-squared error of overlapping neighborhoods around each pixel and is not dependent on the background or foreground color. The resulting alpha matte is processed within the GPU, is easy to calibrate, robust to noise and operates at exceptionally high frame rates. An optional error minimization stage improves the estimated matte, accounting for misclassifications and signal noise. Given the resulting matte, standard alpha blending composites the virtual scene with the video feed to create the illusion that both worlds coexist.

Benefit

Market Application