Abstract
The University of Central Florida invention, Spectral Shift Aware Video Editing (SAVE), fine-tunes the spectral shift of the parameter space, significantly reducing the number of trainable parameters and improving computational efficiency. The UCF method includes a novel text-guided video editing framework and a spectral shift regularizer to capture motion information and preserve scene generation capability. It also incorporates frame attention for spatial and temporal consistency.
Text-to-Image (T2I) diffusion models have achieved remarkable success in synthesizing high-quality images conditioned on text prompts. Recent methods have tried to replicate the success by either training text-to-video (T2V) models on a large number of text-video pairs or adapting T2I models on text-video pairs independently. Although the latter is computationally less expensive, it still takes significant time for per-video adaptation. To address this issue, the UCF spectral-shift-aware adaptation framework enables the spectral shift of the parameter space to be fine-tuned instead of the parameters themselves.
Partnering Opportunity
The research team is seeking partners for licensing, research collaboration, or both.
Stage of Development
Prototype available.
Benefit
More efficient and timesavingLess costlyCan be integrated with any video editing frameworkMarket Application
Video editing softwarePublications
SAVE: Spectral-Shift-Aware Adaptation
of Image Diffusion Models for Text-driven Video Editing, arXiv:2305.18670
[cs.CV], submitted on 30 May 2023 (v1), last revised 1 Dec 2023 (this version,
v2), https://doi.org/10.48550/arXiv.2305.18670
Brochure
