FilmGPT: Autoregressive Transformer for Video Montage
FilmGPT is an autoregressive transformer trained on a large corpus of movies to learn the statistical patterns, or 'grammar,' of film editing. Rather than generating new video frames, it selects the best next shot from raw footage using a footage-constrained decoding algorithm. FilmGPT outperforms previous methods in shot prediction benchmarks and user studies evaluating full film editing tasks.
Why it matters: This work demonstrates a novel AI approach that automates aspects of video montage by learning editing patterns directly from film data, advancing the automation of a complex creative process.
Full story at: arXiv Computer Vision ↗