Animation separates foreground and background elements into layers, with distinct processes for sketching, refining, coloring, and in-betweening. Existing video generation methods typically treat animation as a monolithic data domain, lacking fine-grained control over individual layers. In this paper, we introduce LayerAnimate, a novel architectural approach that enhances fine-grained control over individual animation layers within a video diffusion model, allowing users to independently manipulate foreground and background elements in distinct layers. To address the challenge of limited layer-specific data, we propose a data curation pipeline that features automated element segmentation, motion-state hierarchical merging, and motion coherence refinement. Through quantitative and qualitative comparisons, and user study, we demonstrate that LayerAnimate outperforms current methods in terms of animation quality, control precision, and usability, making it an ideal tool for both professional animators and amateur enthusiasts. This framework opens up new possibilities for layer-specific animation applications and creative flexibility.
Left: Layer Curation Pipeline. The bottom orange dashed box illustrates the curated layer masks with their motion scores. Motion scores remain temporally constant throughout the animation. Right: Motion-state Hierarchical Merging. Layers are merged from the bottom up until the layer count L falls below the maximum layer capacity N and the motion score difference ds exceeds the threshold ηs.
Overview of LayerAnimate. Given reference images, layer masks, and their motion information, LayerAnimate enables animation generation with precise layer-specific control. Besides the motion scores, LayerAnimate supports alternative motion information, such as sketches.
Qualitative comparison with other competitors. We select four clips to exemplify the representative characteristics of animation: ① particle effects, ② light effects, ③ smoke appearing from nowhere, ④ unconventional fade-in visual style.
Layer-specific Application. LayerAnimate provides innovative and user-friendly control options for animation, enabling users to freeze specific elements, animate characters with partial sketches, and switch static or dynamic backgrounds. The layer-specific control over individual layers ensures that foreground layers remain consistent and nearly unaffected by background changes.
@article{yang2025layeranimate,
author = {Yang, Yuxue and Fan, Lue and Lin, Zuzeng and Wang, Feng and Zhang, Zhaoxiang},
title = {LayerAnimate: Layer-specific Control for Animation},
journal = {arXiv preprint arXiv:2501.08295},
year = {2025},
}