assets/README_syn.md

## MotionPro-Dense Video Generation Pipeline


This document provides an introduction to the MotionPro-Dense video generation pipeline, detailing its functionality and workflow. The pipeline is illustrated in the diagram below.

### Pipeline Description

1. **Video Generation with Base Motion Control**
   - First, MotionPro can be used to generate videos with controllable object motion and camera motion.
   
2. **Optical Flow and Visibility Mask Extraction and Merging**  
   - The generated videos are processed using CoTracker, a tool for extracting optical flow and visibility masks for each frame.  
   - The extracted optical flows are accumulated through summation.  
   - The per-frame visibility masks from both sequences are intersected to obtain the final visibility mask.

3. **Final Video Generation with Combined Motions**  
   - The aggregated motion conditions are used as input for **MotionPro-Dense**, which generates the final video with seamlessly integrated object and camera motions. 


<div align="center">
  <video controls autoplay loop muted playsinline src="https://cdn-uploads.huggingface.co/production/uploads/6496f5754a3c31df8e3139f6/jXx075-JEujYHucnClZ47.mp4"></video>
  <p><em>Figure 1: Illustration of video generation with combined motions.</em></p>
</div>


### Synchronized Video Generation

Additionally, the pipeline enables the generation of **synchronized videos**, where a consistent object motion is paired with different camera motions. 


<div align="center">
  <video controls autoplay loop muted playsinline src="https://cdn-uploads.huggingface.co/production/uploads/6496f5754a3c31df8e3139f6/qNreuc2zuKzMFUn2Y0DfK.mp4"></video>
  </video>
  <p><em>Figure 2: Illustration of synchronized video generation.</em></p>
</div>