Put Myself in Your Shoes: Lifting the Egocentric Perspective from
  Exocentric Videos
Mi Luo Zihui Xue Alex Dimakis Kristen Grauman
We investigate exocentric-to-egocentric cross-view translation, which aims to
generate a first-person (egocentric) view of an actor based on a video
recording that captures the actor from a third-person (exocentric) perspective.
To this end, we propose a generative framework called Exo2Ego that decouples
the translation process into two stages: high-level structure transformation,
which explicitly encourages cross-view correspondence between exocentric and
egocentric views, and a diffusion-based pixel-level hallucination, which
incorporates a hand layout prior to enhance the fidelity of the generated
egocentric view. To pave the way for future advancements in this field, we
curate a comprehensive exo-to-ego cross-view translation benchmark. It consists
of a diverse collection of synchronized ego-exo tabletop activity video pairs
sourced from three public datasets: H2O, Aria Pilot, and Assembly101. The
experimental results validate that Exo2Ego delivers photorealistic video
results with clear hand manipulation details and outperforms several baselines
in terms of both synthesis quality and generalization ability to new actions.