FlowHMR: Physically Plausible Motion Capture from Video

This research project tackles the problem of recovering realistic 3D human motion from single‑camera video. Many learning‑based methods struggle with depth ambiguity and regression collapse, which produce motions that look wrong and cannot be tracked with physics‑based controllers. To address this, we treat video‑based motion capture as a video‑conditioned motion‑generation task. First, we pre‑train a flow‑matching model that can generate a wide range of motion candidates from a monocular video. Then we apply post‑training optimization using Group Relative Policy Optimization (GRPO). GRPO is a reinforcement‑learning approach that uses two rewards: * a fidelity reward that keeps the generated motion aligned with the input video, and * a tracking reward that

favors motions a physics‑based controller can actually follow. Together, these steps push the generated distribution toward motions that are physically plausible, trackable, and visually accurate. The project also includes code for testing the model on standard motion‑capture datasets and for studying the trade‑off between physical realism and video consistency.

Message: Hi FE, I need help with “FlowHMR: Physically Plausible Motion Capture from Video” in “Electronics & Communication Engineering”