The Pose Knows: Video Forecasting by Generating Pose FuturesThe Pose Knows: Video Forecasting by Generating Pose FuturesWalker, Jacob and Marino, Kenneth and Gupta, Abhinav and Hebert, Martial2017
Video prediction with human objects
Instead of the common approach of predicting directly in pixel-space, use explicit knowledge of human motion space to predict the future of the video.
1. VAE to model the possible future movements of humans in the pose space
2. Conditional GAN - use pose information for to predict video in pixel space.