Facebook developers learn to control the movements of people recorded on video

Neural networks come up with increasingly more surprises. Artificial intelligence’ rapid progress allows people to do truly unimaginable things, for example control a video character’s movements.

The Facebook AI Research team has presented their new project: they taught artificial intelligence to identify a video character, even if he or she is in motion. This enables to change the surrounding background and the nature of the person’s movements.

The technology is based on Pose2Pose and Pose2Frame neural network algorithms required to transfer objects from one video recording to another. DensePose software was also used to convert 2D images to 3D.

First, the neural network creates a 3D model of the right character, and then converts the recording with the animation into a black-background video. At this stage, it’s already possible to change the character’s movements using a conventional mouse. Then you can restore the original background or add a new one.

For more clarity, the developers have uploaded a video clip demonstrating AI capabilities.

https://www.youtube.com/watch?time_continue=152&v=sNp6HskavBE