2023 Author: Bryan Walter | [email protected]. Last modified: 2023-05-21 22:24
American developers have created an algorithm capable of animating human figures in two-dimensional images. He creates a three-dimensional model for the drawn person, and then reproduces the animation with the model running out of the picture. An article dedicated to development has been posted on arXiv.org.
By looking at a photograph, a person can usually imagine how objects in the frame moved after it was taken. For algorithms, this task is still quite difficult. So far, researchers are mainly engaged in the development of algorithms that can recreate only a small part of the movements of people in images. For example, last year, developers from Tel Aviv University and Facebook taught an algorithm to animate facial expressions in portraits.
Another group of developers from the University of Washington and Facebook, led by Ira Kemelmacher-Shlizerman, has created an algorithm that can create a full-fledged animation with a person running out of the photo from one 2D frame:
The system created by the researchers is a combination of several previously developed algorithms and its own code. Initially, it takes a two-dimensional image and processes it using the Mask R-CNN neural network. At this stage, the algorithm recognizes an area with a person in the image and separates it from the background. Then, another previously developed algorithm turns the image area with a person into a two-dimensional skeleton model, consisting of straight segments and their connections. After that, another algorithm creates a realistic background in areas of the frame that were originally hidden by a person.
