Animate Anyone is a leap forward in image-to-video: Here’s what it means | Technology News
Even as the world is coming to terms with deepfakes and the havoc it can wreak, now, new technology is on the fore that can animate almost anything from an image. Animate Anyone is a project spearheaded by Chinese multinational conglomerate Alibaba Group’s Institute for Intelligent Computing. The technology can be used to animate any image to make it move in whatever way one wants it to.
The new capability is unprecedented as now a simple image can be animated in whatever way one intends. This is particularly concerning, as with most videos on the internet, users tend to overlook the details that may indicate that it’s a fake visual. Experts have been suggesting that if not used thoughtfully, the model can create more trouble.
“Character Animation is a task to animate source character images into realistic videos according to desired posture sequences, which has many potential applications such as
online retail, entertainment videos, artistic creation and virtual character,” read the opening lines of the pre-print research paper, ‘Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation’.
How does it work?
A model like Animate Anyone works by extracting details such as patterns, facial features and even the pose of the person in the image. Based on these details, a series of images is created where the details are mapped onto various poses. These are later motion-captured or extracted from another video. Although some of the earlier similar models looked to be capable of this, they were not without their issues.
Reportedly, hallucination was one of the problems, as the model was expected to create minute details such as the movement of hair or clothes when a person is in motion. The complexity of this process led to strange visuals leading to videos that were unconvincing. Regardless of the early limitations, Animate Anyone seems to be an improved model, however, it is not all that perfect, at least not yet.
According to the researchers, ‘Animate Anyone serves as a foundational method, with the potential for future extension into various image-to-video applications’.