Ramayana English trailer: Namit Malhotra DECODES DNEG’s AI-powered lip-sync technology!
Here's the latest update from the Bollywood world:
Unlike conventional dubbed versions, where a new voice track is placed over the original footage, the English version of Ramayana: Part 1 uses DNEG’s AI-powered technology to digitally adjust the actors’ mouth movements. Producer Namit Malhotra has now explained how the process combines human performances with technology to achieve a more natural result.
Adding to this, In conversation with ET Now, Malhotra stressed that the process did not begin with AI. According to the producer, the first step involved human voice performances, with members of the principal cast also contributing their own English voices.
Meanwhile, “If you've seen that, where we've got tremendous praise, where we've actually brought in human actors, voice actors, who actually come in and performed the voice performances. Some of them actually are done by the actors themselves, who have a good command over the language and are able to emote appropriately in a different language,” he shared.
Notably, Malhotra explained that this human-led approach was important to preserve the emotional quality of the performances. “So we've done that as a first step to make sure that the rendering of that expression and emotion is authentic, and it's led by humans. There is no AI whatsoever,” he added.
As per the latest buzz, The next stage involved creating the appropriate lip movements for the English dialogue. Malhotra shared human artists first established the foundation for how the characters’ lips should move in sync with the new voice performances.
In further updates, “There is no AI whatsoever. We followed that up with what we're calling, because we are trying to make the lip sync in line with the voice. So we have a next step, which is again done by humans, where you actually have people coming in, what we call a human driver that actually creates this foundation of how the lips actually move and how it syncs itself to the voice itself,” he explained.
On top of that, According to Malhotra, the challenge was not simply matching a voice to an star. The team also had to ensure that the voice performance, facial appearance and lip movements worked together convincingly.
“And it looks appropriate because the best voice star may not have the right face for the star on screen. So we are making sure that you've got the voice actors, we've got people whose faces and lips match the look of the actors,” Malhotra shared.
Source: Bollywood Hungama