Cyberpunk 2077 Animated Lip-Sync Feature Coming

The action-packed RPG Cyberpunk 2077 will release just a few weeks later. The game will launch with support for dialogue in 10 languages and subtitle options. Also, CD Project Red is working to use artificial intelligence to lip-sync the dialogue in multiple languages. The game’s character technical lead director, Mateusz Popławski, said in a presentation that Cyberpunk 2077 animated lip-sync quality will be better than The Witcher 3. They wanted to do that in 10 different languages: English, German, Spanish, French, Italian, Polish, Brazilian Portuguese, Russian, Mandarin and Japanese. The studio wanted to do this for every character in the open world. So, they got in touch with Jali Research’s lip-syncing and facial animation tech to procedurally generate how the characters’ faces move.

JALI is a procedural tool that lists the animation up multiple categories. It includes character model, face shape, muscle movement, speech, speech style, eyes and brows to create more realistic and believable facial patterns during a speech. Some characters in Cyberpunk 2077 can speak multiple languages and switch between them in the same sentence. So, transcript tagging appeared. These tags also provided to adjust a character’s facial expressions when their emotional state changed within a dialogue. Plus, the system used audio analysis to mimic the emotions of vocal performance in the animations.

Cyberpunk 2077 Animated Lip-Sync

Additionally, CDPR used algorithms to animate the cutscenes in Witcher 3. Many games use motion capture to lip-sync dialogue in just one language to break the immersion a bit for players who switch to a different one. So, these algorithms are brilliant to experience. This work is generally completed by actors in motion capture rigs. But because of the coronavirus pandemic, this process cannot be used. Instead, JALI will help shape the NPCs of Cyberpunk 2077‘s world. Lastly, you can hear more about the animation process behind Cyberpunk 2077 in the video below.