With more and more viewers tuning in to international television shows and movies on streaming,Israeli based company Adapt Entertainment has found a way for the programming to speak to everyone — literally.
The company’s founder, Darryl Marks, says fans often complain about how poorly a show or film is dubbed. Adapt’s technology combines AI and visual effects to seamlessly convert movie dialogue into English and other languages.
Most recently, the tech was used for the English transfer of Maciej Barczewski’s film “The Champion” — about a prizefighter who must win his matches to survive in Auschwitz — set for a U.S. release later this
year. Barczewski calls the technology a “game changer.” The original movie is in Polish and German; the director wanted to film in English but didn’t have the budget to do so.
Mike Seymour, the film’s VFX supervisor and Adapt’s technical adviser, explains the work and sync come after picture lock.
Typically, a rerecord is done by voice-over actors, but in this rare case, Barczewski was able to bring his cast back to rerecord the dialogue track in English. The track was then completely synced and processed to the original film, using AI neural rendering technology based on the software system PLATO (physics learning through auto-encoding and tracking objects).
Seymour explains the process: “The computer studies the face of the actor in the film and also footage filmed when the new dialogue is rerecorded. Then, using advances in AI and machine learning, the entire film has the actor’s face replaced speaking in English. It is all visually built from actors recording dialogue in a sound studio.”
With the combined efforts of AI and visual effects through PLATO, the subtext of the original performance is maintained. “We need to be faithful to their performance, not an arbitrary technical specification,” Seymour says.
The process for “The Champion” took a total of three months, including a week for the actors to
rerecord their dialogue tracks, which dramatically reduced the per-shot cost of large-scale dialogue replacement.
Seymour notes that the company is not setting out to replace subtitles. “Those are invaluable to the hearing-impaired community,” he says. Rather, Adapt aims to “open up a world of foreign films without taking the viewer out of the cinematic experience, and to do it in a way that was engineered to respect the acting choices of the cast, the intent of the scriptwriters and the creative vision of the director.”
