HeyGen: The Danger of the New Miracle Translator | EUROtoday

Stand: 13.09.2023 | Reading time: 3 minutes

EUS tech blogger and recreation programmer Jon Finger really would not converse German. But in a video he printed on So all I have to do is drop it…” – and only the translation of the word “drop” as “to let fall” unmasks the computer. Because it is not Finger who speaks, but a perfect digital copy of his voice, which reproduces a text that was previously recorded in English in a language that is foreign to him.

The basis is a translation program based on artificial intelligence (AI) from the start-up HeyGen. The software not only adjusts Finger’s voice, but also her entire face. The video algorithm manipulates the tech blogger’s image so perfectly that the lip movements match the German words exactly.

The software not only speaks German, but also French and Spanish. In the future, it could take over the work of translation studios and voice actors worldwide. The only catch: the translation doesn’t yet work in real time.

Users first have to record and upload videos, then they can download the translated versions on the HeyGen website. But given the rapid technical progress, this seems to be just a question of time and available computing power.

Digital avatars, independent of their human counterparts

Finger is enthusiastic, as are his viewers. The conclusion of the comments is: The HeyGen Translator could make it extremely easy for YouTubers and independent filmmakers to reach new target groups worldwide. In the future, they would also be able to easily create content without even sitting in front of the camera themselves.

additionally learn

Because that’s exactly what HeyGen is promoting: Once the software has gotten to know a person using a template, it can then have them speak any text in any language – the digital avatars become independent of their human role models.

Influencers could multiply their work performance because they no longer have to speak texts themselves. Companies could individualize advertising messages, for example, address customers in a personalized way using digitized celebrity avatars. The algorithms remove the connection between the speaker and what is spoken.

More about AI applications

Stocks crash after hype?

The demo sheds light on an industry that is affected like no other by the boom in artificial intelligence: translation work is currently increasingly being carried out by software that can work faster and more flexibly than human translators.

A final frontier is simultaneous translation – for now. Internet giant Google had already presented a program called “Universal Translator” at its “I/O” developer conference in May that can translate live in lip-sync and perfectly imitate the tone of the speaker’s voice.

But James Manyika, head of Google’s “Technology and Society” ethics department, identified the real problem behind the combination of audio and video algorithms in May: Of course, a user could also load a video of a politician into the machine, and then the text specify that the system should generate. Instead of acting as a translator, the software would then work as a perfect propaganda tool; with suitable fakes, populists could cause an uproar on social networks.

additionally learn

This is also why Manyika warned at the launch against simply making AI tools such as the “Universal Translator” available online. You have to weigh up “courage and security”. Google is therefore currently only making its translator available to selected corporate customers.

Various other corporations and AI start-ups are currently proving that simultaneous, simultaneous translation is fundamentally possible with their own translation programs. But so that the translation works synchronously and does not overwhelm the computing power of mobile devices, these programs do without video manipulation and only output appropriate audio or text. Prominent examples are the Microsoft Translator and the live translation on Google’s Pixel phones.