Which actor is irreplaceable in the Avengers

Deepfake technique: why real actors are still irreplaceable

It was one of the goose bumps in the movie "Rogue One: A Star Wars Story". When towards the end of the film none other than the young Carrie Fisher alias Princess Leia took over the stolen Death Star plans and thus laid the bridge to "Episode IV: A New Hope". Admittedly, the now deceased Fischer was no longer young at that time, but 50 years old. Using motion capture, however, it was brought onto the screen as a CGI animation (Computer Generated Imagery).

No job anxiety necessary yet

Such digital acting interludes are established nowadays. However, they are sometimes misused. For example, US actress Scarlett Johansson complained earlier this year that the fight against deepfake porn was pointless. With the help of artificial intelligence, the faces of individual people are deceptively realistically combined with another body and their facial expressions are also imitated.

Johansson may also have to fear other consequences of the burgeoning deepfake technology. The question arises as to whether technological progress will make the acting profession superfluous in the future, and whether at some point there will be nothing but virtual people scurrying around on the screen.

"That is still a long way off," says Darren Hendler to the Guardian. Hendler heads the "Digital Human" department of the special effects company "Digital Domain". The US company contributed special effects for "Star Trek: Nemisis" (2013) and Transformers, among others. "The use of artificial intelligence has spread quickly," says Hendler. "But the further development is difficult to predict".

Motion capture

Blockbusters have already been bursting with digital effects. In films like "Avengers: Endgame", many locations were created entirely digitally. Well, an invented planet is difficult to represent in any other way. But at least the actors stood for authenticity. Actor Josh Brolin gave film villain Thanos life. The entire sequence of movements as well as the facial expressions and gestures of Brolin are recorded using motion capture and then transferred to Thanos on the computer.

Behind every digital person there is a strong performance by a real person, says Hendler. "Somebody has to play the role of the deceased, for example. Somebody really has to study their movements and know facial expressions." Because these motion caption actors breathe life into their characters. Handler describes this effect as a "digital prosthesis" - a suit that a person has to wear.

AI help

While such digital characters are dependent on acting performance, so-called virtual people can act more autonomously. Here, the artificial intelligence generates speech and facial expressions itself after it has been fed with appropriate data or templates.

The virtual reality game for the BBC series "Peaky Blinders" also relies on such AI characters. Ian Hambleton, CEO of the developer company Maze Theory, explains a positive consequence of this creation, which now reacts differently to different emotions: "If the player encounters these AI characters aggressively, they change their behavior, including their body language and facial expressions."

Voice as a hurdle

Nobody believes that actors will soon be completely replaced. Nor is Arno Hartholt, director of research at the Institute for Creative Technology at the University of Southern California. Accordingly, one of the difficulties is to create a completely new one from a collection of past performances by an actor. The voice is also difficult to replace. "You need a lot of templates for that. How does it sound angry, like a combination of anger and vulnerability and what if someone is out of breath?" So here the human component is still at an advantage.

Hendler believes, however, that this technology is becoming more and more accessible to ordinary consumers. "Perhaps in ten years you will have a huge screen hanging on the wall where you can interact with a virtual assistant. He will then accompany you to bed or to the refrigerator. That may sound creepy, but it is on the rise" (red, 4.1.2020 )