New, easier way to make deepfakes emerges from Russia

Russian scientists are claiming advances in creating deepfakes that could make it easier to produce surprisingly believable avatars.

Their techniques seem to create a high resolution deepfake video of a portrait using almost anyone as a motion model, or driver, and a single frame of the image to fake.

The researchers are from the Samsung AI Center in Moscow; the Skolkovo Institute of Science and Technology, also in Moscow; and YandexCommenta Russian Internet service company.

They describe their innovation as “the unique creation of high-resolution human avatars, called megapixel portraits, or MegaPortraits.” It is a set of new neural architectures and drives that stream high-resolution video.

It’s eye-catching, but by no means undetectable.

In short videos, pilots with little obvious resemblance to the source image to be animated look up, down, left, and right; words of mouth; smile and nod your head.

The result is, for example, an animated Mona Lisa and the American actor Brad Pitt which are generated in real time. The movement of the deepfake is as smooth as that of the driver. Realistic sparkles in the eyes and facial features like high cheekbones (in source Angelina Jolie) that are subtle or non-existent in the source image are particularly noteworthy.

In a video accompanying their research paper, the team praises the realism they create around the eyes, providing a strong approximation of authentic emotion – and they are right to do so.

Clothes and shoulders, they admit, need work. The shoulders resemble those seen on the puppets and the clothes float and ripple unrealistically.

The backgrounds also need work, which is odd. The algorithms create angles and features that are not in the source material, so when an avatar’s head moves it should be simple enough to generate realistic features. Instead, a mottled gray shadow forms and freezes in place.

Proportionality also suffers. Pitt and Jolie’s faces are bulky and rubbery around the chin and forehead. And in a few deepfake portraits, the white-white teeth don’t move side-by-side with the drivers’ head movements.

Each deepfake sample lasts a few seconds and is silent, limitations that are unlikely to remain for long.

Article topics

AI | biometrics | fake fake | research and development | Russia | Samsung

About Geraldine Higgins

Check Also

Building relationships in a post-pandemic world

The research you described above highlights the positive impact that well-designed team structures can have …