Last autumn, hundreds of computer science specialists gathered at a conference in Denver. Graduate student Tran Tran decided to test their attentiveness: he showed the audience a video in which the real Brad Pitt appeared alongside a version created by artificial intelligence. Only half of the viewers could tell the fake from the original. "Models that generate deepfakes are becoming very strong," Tran stated.
A deepfake is a video in which a person's face or voice is replaced using AI. Such clips are already being used by fraudsters: a case is known in Hong Kong where an employee transferred 25 million dollars after believing a fake video featuring the company's executive. Millions of people remain vulnerable when they join video calls. Popular services - Zoom, Apple FaceTime, Google Meet and Microsoft Teams - cannot yet detect deepfakes in real time, although Microsoft recently announced it had begun work in this area. Existing detection systems require enormous computing power, so they are expensive and scale poorly.
Tran's team, led by Professor Sangtae Ha, asked whether deepfake detection could be simplified enough to make it widespread. "Some considered this impossible because of system requirements and the current architecture," Ha says. "But if you look at the system from a different angle, the problem can be solved."
The researchers found that detecting a deepfake does not require reconstructing the video at full resolution. Video is transmitted over the internet in compressed form, and existing detectors first decompress each frame, which consumes a lot of resources. The new method analyzes already compressed data. During video compression, not all pixels are described, but rather the changes relative to the previous frame. If a wall in the background barely changes, it is encoded in large blocks, while a face and its facial expressions are encoded in many small ones. When a person's appearance changes sharply, the compression captures a lot of new information. The scientists measure how much of this new data falls on the face and how widely it is distributed. The video is not decompressed. Only if something looks suspicious does the detector request a full fresh frame from the sender's device and check it using AI. "We looked at data from only one frame out of a hundred instead of every one," Ha explains.
On a library of 1,000 online videos with deepfakes, the system showed almost perfect accuracy while using 143 times fewer computing resources than existing models. The results were presented at the SIGCOMM conference in August. Co-author Shirin Ebadi notes: "The most exciting thing is doing what has never been done." The team plans to adapt the system for large-scale video conferences. The scientists expect that demand for such technologies will grow as deepfakes spread.