South Korea’s Women Web Sleuths Fight AI Deepfake Porn

Under banners emblazoned "Solidarity" and "We stand together with victims", a group of South Korean women digital detectives dredges the internet for a scourge of illegal AI-enabled pornography.

South Korea -- home to the world's leading smartphone and memory chipmakers -- takes huge national pride in its tech prowess.

But the success story has a dark side: women suffer rampant "digital sex crimes", rights groups say. Spycams film them in private while chat groups blackmail them for explicit content and share so-called "revenge porn" without consent.

AI-generated deepfakes are the newest threat. Using the generative technology and a picture of their victim's face, harassers can fabricate invasive pornographic images portraying their targets.

At the National Center for Digital Sexual Crime Response -- housed in an airy Seoul office -- 23 women are tasked with turning the tide, trawling the web and filing delete requests backed by threats of legal action.

Last year, the group pulled down 318000 videos and photos -- deepfakes, spycam content and intimate imagery shared without consent -- on behalf of 10,600 people, 75 percent of whom were women and girls.

"South Korea acted faster than others because not only digital sex crimes became common earlier than elsewhere, but also women spoke out against them just as quickly and vigorously," Kim Mi-soon, the head of the government-founded centre, told AFP.

- Deepfake deluge -

The centre opened in 2018, a year that saw tens of thousands of women demand action through anti-spycam protests.

The organisation's headquarters and 17 regional hubs also offer counselling and legal advice and collect evidence for prosecutions.

One 33-year-old woman, who asked to be identified only by her surname Kim, said she had deepfakes shared in 2024 on encrypted apps beyond the reach of the centre's specialists.

"The centre may not be perfect, but it can make a huge difference for victims going through this nightmare alone," she said.

Kim, a member of the military, said she felt "healed" by their support -- including financial aid to secure a trauma diagnosis for court evidence.

"It gave me a huge sense of relief and solace," she said.

"Navigating the whole legal procedure alone while suffering trauma would have been impossible."

Digital sex crimes are proving a mutable menace.

In 2020, the "Nth Room" scandal revealed men were exploiting dozens of women and girls they called "slaves" in pay-to-view "sextortion" chatrooms.

Since the 46-person national response centre was founded, sexual deepfakes have become the new trend.

Between 2019 and 2023, global deepfake circulation surged 550 percent, according to web safety firm Security Hero.

Nearly 98 percent of content was AI-generated pornography of women, with more than half the victims in South Korean, according to company research.

K-pop stars are main targets, but ordinary women are increasingly ensnared in a country where police data show digital sex crimes account for 40 percent of all sexual offences.

"Since last year, we've seen more and more deepfakes of ordinary women and girls -- with the images becoming way more sophisticated than before and victims getting younger and younger," said deletion specialist Kim Yun-hee.

The centre uses its own AI tools, including a monitoring system capable of detecting exploitative material and messages targeting children.

"This job is not easy," the 31-year-old told AFP. "But it feels really rewarding whenever our clients say our service is a true life-saver."

- 'Leading the way' -

Deepfake pornography is illegal in South Korea, with possession punishable by up to three years in prison and creators facing up to seven years behind bars.

Penalties were increased in 2024, the year a series of Telegram deepfake chatrooms were found to be sharing AI-generated sexualised footage of female acquaintances, classmates, colleagues and relatives, sometimes created from photos taken from victims' social media accounts.

Clare McGlynn, law professor at Britain's Durham University and an expert in image-based sexual abuse, said Seoul is "leading the way on providing holistic support services and investing in the latest technology to try to reduce the harm".

South Korea's approach offers "an example for the rest of the world", she told AFP.

Yet campaigners warn the problem's root cause is a patriarchal culture that regards women as sexual objects undeserving of dignity.

Kim Yeo-jin, head of the nonprofit Korea Cyber Sexual Violence Response Center, told AFP the online problem needs an offline solution.

"Unless we tackle this fundamental cause by fostering more gender equality in wider society and strengthening gender equality education at school, no amount of removal efforts will be enough," she said.
-AFP