Police launch internal probe into Telegram as deepfake perpetrators estimated in tens of thousands

Social Affairs

Police launch internal probe into Telegram as deepfake perpetrators estimated in tens of thousands

Published: 02 Sep. 2024, 19:12 Updated: 02 Sep. 2024, 19:40

Students show their phone screens displaying their Kakaotalk friend lists in Daegu on Monday. The friends on the lists all deleted their profile photos. [NEWS1]

Police launched an internal investigation into the messaging platform Telegram, which has been accused of aiding online sexual crimes involving deepfake technology, in a step aimed at capturing potentially tens of thousands of perpetrators.

Woo Jong-soo, head of the National Office of Investigation (NOI), said the Seoul Metropolitan Police Precinct initiated a preliminary probe to examine whether Telegram has been aiding the distribution of sexually explicit deepfake content, during a briefing with reporters on Monday.

Woo acknowledged the challenges involved in the investigation, given Telegram's longstanding history of “non-cooperation with various countries, including Korea.”

Since its launch in August 2013, Telegram has consistently refused to comply with investigations from multiple nations.

Despite these obstacles, Woo assured that the NOI is fully committed to the effort, employing its own investigation methods.

Woo also said the NOI would coordinate with international organizations as well as French authorities, which detained Telegram CEO Pavel Durov on Aug. 24 at an airport north of Paris for failing to take steps to curb the criminal use of Telegram.

The probe marks the first instance of Korean law enforcement targeting a foreign platform, reflecting the government's commitment to tackling the surge in digital sex crimes amid growing international concern.

Between Aug. 26 and 29, more than 88 reports related to deepfake sex crimes were filed, with 24 perpetrators identified. This marks a tenfold increase compared to the weekly average of 9.5 cases reported between January and July, according to the NOI.

Woo attributed this spike to more victims coming forward to report these crimes.

While law enforcement is implementing strong action against digital sex crimes and the public is actively engaged in finding perpetrators, some offenders remain undeterred, with some even sharing strategies for evading law enforcement on online communities.

A screen capture of the online community ″Deepfake Task Force for School Violence.″ Some users exchange their experiences of creating deepfake pornography on Telegram and share tips on avoiding police investigations and potential penalties. [SCREEN CAPTURE]

An online community named "Deepfake Task Force for School Violence” has become a space where users are exchanging experiences of creating deepfake pornography on Telegram and sharing tips on avoiding police investigations and potential penalties.

The forum, originally established in 2011 under a different name, rebranded itself on Aug. 28 following the public exposure of deepfake sex crimes. The community was initially intended as a space to discuss measures against deepfake pornography and address issues related to school violence.

As of Monday, the forum had approximately 21,000 members.

Posts in this forum include questions from high school students about the legal risks of creating deepfake content, with some users reassuring others that there are no legal grounds to punish someone for creating and viewing deepfake pornography.

Some were rather confident that they wouldn't face charges.

In a post titled "Why it's an overreaction to be frightened over this" regarding deepfake sex crimes, the writer mocked the law enforcement's ability, citing the difficulties police face in identifying chat room participants and proving their activities. The post has since been deleted as of Monday.

Another user dismissed the government's investigation, pointing out that Telegram has never complied with any probes and reassured others to "not worry about it and just live their lives."

Some users, who identified themselves as mothers, discussed their sons being accused of using deepfakes to sexually harass female friends and sought advice on how to handle the situation.

Chat rooms suspected of continuing sexual exploitation activities persist on Telegram. These groups often implement stricter identification processes for entry.

Experts attribute the ongoing prevalence of these crimes to weak penalties.

Lee Soo-jung, a forensic psychology professor at Kyonggi University in Seoul, told Yonhap News Monday that while laws related to digital sex crimes were revised following the notorious "Nth Room" and "Baksa Room" cases, “possessing or viewing [illegal] content whose distribution intent is not proven is still not subject to criminal charges.”

The "Nth Room" and "Baksa Room" were Telegram chat rooms used to sexually exploit dozens of women and young girls from 2018 to 2020.

Additionally, many offenders receive suspended sentences, which some do not view as a serious punishment, according to Lee.

Lee called for stronger regulations targeting platforms where sexual exploitation content is prevalent.

“Since 2022, many countries around Europe have been implementing laws that shift the responsibility to platforms. But such regulations are lacking in Korea, which currently relies primarily on monitoring,” she said.

BY WOO JI-WON [woo.jiwon@joongang.co.kr]