SAN FRANCISCO – Discord’s head of trust and safety said Tuesday that the popular chat platform was changing and clarifying its child safety policies, including those related to teen dating and AI-generated child sexual abuse material, a announcement that comes after an NBC News investigation last month. on the safety of children on the platform.
John Redgrave, Discord’s vice president of trust and security, said that Discord was expanding its policies to address generative artificial intelligence. that can create fake content and the sexualization of children, specifically banning AI’s depictions of child sexual abuse and even the sexualization of children in text chats. the washington post reported in June that AI-generated images of child sex have proliferated on the internet in recent months.
Discord has been a hub for communities dedicated to creating AI generative images and has hosted several integrations that allow users to generate them. Sexually themed images are frequently created on those servers.
The company said in a blog post announcing the changes that the updated child sexual abuse material policy would include “any text or media that sexualizes children, including photorealistically drawn, photorealistic, and AI-generated child sexual abuse material. The goal of this update is to ensure that the sexualization of children in any context is not normalized by bad actors.”
Redgrave also said the company was instituting policy changes and clarifications to explicitly ban teen dating, which experts previously told NBC News represented a significant opportunity for adults seeking to exploit or groom children.
Discord wrote in his blog post: “In this context, we also believe that online dating can put oneself in danger. According to this policy, teen dating servers are prohibited on the platform and we will take action against users who behave in this way.”
Redgrave said in a presentation at TrustCon, a conference for trust and safety professionals held in San Francisco, that the company saw such online relationships as a significant risk for young people.
“We will no longer allow teen dating on our platform because we recognize that it is a substantial harm vector for predators to go after teens,” he said.
Discord Guidelines He had already said that the company would “eliminate spaces that encourage or facilitate dating among adolescents.” In June, an NBC News investigation found hundreds of Discord servers that appeared to promote child abuse material and some servers that advertised themselves as dating servers for teens or children requesting nude images of minors. Additionally, NBC News identified 35 cases over the past six years in which adults were prosecuted on charges of kidnapping, grooming or sexual assault that allegedly involved communications on Discord.
NBC News identified an additional 165 cases, including four alleged criminal networks in which adults were prosecuted for transmitting or receiving child sexual abuse material through Discord, or for allegedly using the platform to extort children to send sexually graphic images. themselves, also known as sextortion. .
Redgrave noted that the company was also updating its policies to prohibit cases of older teens grooming younger teens. In the blog post, the company said: “Older teens who engage in grooming of a younger teen will be reviewed and acted upon under our Child Sexual Misconduct and Grooming Policy.”
As part of its policy updates, Discord announced the release of more parental control tools. in a new Family Center Toolparents and children can opt-in for parents to receive updates about their children’s activities on the platform.
