🔗 Share this article

Technology companies and child protection agencies will receive authority to evaluate whether AI tools can produce child exploitation images under recently introduced British legislation.

Substantial Rise in AI-Generated Illegal Content

The declaration came as revelations from a safety monitoring body showing that cases of AI-generated child sexual abuse material have increased dramatically in the past year, rising from 199 in 2024 to 426 in 2025.

New Legal Framework

Under the changes, the authorities will allow designated AI companies and child protection groups to inspect AI systems – the foundational technology for chatbots and visual AI tools – and verify they have adequate protective measures to prevent them from creating depictions of child exploitation.

"Ultimately about stopping exploitation before it happens," stated Kanishka Narayan, noting: "Specialists, under rigorous conditions, can now detect the risk in AI models early."

Addressing Regulatory Challenges

The changes have been implemented because it is against the law to create and possess CSAM, meaning that AI creators and other parties cannot generate such content as part of a evaluation process. Until now, officials had to wait until AI-generated CSAM was uploaded online before dealing with it.

This law is aimed at averting that problem by enabling to halt the production of those materials at their origin.

Legal Framework

The amendments are being added by the authorities as modifications to the crime and policing bill, which is also establishing a ban on owning, producing or distributing AI systems designed to generate child sexual abuse material.

Real-World Impact

This week, the official visited the London headquarters of Childline and listened to a mock-up conversation to counsellors involving a account of AI-based exploitation. The call portrayed a teenager requesting help after being blackmailed using a sexualised deepfake of himself, created using AI.

"When I learn about young people facing extortion online, it is a source of intense frustration in me and rightful anger amongst families," he said.

Concerning Data

A prominent online safety foundation stated that cases of AI-generated abuse material – such as webpages that may contain multiple files – had significantly increased so far this year.

Cases of category A material – the most serious form of abuse – increased from 2,621 images or videos to 3,086.

• Girls were predominantly victimized, accounting for 94% of prohibited AI depictions in 2025
• Depictions of infants to two-year-olds increased from five in 2024 to 92 in 2025

Industry Response

The law change could "constitute a vital step to ensure AI tools are secure before they are launched," stated the chief executive of the online safety organization.

"AI tools have enabled so survivors can be victimised all over again with just a few clicks, providing offenders the ability to create potentially limitless quantities of sophisticated, lifelike child sexual abuse material," she added. "Material which additionally commodifies victims' suffering, and renders young people, particularly girls, more vulnerable on and off line."

Support Session Information

The children's helpline also published information of support sessions where AI has been mentioned. AI-related risks discussed in the sessions include:

• Employing AI to rate body size, body and appearance
• Chatbots discouraging young people from consulting safe guardians about harm
• Being bullied online with AI-generated material
• Online extortion using AI-manipulated images

During April and September this year, Childline delivered 367 support interactions where AI, chatbots and related topics were discussed, significantly more as many as in the same period last year.

Half of the references of AI in the 2025 interactions were connected with mental health and wellbeing, encompassing using AI assistants for support and AI therapeutic applications.