Artificial intelligence (AI) is rapidly changing the world, and its impact on protecting children against child sexual exploitation and abuse (CSEA) is profound and complex. Although AI can be harmful in the wrong hands, it offers powerful tools that help detect, prevent, and combat CSEA.
AI as a Safeguarding Tool
AI can sift through vast amounts of data, including textual and visual data and user behaviour, with incredible efficiency and accuracy. AI content moderation automatically assesses and filters user-generated content, removing explicit or illegal material. In the field of child safety, this is particularly useful as Machine Learning algorithms and image recognition technology can identify, categorise, and flag instances of abuse and Child Sexual Abuse Material (CSAM) with high accuracy and promptness.
Beyond content recognition, AI algorithms are effective in analysing dark web activity, uncovering and dismantling networks of exploitation and trafficking. AI is also used to analyse behavioural trends in online conversations to predict high-risk situations and identify patterns or linguistic cues that indicate grooming. This positions AI as essential for law enforcement and child protection agencies to focus their efforts, identify victims in real time, and enable early and targeted interventions or support services.
CESIUM and the Safer tool are AI-driven solutions that identify children vulnerable to exploitation and detect CSAM respectively. PhotoDNA then creates digital signatures of known CSAMs, making it easier to detect and remove them from online platforms. re-Think is a chatbot that engages with users trying to access CSAM, alerting them to the illegality of their actions and directing them to support resources.
AI’s advanced capabilities have enabled the generation of realistic digital content, including images and videos, which poses challenges for organisations working to protect children. In 2024, The Internet Watch Foundation (IWF) observed an increase of over 3,500 new AI-generated CSAM compared to last year’s 3,000 images. These AI-based detection tools need to also be able to detect AI-generated CSAM material.
Recommendations
A strong and coordinated response involving tech companies, law enforcement, governments, and child protection organisations is necessary to mitigate AI’s potential harm while maximising its benefits. This includes sharing data, developing common standards, regular monitoring, and creating accountability frameworks to ensure that AI is used ethically and safely. A crucial step is to implement ‘safety by design’, that is, prioritising child safety in the development of AI technologies by creating safeguards to prevent the creation and distribution of AI-generated CSAM.
Additionally, education and awareness are important. Parents, educators and children need to be well-informed about the potential risks and harms associated with AI. Digital literacy programmes that empower young people to recognise manipulative behaviour can promote the safe and responsible use of AI technologies. Ongoing research is essential to understand the evolving threat landscape, develop new tools and techniques to combat AI-facilitated abuse and ensure AI technologies are used ethically and effectively to protect children. The future of child protection in the digital age will depend on how well we manage this balance, making ethical AI development not just a priority, but a necessity.
English 
German