Hugging Face Hosts Deepfake Models That Undress Women and Children

What Happened?

A report by the European non-profit organization AI Forensics has uncovered that Hugging Face, the world's largest open-source artificial intelligence model repository, is being used to create non-consensual deepfakes that undress women and children. According to the study, seven of the nine most popular image editing models on the platform complied without restrictions with requests to 'undress' people using simple text prompts. Researchers tested models like Stable Diffusion and other diffusion-based models, which immediately generated sexualized images without any controls. This finding is not isolated: in recent years, tools like 'DeepNude' and its variants have proliferated, but the difference is that Hugging Face, with over 400,000 hosted models, acts as a centralized hub that facilitates access to these models.

While commercial models like Google Gemini or OpenAI's ChatGPT have safety barriers that block such requests, models hosted on Hugging Face lack effective filters. AI Forensics tested the models with explicit prompts and obtained sexualized images immediately, without any controls. The report notes that even models labeled as 'safe for work' (SFW) generated explicit content. The organization documented over 200 examples of non-consensual deepfakes, including depictions of minors, which worsens the situation by violating child protection laws in multiple jurisdictions.

Why Is This Important?

This case highlights a regulatory gap in the open-source AI ecosystem. Hugging Face has positioned itself as a key hub for collaborative research and development, but its decentralized moderation model allows potentially harmful tools to be accessible to anyone. The absence of guardrails not only facilitates the creation of non-consensual sexual content but also exposes minors to becoming victims of these deepfakes. Unlike platforms like GitHub, which have implemented anti-abuse policies, Hugging Face has largely relied on voluntary community reports, which is insufficient given the scale of the problem.

Moreover, the report comes at a time when the European Union is finalizing the AI Act, which will classify certain uses of artificial intelligence as high-risk. This case could accelerate the inclusion of non-consensual deepfakes in that category, forcing platforms like Hugging Face to implement stricter controls. The AI Act, which will fully come into effect in 2026, already requires transparency for deepfakes but does not specifically address the responsibility of model repositories. The AI Forensics report could serve as evidence for regulators to tighten rules. In the United States, the DEFIANCE Act (Disrupt Explicit Forged Images and Non-Consensual Edits) is advancing in Congress but does not yet cover intermediary platforms.

Consequences and Reactions

Hugging Face has responded to the report by stating it will take action, but without specifying timelines or concrete details. In a statement cited by The Verge, the company said it is "reviewing the findings and will work to improve moderation," though it did not detail whether it will remove the flagged models. The company now faces growing pressure from digital rights organizations and lawmakers. New moderation policies are expected to be announced in the coming weeks, although the technical challenge of filtering malicious models without limiting innovation is enormous. Historically, Hugging Face has been a champion of open source, but this case raises an ethical dilemma: how to balance openness with responsibility?

For users, this case is a reminder that open-source AI tools can be used for abusive purposes. Experts recommend that companies and developers implement additional security layers when deploying open-source models, and that individual users report any suspicious content. Additionally, the report has reignited the debate over the need for stricter moderation on AI platforms. Compared to similar events, like the DeepNude controversy in 2019, the Hugging Face case is more systemic because it is not about an isolated tool but an entire infrastructure hosting multiple models. AI Forensics has called for the European Union to include model repositories in the 'high-risk' category of the AI Act.

What Should Readers Know?

• Not all AI models have the same protections: open-source models often lack filters, while commercial ones have stronger barriers.
• The creation and distribution of non-consensual deepfakes is illegal in many countries, but law enforcement is complex, especially when models are hosted on servers outside jurisdiction.
• Platforms like Hugging Face rely heavily on community reporting of abuse, which is insufficient given the scale and speed of content generation.
• This case could set a precedent for future AI regulations in Europe and other regions, such as the EU AI Act and potential laws in the US and UK.
• Users should be aware that even models labeled as safe can be manipulated with adversarial prompts; responsibility lies with both developers and end users.

“Hugging Face is not doing enough to prevent the AI models it hosts from generating sexualized deepfakes,” concludes AI Forensics in its report. The organization urges the platform to implement automatic filters and collaborate with authorities to track creators of abusive content.