EU to Ban Nudifying Apps, but the Real Problem Lies in Hugging Face

What happened?

The European Union is close to approving a ban on applications that allow digitally 'nudifying' people without their consent. However, a report by the European NGO AI Forensics warns that this measure will not address the root of the problem. The tools that perform this manipulation do not require being apps; they are available as open-source models on platforms like Hugging Face. The report, published in July 2025, analyzed the most popular image editing tools on Hugging Face and found that seven of them can generate non-consensual deepfakes. These models can be downloaded, modified, and run locally, making them difficult to track and regulate. According to AI Forensics, these models have been downloaded thousands of times, and some are specifically designed to remove clothing from images, constituting a serious violation of privacy and consent.

The current regulatory context focuses on commercial applications, such as those found in app stores, which would be outlawed under the new European directive. However, open-source models on Hugging Face are not covered by this legislation, creating a significant legal loophole. The NGO points out that the EU should broaden its approach to also include regulation of base models and open-source repositories, as these are the true enablers of non-consensual deepfake creation.

Why is it important?

The ban on nudifying apps is a positive step, but insufficient. Hugging Face has become the central repository for open-source AI models, allowing anyone to access image manipulation tools without oversight. This creates a legal vacuum: while commercial apps will be illegal, individuals can continue creating deepfakes using open models. The problem is not new: in 2023, similar models were already detected on GitHub and other repositories. The difference is that Hugging Face has been slower to implement controls. The EU must consider regulating base models and open-source repositories as well.

The societal impact is profound. Non-consensual deepfakes, especially those that sexualize people without their permission, cause severe psychological and reputational harm. A 2024 study by Stanford University found that 90% of online deepfakes are pornographic in nature, and most are non-consensual. The ease of access to open-source models exacerbates this problem, as anyone with basic technical knowledge can create and distribute this content undetected. Moreover, the decentralization of open source complicates law enforcement, as models can be hosted on multiple servers and jurisdictions.

Consequences

• For users: 'Nudifying' apps will disappear from official stores, but non-consensual deepfakes will continue to circulate, created with open-source tools. Users, especially women and minors, remain at risk of being victims of this technology. Awareness and digital education are key to mitigating harm.
• For companies: Hugging Face could face regulatory pressure to moderate models, affecting the entire open AI community. The company has already implemented content filters, but according to AI Forensics, these are easily bypassed. If the EU demands stricter moderation, Hugging Face may have to remove popular models, creating tensions with the open-source community.
• For the market: Asymmetric regulation could stifle innovation in responsible AI, while malicious actors adapt quickly. Startups developing ethical AI could be harmed if regulations are too restrictive, while malicious model developers will simply migrate to less regulated platforms. This could lead to fragmentation of the open AI ecosystem.

What should readers know?

The ban on apps is just the first step. The real battle lies in the governance of open AI models. Hugging Face has already begun implementing filters, but effectiveness is limited. Users should be aware that any image can be manipulated and verify sources. Tech companies must collaborate with regulators to create transparency standards for AI models. Additionally, it is crucial that citizens demand broader regulation that includes open source, and that research into deepfake detection techniques is promoted.

“Banning apps is like closing the door while the window is open,” warns the AI Forensics report.

Historical context

This case recalls the fight against deepfakes in 2019, when swapped-face videos became popular. At that time, platforms like Reddit banned such content, but open-source models remained available. Now, the EU attempts a broader approach, but the decentralized nature of open source complicates regulation. In 2023, GitHub removed several repositories containing nudifying tools, but models resurfaced on other platforms. The difference with Hugging Face is its central role in the AI ecosystem; it hosts over 500,000 models and is used by researchers and companies worldwide. Regulating it could set an important precedent, but could also trigger a debate on research freedom and open access to AI.

Compared to regulation of other harmful content, such as hate speech or child sexual abuse material, the fight against non-consensual deepfakes faces unique challenges due to the speed of technological innovation and ease of anonymization. The EU has already passed the AI Act, classifying deepfakes as high-risk, but its practical implementation remains uncertain. The AI Forensics report underscores the need for complementary measures, such as requiring model repositories to implement identity verification systems and more effective abuse reporting mechanisms.

In conclusion, the ban on apps is progress, but not enough. The international community must address the problem holistically, involving platforms, developers, regulators, and users. Transparency, education, and cooperation will be key to closing the window that non-consensual deepfakes have opened.