Apple and Google Pressured to Remove Non-Consensual AI Nude Apps

What Happened?

On July 17, 2026, San Francisco District Attorney David Chiu sent cease and desist letters to Apple and Google, demanding the immediate removal of eight 'nudify' apps from their respective stores (App Store and Google Play). These apps use artificial intelligence to generate nude images from photos of real people without their consent. According to the DA's office, the apps violate state and local laws against non-consensual pornography and defamation. The named apps, whose names were not publicly disclosed, have reportedly been downloaded millions of times combined, according to the DA's estimates. The action is based on California Penal Code (PC 647.8), which prohibits the distribution of non-consensual intimate images, and a local San Francisco ordinance that criminalizes the creation of sexual deepfakes. David Chiu stated: 'These apps are abuse machines. They allow anyone to turn a real person into a sexual object without their permission, and the app stores profit from it.' This move follows a months-long investigation that included downloading and testing the apps by the DA's cybercrime unit, which found that many evaded content policies through misleading descriptions and false age ratings.

Why Is This Important?

This case represents a milestone in the fight against generative AI abuse. Unlike other isolated incidents, this involves commercially available apps that have been downloaded millions of times. The legal action targets not only developers but also the platforms distributing them, potentially setting a precedent for Apple and Google's responsibility in moderating AI-generated content. It also highlights the inadequacy of current app review policies in detecting malicious deepfake tools. Historically, app stores have dealt with apps violating their terms, but the rise of generative AI has multiplied the difficulty: 'nudify' apps can pass automated reviews by not containing explicit malicious code, and human reviews often fail to detect the real purpose until runtime behavior is analyzed. This case adds to recent others, such as a 2023 lawsuit by a New Jersey high school student against deepfake creators, or the FTC fine on an AI company for generating non-consensual images. However, the key difference here is the direct targeting of platform gatekeepers (Apple and Google), which could force systemic changes in AI app review.

Potential Consequences

If Apple and Google do not comply, they could face lawsuits and significant fines. The San Francisco DA has already warned it will initiate legal action if the apps are not removed within 30 days. Fines per violation could reach $10,000 per day under local law, and individual victims could also file civil lawsuits. Beyond this specific case, this pressure is expected to accelerate stricter regulations, such as the EU's Digital Services Act, which already requires platforms to assess systemic risks of their services, or similar bills in states like New York and Texas. In the market, this could affect investor confidence in AI startups operating in gray areas and increase compliance costs for big tech. For users, it means greater awareness of the risks of AI apps and the need for content verification tools, such as digital watermarking systems being developed by companies like Microsoft and Adobe. For tech companies, it implies greater investment in automated abuse detection systems and more transparency in their review processes. Apple has already invested in human review teams and machine learning, but this case shows gaps remain. Google, for its part, updated its AI-generated content policies in 2025, but 'nudify' apps have managed to evade them using obfuscation techniques.

What Readers Should Know

'Nudify' apps are not new, but AI has made them more accessible and realistic. Early cases date back to 2019 with apps like DeepNude, which was taken down after a viral controversy. However, since then, improved versions using diffusion models and generative adversarial networks (GANs) have proliferated, producing more convincing results. Although some stores explicitly prohibit non-consensual sexual content, these apps often disguise themselves as photo editing tools or entertainment, using keywords like 'body editor' or 'photo enhancer' to evade filters. Users should be wary of any app that promises to undress images, as its use can have serious legal and ethical consequences: in several US states, it is a misdemeanor or felony, and victims can sue for damages. Additionally, victims of these deepfakes have legal recourse, such as requesting removal under privacy laws like the CCPA in California, or filing lawsuits for defamation or violation of image rights. Organizations like the Cyber Civil Rights Initiative offer guides for removing non-consensual content. For app developers, this case is a warning: store policies are tightening, and lack of transparency in generative AI use can lead to permanent expulsion. In summary, San Francisco's action marks a before and after in platform responsibility regarding AI abuse, and we are likely to see more regulations and lawsuits in the near future.