Apps That Make Nonconsensual AI Nudes Now Risk Removal From the App Store

Apple has explicitly required App Store apps to prevent and remove nonconsensual intimate images, including AI-generated pictures, giving developers a written duty backed by the threat of removal. ^[2][5]

The update, announced on October 16, 2026, applies through Apple's rules for apps that let users create or share content. Apple says apps designed to generate nudity or sexually explicit imagery, including tools that turn pictures of real people into explicit images without consent, were already prohibited. ^[2]

Our reading is that the duty reaches general-purpose image generators and face-swapping apps because they let users generate images, although Apple has not said how it will apply the rule to those tools. Apple says developers that lack adequate safeguards, or fail to fix violations after notice, risk removal from the App Store. ^[2][5]

That puts a more explicit obligation on developers after months of pressure over sexualized deepfakes made with Grok, the AI service developed by xAI, and over face-swapping apps with sexual features. Users used Grok earlier this year to make sexualized images that included minors, according to The Verge, which also reported that Apple did not remove either Grok or X from its store. ^[1]

A written rule gives Apple a standard against which to judge an app and gives people reporting abuse a specific obligation to point to. Its value will depend on whether Apple uses it before harmful tools have to be exposed by victims, researchers or public officials.

What developers must do

Section 1.2 of Apple's App Review Guidelines covers apps that let people create, generate, upload, share or otherwise contribute content. It requires filtering of objectionable material, a way to report offensive content, tools to block abusive users and published contact information.

Apps must also remove content that violates the guideline or their own rules. The updated language explicitly brings nonconsensual intimate imagery, including AI-generated imagery, within those requirements. ^[5]

The duty therefore has two parts: prevention and removal. A reporting button deals with material after someone encounters it. Apple's new language also requires developers to take steps to stop the material appearing in the first place. ^[2][5]

That matters for software with many legitimate uses. A face-swapping tool can change a face in an ordinary picture and also be used to manufacture an intimate image. In our reading, the requirement to prevent harmful content puts responsibility on the developer even when the app is presented as a general-purpose tool.

Apple has not specified which safeguards it will consider adequate, how reviewers will test features changed remotely after approval, or what would trigger removal of a large general-purpose platform. Its announcement also leaves open how quickly it will act on reports and whether it will identify apps rejected or removed under the rule. ^[2][5]

Apple separately clarified rule 2.5.19, which addresses apps that misrepresent their purpose during review and add features afterward. Such apps can lose both their App Store listing and their developer's membership in the Apple Developer Program. ^[2]

Those are different enforcement levers. One addresses what an app allows people to make and share. The other addresses a developer concealing what the app does.

The features behind the listing

The pressure on Apple became concrete in July, when San Francisco City Attorney David Chiu sent legal notices to Apple and Google demanding the removal of 13 face-swapping apps that his office said could produce nonconsensual nude images. Eight were available on Apple's App Store and five on Google Play, according to WIRED. ^[4]

Apple told WIRED it had removed three flagged apps and was terminating their developers' accounts. It said four other developers had to address policy violations or risk removal. ^[4]

The apps were broadly presented as face-swapping tools, while sexual deepfake features were available inside them, WIRED reported. That gap between the storefront description and the functions inside an app is precisely where a ban on pornography needs help from rules governing moderation and deceptive review practices. ^[4]

WIRED also described a Cornell and Georgetown preprint, a study shared before formal review by outside experts, whose researchers tested 155 out of 420 face-swapping apps. They found that 70 percent of the tested apps could produce nude face swaps without safety measures stopping them. That result concerns the tested group, rather than every face-swapping app or the App Store as a whole. ^[4]

Apple says it prevented more than 150,000 apps with safety issues from reaching customers last year and terminated more than 193,000 developer accounts for fraud and abuse. Those figures describe its broader enforcement work, rather than actions specifically against nonconsensual intimate imagery. ^[2]

Grok is the enforcement test

Apple's dealings with Grok show how enforcement can stop short of removal. According to NBC News, a letter Apple sent to senators in January said the company privately threatened to remove Grok after deciding that xAI had not done enough to prevent sexualized deepfakes. NBC reported that Apple later approved a Grok submission after further engagement and developer changes. ^[3]

The updated rules make the standard more explicit for the next such decision. An app's size should carry no weight in whether its safeguards meet that standard. A duty applied firmly to small developers but loosely to major platforms would be an uneven protection.

The distribution boundary also matters. Apple's announcement says some apps can reach users through alternative marketplaces or developers' websites. It specifically says apps containing material that facilitates human trafficking or child exploitation, including child sexual abuse material, will be removed from the App Store and blocked from alternative distribution.

It does not announce that same outside-store restriction for every violation involving adult nonconsensual imagery. ^[2]

The next test is an enforcement decision: whether Apple rejects an app, demands changes or removes it under the clarified duty to prevent and remove nonconsensual intimate images.