




Furthermore, Apple also plans to update Siri and Search to stage an intervention when users try to perform searches for CSAM-related topics, alerting that the "interest in this topic is harmful and problematic."

"Messages uses on-device machine learning to analyze image attachments and determine if a photo is sexually explicit," Apple noted. "The feature is designed so that Apple does not get access to the messages." The feature, called Communication Safety, is said to be an opt-in setting that must be enabled by parents through the Family Sharing feature.

How Child Sexual Abuse Material is Detected

Detection of known CSAM images involves carrying out on-device matching using a database of known CSAM image hashes provided by the National Center for Missing and Exploited Children (NCMEC) and other child safety organizations before the photos are uploaded to the cloud. "NeuralHash," as the system is called, is powered by a cryptographic technology known as private set intersection. However, it's worth noting that while the scanning happens automatically, the feature only works when iCloud photo sharing is turned on.

What's more, Apple is expected to use another cryptographic principle called threshold secret sharing that allows it to "interpret" the contents if an iCloud Photos account crosses a threshold of known child abuse imagery, following which the content is manually reviewed to confirm there is a match, and if so, disable the user's account, report the material to NCMEC, and pass it on to law enforcement.

Researchers Express Concern About Privacy


U.S. whistle-blower Edward Snowden tweeted that, despite the project's good intentions, what Apple is rolling out is "mass surveillance," while Johns Hopkins University cryptography professor and security expert Matthew Green said, "the problem is that encryption is a powerful tool that provides privacy, and you can't really have strong privacy while also surveilling every image anyone sends."

Apple CSAM Detection

Apple already checks iCloud files and images sent over email against known child abuse imagery, as do tech giants like Google, Twitter, Microsoft, Facebook, and Dropbox, who employ similar image hashing methods to look for and flag potential abuse material, but the company's attempt to walk a privacy tightrope could renew debates about weakening encryption, escalating a long-running tug of war over privacy and policing in the digital age.


The impending changes also mark somewhat of an about-face for a company that, along with Facebook-owned WhatsApp, has continually resisted efforts to intentionally weaken encryption and backdoor its systems. That said, Reuters reported last year that the company abandoned plans to encrypt users' full backups to iCloud in 2018 after the U.S. Federal Bureau of Investigation (FBI) raised concerns that doing so would impede investigations.

"Child exploitation is a serious problem, and Apple isn't the first tech company to bend its privacy-protective stance in an attempt to combat it. But that choice will come at a high price for overall user privacy," the Electronic Frontier Foundation (EFF) said in a statement, noting that Apple's move could break encryption protections and open the door for broader abuses.

"All it would take to widen the narrow backdoor that Apple is building is an expansion of the machine learning parameters to look for additional types of content, or a tweak of the configuration flags to scan, not just children's, but anyone's accounts. That's not a slippery slope; that's a fully built system just waiting for external pressure to make the slightest change," it added.

The CSAM efforts are set to roll out in the U.S. in the coming months as part of iOS 15 and macOS Monterey, but it remains to be seen if, or when, it would be available internationally. In December 2020, Facebook was forced to switch off some of its child abuse detection tools in Europe in response to recent changes to the European commission's e-privacy directive that effectively ban automated systems scanning for child sexual abuse images and other illegal content without users' explicit consent.