AI or Not Detects 98% of Cropped Meta AI Images. Meta's Own Tool Catches 35%.

Meta's AI Image Watermark Fails the Moment Someone Crops the Photo

On July 19, AI or Not published a 205-image benchmark comparing its own detector against Meta's "Identify images generated with Meta AI" tool, launched days after Meta shipped its first in-house image generator on July 7. On 103 untouched Meta AI images, AI or Not caught 100% and Meta's tool caught 98.1% (101/103) — close enough. But on 102 cropped versions of the same images, AI or Not still caught 98.0% (100/102) while Meta's Content Seal watermark detector caught only 35.3% (36/102). Reuters ran a separate, smaller test of 40 Meta AI images and found the same pattern: Meta's tool confirmed 100% of untouched originals but only 45% of images cropped to between a third and a half of their original size — a 55% miss rate.

Does Meta's own watermark actually survive editing?

No. Meta's Content Seal detector loses more than half its accuracy on cropped images in two independent tests, by two different testers.

Meta's launch materials claimed the watermark holds up "even when cropped, compressed, resized, or screenshotted." Both benchmarks contradict that specifically for cropping, the most common edit a real photo undergoes before it ends up recirculated on social media or in a listicle. AI or Not's breakdown also shows the failure isn't uniform: cropped landscape and scene photos were caught just 9.6% of the time (5 of 52), versus 62.0% for cropped portraits, food and objects (31 of 50) — suggesting whatever signal Content Seal relies on is spatially fragile in a way that depends on composition, not just crop percentage.

Why does a classifier like AI or Not hold up where a watermark doesn't?

Because it looks for generation artifacts in pixels, not an embedded signal that a crop can simply cut away or dilute.

A watermark like Content Seal is a fixed marker seeded at generation time; if the crop removes or degrades the region carrying it, there's nothing left to detect. A forensic classifier is instead scoring statistical fingerprints across the whole image, so it degrades more gracefully. That's a structural distinction, not proof AI or Not is simply "better" — it's a vendor testing its own product against a rival's, and readers should weight that accordingly.

What should buyers actually ask before trusting a provenance claim?

Ask what edit operations were tested, by whom, and whether the detector's own maker ran the benchmark.

Content Seal only works on images generated by Meta's own models — it can't read C2PA content credentials, Google's SynthID, or older un-watermarked Meta AI outputs. That narrow scope, combined with a documented crop failure, means any platform, newsroom or moderation team relying on Meta's tool alone is verifying a shrinking slice of what actually crosses their desk. Meta told Reuters the signal "may be lost if an image is heavily cropped," which is a fair disclosure but not one that was prominent in the original launch claim.

Where does this fit with the rest of the provenance debate?

It confirms watermarking alone is not a durable detection layer, reinforcing this site's coverage of governance gaps around AI content controls.

Siwei Lyu, a computer science professor at the University at Buffalo who researches image forensics, told Reuters that watermark methods "can be highly effective when the watermark remains intact, but any modification that removes or weakens the embedded signal... may reduce their effectiveness." That's the core caveat for anyone building moderation or compliance workflows around a single vendor's watermark rather than layering in independent classifiers.

Frequently asked questions

Does this mean Meta AI images are undetectable once cropped?

Not undetectable — just harder for Meta's own tool. AI or Not's classifier still caught 98.0% of the same cropped set, and never fell below 96.2% across either batch.

Was this an independent test?

AI or Not is a detection vendor testing a rival's tool, so its 205-image benchmark should be read alongside Reuters' separate 40-image test, which found a comparable gap.

Does Content Seal work on images from other AI tools?

No. It only detects Meta's own Content Seal watermark and cannot read C2PA content credentials or Google's SynthID.

Sources: AI or Not; Reuters' testing as reported within the AI or Not analysis.