In a worrying twist, AI-powered chatbots are failing to detect manipulated images that were generated by the very same technology, casting doubt on their value as real-time verification tools. Online users who have begun relying on these bots to confirm the authenticity of suspicious photos are finding that, in many cases, the tools wrongly affirm fakes as genuine.
One standout example involves a fabricated photo of Elizaldy Co, a former Philippine lawmaker accused in a massive corruption scandal. The image circulated on social media as though it showed him in Portugal — but alarmingly, Google’s AI verification mode declared it “authentic,” even though it was eventually traced back to being created by Google’s own generative model. This happened despite AFP fact-checkers confirming that the picture was fabricated.
AI Tools Struggle to Spot Deepfakes They Themselves Create
Experts warn that the root of the problem lies in the design of these tools. Many large language model (LLM)-based systems are primarily tuned for text, not for the nuanced visual reasoning required to spot synthetic imagery. Alon Yamin, CEO of AI content-detection firm Copyleaks, told AFP that the absence of “specialized visual understanding” renders these bots inconsistent or overly vague in their judgments.
Another case emerged during protests in Pakistan-administered Kashmir, where a widely shared torchlit crowd photo — later found to be AI-generated — was analyzed by both Gemini and Microsoft Copilot, which incorrectly authenticated it as real. Researchers argue that this blind spot isn’t just a technical bug; it could undermine trust in verification systems exactly when humans are stepping back. Many major social platforms are scaling down third-party human fact-checking, putting disproportionate weight on automated systems.
Rossine Fallorina of the Sigla Research Center described the issue bluntly: “These models are programmed only to mimic well … they can only generate things to resemble. They cannot ascertain whether the resemblance is actually distinguishable from reality.” As a result, some researchers say AI-driven image verification can’t yet replace human fact-checkers — especially where trust, nuance, and visual context are critical.
