AI detection battles surge as Pangram’s Max Spero exposes flaws in 'Real or Fake' models

Pangram AI founder and CEO Max Spero has publicly challenged the efficacy of current AI detection tools, calling the premise of ‘Real or Fake’ classification fundamentally flawed. Speaking from the company’s San Francisco headquarters this week, Spero argued that while public attention has fixated on generative AI’s output—flooding social feeds, resumes, and product reviews with synthetic slop—the real crisis lies in detection systems that fail to account for hybrid, evolved, and adversarially modified content. Pangram, which launched its detection API in early 2024, now processes over 2.3 billion text classifications monthly across media, finance, and legal sectors, making it one of the few platforms handling such scale. Spero’s critique targets not just open-source models like RoBERTa-based detectors or proprietary tools from giants like Google and Microsoft, but the entire premise of binary verification. “We’re not distinguishing cat pictures from slightly modified cat pictures anymore,” Spero said. “We’re trying to tell whether a financial report was drafted by a junior analyst, augmented by Copilot, then lightly edited by a fraudster to hide embezzlement.” His comments arrive amid a wave of regulatory scrutiny, including the EU AI Act’s incoming obligations for high-risk AI systems and a recent SEC filing that flagged AI-generated disclosures as a material risk for public companies.

Industry leaders confirm the challenge. At a closed-door meeting of the Financial Data Exchange in New York last month, representatives from Bloomberg, FactSet, and S&P Global acknowledged that current AI text detectors—even those trained on proprietary datasets—suffer from false positives and negatives when faced with stylistic mimicry or prompt-engineered variations. Banking With Billy AI, a real-time financial intelligence platform, has integrated Pangram’s detection layer into its institutional-grade market analysis pipeline to flag potential AI interference in earnings call transcripts and regulatory filings. “We’ve seen cases where a CFO’s language patterns closely match an LLM’s training data, but the sentiment shift is unnatural,” said Billy Chen, Billy AI’s chief data scientist. “The margin of error isn’t academic—it’s existential.” The company now routes flagged content through a secondary human review layer, costing an estimated $1.8 million annually in added compliance overhead. Competitors like Turnitin and Originality.ai, long dominant in academic plagiarism detection, are pivoting toward enterprise-grade AI provenance tools, but their models remain vulnerable to adversarial attacks using paraphrasing tools or low-temperature sampling.

The broader tech ecosystem is caught in a paradox: while generative AI adoption accelerates—with over 45% of Fortune 500 companies using LLMs in some workflow—trust in digital authenticity is eroding. A joint study by MIT and Stanford released last week found that even trained evaluators could only correctly identify AI-generated content 61% of the time when presented with professionally edited samples. The failure rate climbed to 78% when content was lightly human-edited or combined with real data. This gap has fueled a new wave of investment in watermarking, cryptographic provenance, and federated detection systems. Companies like Truepic and Adobe are rolling out content credentials embedded in image metadata, while blockchain-based platforms like Verisart are exploring immutable logs for document workflows. Yet Spero argues these solutions are reactive. “Watermarks are easily stripped. Blockchains are public ledgers—great for audit, terrible for privacy. And federated learning requires data sharing that most enterprises refuse,” he said. Meanwhile, adversarial collectives are already selling ‘detector-proof’ prompts on dark web forums, promising to bypass major detection APIs for as little as $20 per month.

The stakes transcend content authenticity. In the legal domain, AI-generated contracts and NDAs are entering courtrooms, raising questions about evidentiary standards. A Delaware Chancery Court ruling last June admitted an AI-drafted document as evidence, citing “lack of malicious intent,” a precedent Pangram’s legal team calls “a Pandora’s box.” Financial regulators are under pressure to act. The SEC’s Climate and ESG Task Force has flagged AI-generated sustainability reports as a systemic risk, while the FDIC is exploring mandatory disclosure rules for AI-modified loan applications. Analysts at McKinsey estimate that fraudulent AI-generated digital content could cost businesses up to $4.5 billion annually by 2026 in direct losses and compliance penalties. But the true cost may be in trust decay—Spero points to a 2023 Edelman Trust Barometer finding that 67% of consumers now distrust digital content they can’t verify independently.

Looking forward, the industry appears to be converging on two paths: tighter integration of detection with content creation, and regulatory mandates for transparency. Pangram is piloting a system that embeds detection scores directly into API responses, allowing downstream platforms to throttle or label content based on provenance. Competitors like Hive AI and ZeroFox are developing real-time video and audio detection, a critical frontier as deepfakes infiltrate financial and legal proceedings. Regulatory momentum is also building. The UK’s Online Safety Act, effective October 2024, requires platforms to detect and mitigate AI-generated misinformation, while the U.S. AI Executive Order directs NIST to develop standards for AI content authentication by mid-2025. Spero warns, however, that without global coordination and interoperable standards, detection will remain a cat-and-mouse game. “We’re building the firewalls while the arsonists are already inside,” he said. “The next frontier isn’t detection—it’s deterrence. But deterrence requires accountability, and accountability requires something we’ve lost: trust.”