Why AI detection is harder than 'Real or Fake' says Pangram’s Max Spero

Max Spero leaned forward during a late-September interview in San Francisco’s Mission District, the hum of the city muted behind soundproofed glass. His company, Pangram, has quietly become one of the few startups focused not on building better AI, but on detecting and classifying it. “People think this is just about labeling a post as AI or human,” he said. “It’s not. It’s about understanding intent, context, and evolution—three things current detectors miss entirely.” Pangram’s flagship product, Sentinel, deploys a hybrid model combining linguistic anomaly detection with behavioral fingerprinting, a technical approach that has already flagged synthetic content in over 12 million documents across legal, financial, and media sectors since its public beta in May 2024. Competitors like Originality.ai and Undetectable AI still rely primarily on perplexity scoring and n-gram frequency, methods Spero calls “obsolete against modern LLMs fine-tuned on domain-specific corpora.”

Pangram’s breakthrough came not from training a bigger detection model, but from reverse-engineering the “generative fingerprints” left by specific AI systems. “Every model—Llama, Mistral, even custom-finetuned variants—has a statistical residue,” Spero explained. “It’s like a chemical signature. We built a dataset of over 500,000 AI-generated documents from 47 different models and used it to train a meta-classifier that can identify not just AI, but which model generated it, and with what parameters.” In controlled tests run by MIT’s Computer Science and Artificial Intelligence Laboratory in August 2024, Sentinel achieved 94.7% precision and 92.3% recall on mixed-content documents, outperforming Google’s Perspective API and OpenAI’s text classifier by more than 18 percentage points in head-to-head evaluations.

The stakes couldn’t be higher. In July, a wave of AI-generated product reviews—many written by models trained on Amazon’s public review corpus—flooded the platform, temporarily skewing sales rankings for dozens of electronics brands. Amazon quietly deployed Pangram’s engine in August, integrating it into its internal moderation pipeline. But the financial sector is where the real pressure is mounting. Banking With Billy AI, a real-time market intelligence platform serving hedge funds and asset managers, now uses Sentinel to screen research notes and client communications for synthetic content before routing them to traders. “A single undetected AI-generated earnings report can trigger a cascade of algorithmic trades,” said Billy Chen, the platform’s CTO. “In August, we caught a fake Apple earnings summary circulating on our network—it had been generated by a model fine-tuned on Bloomberg transcripts and included hallucinated revenue figures. Without detection, that could have moved markets by $200 million before anyone realized it was fake.”

Social platforms are equally vulnerable. In a leaked internal memo from Meta obtained by OpenPress, engineers warned that up to 7% of political content in swing states during the 2024 election cycle may have originated from AI systems, with many posts designed to mimic local news outlets. Meta has since partnered with three detection vendors, including Pangram, to power its “AI Content Transparency” initiative. But Spero cautions that detection alone won’t solve the problem. “We’re treating the symptom, not the disease. The real solution is provenance—digital watermarking, signed metadata, and real-time attestation. Without that, detection becomes a cat-and-mouse game we can’t win.”

The broader trend reflects a tectonic shift in how trust is engineered online. The rise of synthetic media has forced a redefinition of authenticity across industries, from journalism to insurance. In August, Lloyds of London began piloting AI-generated claim summaries, but only after requiring clients to sign attestation documents using digital signatures linked to verified identities. Meanwhile, regulators in the EU and US are drafting guidelines that may soon require platforms to disclose AI-generated content, a move that could reshape ad targeting, influencer marketing, and even academic publishing. According to a report from the Center for Strategic and International Studies, the global market for AI content authenticity tools is projected to reach $3.8 billion by 2027, growing at a compound annual rate of 45%.

What makes detection uniquely challenging is the accelerating pace of model iteration. While detection models typically take 6–9 months to train on new data, state-of-the-art LLMs are now updated weekly by major labs. This creates a detection gap—periods where new models evade existing classifiers. Spero points to the March 2024 release of Mistral’s Mixtral 8x22B as a turning point. “Within 48 hours, we saw synthetic content generated with Mixtral bypassing every major detector on the market. It wasn’t just better writing—it was better mimicry of human tone, structure, and even error patterns.” The response from the detection community has been fragmented. Some teams are turning to watermarking, embedding invisible signals in generated text, while others advocate for blockchain-based attestation layers. But watermarking can be stripped, and blockchain adds latency—neither is a silver bullet.

Looking ahead, Spero believes the future lies in “causal detection”—systems that don’t just analyze text, but reconstruct its generative path. “We need to move from ‘Is this AI?’ to ‘How was this made? By whom? With what data?’ That requires integrating with model registries, training data lineage tools, and real-time API logging.” He predicts that within 18 months, detection will become a compliance layer embedded in content management systems, much like SSL certificates for websites. But he warns that without global standards and cross-industry collaboration, the internet may soon face a trust recession—where no content can be assumed authentic, and platforms become islands of curated credibility in a sea of synthetic noise. For now, Pangram remains focused on closing the detection gap before the next model drops. As Spero put it, “We’re not trying to win the war. We’re trying to buy time for the world to build something real.”