AI Scraping Is Exploding: What Publishers Must Do Before Their Traffic Declines Further

If you’re a publisher in 2025, you’ve probably felt the shift: referral traffic is flattening, discovery feels unpredictable, and AI-driven summaries often surface before your own articles. But the scale of what’s happening is far more dramatic than most realize.

At Evolve Hamburg, we shared Cloudflare data that stunned the room. AI scraping has grown not by percentage points—but by orders of magnitude:

• 2 → 18 pages scraped per referral on traditional search

• 250 → 1,500 for OpenAI

• 6,000 → 60,000 for Anthropic

This isn’t a trend.
It’s a structural rewrite of how content flows across the internet.

And it means publishers must act now.

1. Understand the new discovery landscape

AI models are becoming the starting point for information—especially for younger audiences across Europe. Instead of clicking through search results, readers increasingly ask:

“Give me a summary of today’s news.”

“What’s happening with inflation in Germany?”
“What are the top 5 things to know about X topic?”

Large models pull from thousands of pages, blend it, and produce an answer that may or may not cite or surface your brand.

The result?

• Less direct traffic

• Fewer page views

• Weaker first-party data

• Lower conversion potential

Your content is still valuable — but you are seeing less of the return.

2. Protect your content (and get compensated for it)

Publishers have three strategic options:

Option A — Disallow bot access

This is becoming more common. It protects your content, but limits discoverability inside AI tools. That may be fine for premium or specialized content.

Option B — License your content

AI companies want high-quality journalism.
Sam Altman and others have publicly acknowledged this.

Licensing gives you:

• Revenue

• Control

• Visibility

• Protection against misuse

This will likely become a major revenue stream in the next 3–5 years.

Option C — Allow scraping selectively

A hybrid model: open access for specific agents, restrict others.

This works well when you pair it with rights and reporting.

3. Build experiences AI cannot replicate

Scraping affects informational queries. But AI cannot replicate:

• Events

• Membership programs

• Exclusive experiences

• Box programs

• Premium newsletters

• Community-driven interactions

• Courses or academies

These are the formats driving growth across German and European publishers.

AI may summarize your articles, but it cannot replace:

Connection + Access + Belonging.

4. Lean aggressively into first-party data

The best defense against scraping pressure is a strong direct relationship with your audience. That requires:

• A modern paywall that adapts to behavior

• Personalized recommendations

• Predictive churn models

• Ability to segment and act on data in real time

• Content journeys that lead to conversion

Legacy systems cannot support this. Modern PaaS platforms can.

5. Treat AI as a tool, not a threat

The right AI can:

• Optimize your pricing

• Improve your offers

• Clone your best-performing campaign

• Run multi-variant tests across channels

• Predict what subscribers are likely to do next

AI scraping may reduce traffic, but AI inside your business increases revenue.

It’s not about fighting the technology.
It’s about using it to outperform the market.

The next decade belongs to publishers who act now

Traffic may decline. Referrals may shrink. AI summaries may grow. But publishers who diversify, protect their IP, strengthen direct relationships, and modernize their tech stacks will not just survive the shift — they’ll lead it.

Scraping doesn’t have to be the beginning of decline.
Handled correctly, it becomes the beginning of reinvention.