AI Scraping of Publisher Content Surges, Defying Protections
New research highlights a sharp rise in AI-driven scraping of publisher websites, with bots bypassing blocking measures and licensing agreements during 2025.
A report from TollBit shows a 29% increase in AI bot scraping between Q2 and Q3 of 2025, followed by a 20% uptick from Q3 to Q4. Despite publishers enforcing robots.txt permissions, 30% of scraping attempts in Q4 still ignored these rules, undermining website protections.
How AI bots affect referral traffic
AI models are increasingly crawling sites for content summaries and training data, but the traffic they return is minimal. Publishers with content licensing deals have also reported significant declines in click-through rates (CTR), indicating that such agreements are not effective safeguards against reduced referral traffic from AI platforms.
Key findings from the TollBit report
- AI-driven scraping grew by 29% between Q2 and Q3 2025.
- A further 20% increase occurred from Q3 to Q4 2025.
- 30% of AI scraping attempts bypassed robots.txt rules in Q4 2025.
- Licensed websites experienced notable CTR declines across 2025.
Addressing the impact
For WordPress developers, SEO specialists, and content marketers, understanding these scraping trends is critical. Minimizing traffic losses may require creative strategies that adapt to AI-driven disruptions while safeguarding valuable content.
News-to-Post Feature empowers users to streamline publishing, offering an efficient solution in a challenging traffic ecosystem.
Source: Digiday
Source: Digiday
