Soaring AI bot traffic presents challenges for publishers – The AI Journal

Welcome to the forefront of conversational AI as we explore the fascinating world of AI chatbots in our dedicated blog series. Discover the latest advancements, applications, and strategies that propel the evolution of chatbot technology. From enhancing customer interactions to streamlining business processes, these articles delve into the innovative ways artificial intelligence is shaping the landscape of automated conversational agents. Whether you’re a business owner, developer, or simply intrigued by the future of interactive technology, join us on this journey to unravel the transformative power and endless possibilities of AI chatbots.
AI, LLM-based search and bot-driven content consumption are reshaping, and in many ways destabilising, the digital media and publishing industries. The recent layoffs across the digital media landscape, from the Washington Post to the Huffington Post, demonstrate a shift in the way people are sourcing information. Many no longer click on links for information, they are getting answers directly from AI chatbots. For example, OpenAI’s ChatGPT was reportedly managing 143 million searches per day by the second half of 2025.     
Unlike traditional search engines, LLM search queries often don’t provide direct links to source content, drastically reducing click-through rates and publisher traffic. Research from the Akamai SOTI AI Botnet Report 2025, which analysed the changing landscape of AI-powered bots and their impact on the internet, found that the publishing industry accounted for 63% of AI bot triggers within the broader digital media landscape, likely tied to extensive content scraping.  
It’s important to note that not all of these AI bots are ‘bad. Many are conducting essential jobs that keep internet services running. The challenge is that, with human traffic falling and AI bot traffic rising, it’s becoming increasingly difficult for resource-stretched publishers to distinguish between helpful bots, and those which may be less desirable due to their tendency to divert traffic away from the source website. Digital media is one of the first industries to feel this shift, but it won’t be the last.  
Our Akamai SOTI AI Botnet research reports that there are now roughly 3 billion AI bots scouring the internet in the UK. AIdriven bot traffic has surged to the point where it is growing at a pace that far exceeds traditional bot activity.  
This acceleration doesn’t just add volume; it adds complexity. The task of accurately distinguishing between the legitimate automated services that support business operations from potentially malicious activity is made significantly more difficult.  
When organisations fail to manage their bot traffic effectively, things start to break quickly, and the impact is immediate and measurable. Infrastructure costs rise, digital experiences slow down, and critical business metrics become polluted with bad data. In other words, unmanaged AI bot traffic quickly becomes a business problem, not just a technical one. 
Publishers are facing unprecedented levels of web scraping and are actively exploring solutions to protect their content. Nieman Lab reported on an increasing number of publishers like The Guardian and New York Times that are scrutinising their archives to identify and block AI bots scraping for content. Unfortunately, this proves to be technically complex. 
The problem is that AI bots are becoming indistinguishable from human traffic, using headless browsers, authentic browser fingerprints, and realistic interactive patterns to avoid detection. This creates a blind spot for systems and is an obvious security weakness.  
AI bots have become more aggressive, with operators bypassing firewalls using AI-driven obfuscation, device fingerprinting manipulation, and adaptive evasion to stay ahead of existing detection models. This evolution has become a significant threat for publishers and other web-based business models.  
As AI bots increasingly mimic real users, identity-based security has become less effective, leaving an opening for attackers to exploit digital platforms in new ways. The democratisation of AI tools means that even entry-level cybercriminals can coordinate complex attacks, leading to operational disruption.  
But as AI further proliferates, the hardest job has become distinguishing the good bots from the bad ones. 
The existential threat facing publishing and other industries is undeniable – but the solution remains elusive.  As bots increasingly blur the lines between real users and imposters, distinguishing between harmful and helpful bot activity becomes increasingly challenging without specialist tools. The real challenge isn’t just blocking bots , but gaining full visibility into all bot traffic.  
To address these challenges, organisations must rethink their approach to bot management and digital security. This involves three critical strategies: combining identity and behavioural strategy to analyse and identify malicious bots; strengthening API security to achieve greater visibility across the network; and exploring new content monetisation models to control how their content is accessed and valued.   
Once identity and behaviour are analysed together, businesses can apply more granular controls that allow access to beneficial bots but block malicious scrapers or impersonators. An adaptive approach is critical to evolving attack profiles.  
Secondly, API security is becoming even more essential in an era of mass AI bot traffic. Akamai researchers have observed that some AI bots are capable of intensifying risks in applications and APIs by automating advanced attacks that exploit session flows, authentication mechanisms, and API business logic. API security is therefore becoming even more critical as a core pillar of cybersecurity best practice. OWASP frameworks can be a helpful guide for identifying common vulnerabilities, such as broken authentication and authorisation flows, which are frequently exploited in fraud and abuse scenarios.   
Lastly, an increasingly important part of the equation is content monetisation. Publishers can redirect AI bots through platforms that require bots to identify themselves and pay for access. This puts control back in the hands of publishers, allowing them to reclaim the economic value of scraped content. 
If used in the right way, AI can be a great benefit to enable us as a tool to help us be more creative and produce more engaging and actionable content, and get more done, more quickly. But in order for the workforce, media, and society to benefit, we have to get it right.  

source

Scroll to Top