🧠 Food for thought

Implications, context, and why it matters.

Independent web data suggests AI bots are still a relatively small share of page requests

• Lumen Technologies’ CEO says AI bots drive more than half of internet traffic, while a 2025 slice of web data paints a narrower picture.
• In a dataset limited to HTML page requests, AI bots made up 4.2% of requests, while Googlebot, Google’s web-crawling bot, made up 4.5% 1.
• Bot activity can top 50% for certain measurements. In Cloudflare’s early 2025 HTML analysis, non-AI bots handled about half of requests, yet AI bots remained a small fraction within the same dataset 1.
• Even with fewer requests, AI crawlers can put outsized pressure on networks.
• AI training crawlers from companies such as Anthropic, an AI model developer, pull large volumes of data, with crawling-to-referral ratios as high as 500,000-to-1. That gap helps explain why businesses may seek more flexible network capacity models even when AI bots stay a minority in this dataset 1.

Legacy telecom companies are using AI to reshape their own operations

• At Lumen, the AI effort covers new products plus work to steady finances and day-to-day execution.
• Lumen relies on AI software from Palantir, a US data analytics company, tied to an internal target of US$1 billion in cost savings by 2027 2.
• The shift began when Lumen brought Palantir’s tools into its own operations, then later teamed up to sell joint products to enterprise customers 3.
• Demand from large AI firms backed the move, including a contract to expand Anthropic’s fiber network 4.
• Those deals helped Lumen land debt rating upgrades and recast itself from a legacy telecom provider to a digital infrastructure company 4.