I was deep in research on AI harnesses when another question grabbed my attention: will the inference market eventually evolve to look like the CDN (content delivery network) market? Will differentiation among inferencing players eventually disappear, as it did in the CDN space? For the telcos watching AI traffic reshape network economics and contemplating new inference-based plays, this matters. If inferencing looks like CDN, then the AI inferencing buildout reduces to a commodity race, rather than a sustainable new revenue (and profit) layer.
Let me start by taking the position that inferencing is starting to look like the undifferentiated portion of the CDN market. Raw token generation looks like it’s heading toward commodity pricing, much like raw CDN bandwidth did. Open-source inference engines like vLLM and SGLang keep closing the performance gap with proprietary stacks. NVIDIA keeps pushing optimization into the hardware layer itself and bundling inference software with the chips, so the capability comes free with the (big) purchase order. Even the frontier labs, whom you would expect to be customers of this market, started competing on their own token prices once DeepSeek entered the market in 2025. When something becomes easier to produce at the same level of quality, price competition follows, and buyers don’t care who they are purchasing from (brand matters less). In a market like that, the winner will be whoever puts more intelligence into the tokens rather than the price of tokens. Value is created by the types of tokens to generate, when to generate them, and how they can drive useful work.
The CDN market's history provides some insight into how this could go. Limelight Networks was a pioneer in CDN, arriving almost a decade before Cloudflare. It posted record traffic during the 2020 streaming boom while revenue declined. It rebranded as Edgio (after acquiring Edgecast) in 2022, filed for bankruptcy in 2024, and shut down for good in early 2025. It wasn’t a market demand issue because traffic actually grew. It was an inability to command margin for its undifferentiated offering. Cloudflare started similarly by selling basic caching services, but quickly built security, developer tools, and edge compute on top of the same commodity network. Being cloud-native also let Cloudflare ship add-ons faster than Limelight ever could, resulting in a dramatically different outcome from Limelight. The differentiator was not caching, but what Cloudflare built on top of it.
I believe inference will split the same way. The commodity layer looks like "run this model, return tokens." Early inference companies are doing fine just renting GPUs and selling token generation. But the longer-term defensible layer is the services on top of that: routing the right task to the right model, maintaining an agent's memory and context over time, guaranteeing security and traceability, and meeting data-sovereignty requirements. That's a services-and-orchestration business, not a compute-rental business, and it behaves.
There's one structural difference worth mentioning. Switching CDN providers was mostly a DNS change (perhaps a little more than that, but it wasn’t a heavy lift). Cached content had almost no stickiness to a specific vendor. The KV-cache used in disaggregated inferencing (which helps reduce the compute intensity) doesn't work the same way. A KV cache is the precomputed memory state an LLM builds after processing a block of text to understand the context (prefill stage), and that gets used in the generation of the response (decode stage), so it can skip reprocessing similar text on the next query.
Picture a company support chatbot that fields thousands of questions a day, many using the same 20-page policy document as context with a distinct question at the end. Without caching, the model reprocesses those 20 pages thousands of times a day. A CDN analogy only goes so far here: a CDN fetches the same static file and hands it back unchanged, while a KV cache holds a piece of the model's working memory and produces a differentiated response every time. And that's exactly what makes it sticky.
Once a provider has cached your context, conversation history, and agent memory, switching to a competitor means losing that state. More importantly, the inference pipeline you had to build with one provider isn’t easily portable to another. This is closer to database lock-in than bandwidth switching. Researchers at the University of Chicago made this case in a 2024 paper proposing a "Knowledge Delivery Network," a system that stores, compresses, and delivers KV caches, much like a CDN stores and delivers files. Their prototype showed improved performance and lower costs than standard retrieval-augmented generation. But the architecture is still sticky and significantly less portable than a static video file.
There’s a lesson for telcos contemplating building edge inferencing as a business. Telcos should not compete on raw connectivity or edge presence, because that layer is likely to commoditize. A telco that only gets faster or cheaper at moving packets to a nearby node is repeating Limelight’s folly. The smart telcos will want to build intelligent routing, sovereignty controls, strong governance, and vertical-specific orchestration on top of their platform. Telcos hold real assets for playing this game, including proximity to users, established regulatory relationships, and a physical footprint that took decades to build. But those assets only pay off if something differentiated gets built on top of them.
The market is already showing signs of this pattern. On its Q1 2026 earnings call, Akamai, a leading CDN, disclosed a $1.8 billion, seven-year cloud infrastructure deal with a "leading frontier model provider," which Bloomberg identified as Anthropic (neither company has officially confirmed the customer's name). Whomever the counterparty is, this is a CDN player transforming into an AI infrastructure provider. Akamai isn’t counting on the speed of its CDN here. It’s leveraging its cloud compute, security, and edge inference capacity atop its original network.
Another signal comes from deep in the AI plumbing. In August 2026, Google open-sourced TPU Raiden, a library built to move KV caches between machines during inference, sitting roughly at the same layer in the stack as NVIDIA's NIXL, which does the same job for GPUs. Here's how I read it: Akamai's inference cloud spreads GPUs across geographies to reduce latency for agent workloads, and that distributed setup only pays off if a model's KV cache can move efficiently between locations rather than being rebuilt from scratch at every hop. When you see Google and NVIDIA ship similar capabilities to improve the mobility of KV caches, it’s an indication to watch this space carefully.
Getting back to the original question. AI inferencing might go the same way as the CDN market did. Token prices will go down; tokens might become less differentiated. However, there is a path for telcos who want to build a sustainable business around AI inferencing. And it involves adding smart routing, governance, and the other value-adds I discussed earlier. In addition, telcos need to understand how to manage and leverage KV caches (placement, routing, distribution), which are inherently stickier than static content. Telcos that can build differentiated layers on top of their underlying infrastructure estate might have a shot at competing with the Cloudflares and Akamais already in the game.