Cisco sees rising role for service providers in AI's inferencing era

　　Amid the training of AI models, much of the infrastructure focus has centered on processing-rich and power-hungry data centers. As more of AI shifts to inferencing and autonomous AI agents that require less power and less latency, a window of opportunity is opening for the network access players, including cable operators and telcos, Cisco execs said at last week's Cisco Live event in Las Vegas.

　　Cisco was stating the case even prior to the event, distributing a study indicating that AI is reshaping infrastructure requirements, and forecasting that AI inference traffic could represent about 25% of total network traffic by 2035.

　　But, as Light Reading has explained, the degree of that network and data shift is open to debate, given the ample headroom that currently exists on most of today's networks. Nevertheless, Cisco, which teamed with telcos and applied sensors to access routers to scan AI data flows to support its AI traffic trends report, sees this emerging trend as an opportunity-in-the-making for service providers.

　　That initial report will serve as a "foundation" as Cisco expands the number of network sensors that are capable of detecting and tracking AI-based traffic, Guru Shenoy, SVP of Cisco's provider connectivity group, told Light Reading.

　　"We are monitoring this on a continuing basis now," he said, noting that Cisco intends to publish fresh reports on a regular basis of six months, and possibly less.

　　AI agents poised to change the nature of Internet traffic

　　Cisco believes that the emergence of agentic AI traffic, and the expectation that agents will be interacting with other agents in increasing instances, will alter the very nature of Internet traffic, with the duration of those "transactions" taking much longer than human-led actions.

　　like us humans, AI agents scan and read data, but do it much faster and for longer periods. Cisco estimates that, on average, AI agents consume 450% more data traffic than humans to do the same task while also taking about two times longer to perform it.

　　"And a lot of these flows are multidirectional…the traffic patterns with AI are different," Shenoy said. Agents, he explained, can "farm out" these things simultaneously, process them and then output a response.

　　For now, much of this is happening with enterprise traffic, with coding via platforms such as Codex (via ChatGPT) and Claude being the greatest source of traffic, he said.

　　Cable ops and telcos positioned to stand up and monetize 'inferencing clouds'

　　But the inferencing trend and the emergence of agents as the primary consumers of AI is causing AI to extend beyond the data center and drive things toward a more distributed architecture that could, for example, push cable operators to set up their own "inferencing clouds" and enable them to extend well beyond providing connectivity. These set ups could end up competing with or complementing the GPU-centric neoclouds that are being stood up for inferencing and to support the desire for secure, sovereign networks for AI.

　　"The telco and neocloud worlds are colliding," Shenoy said. "I think every service provider should look very closely at the AI traffic trends. The need to be prepared to react to it very fast or be proactive about it already."

　　Some operators already see a rising role for them to play in this phase of the AI era. Comcast and charter, for example, are already pushing ahead with respective AI edge initiatives, and Bell Canada teamed with Nvidia partner Groq on its own inferencing projects. Meanwhile, these trends could finally provide the spark for mobile edge compute that was envisioned during the early days of 5G, but was ahead of its time.

　　"The telcos need to look beyond connectivity and tap into this opportunity to become inferencing cloud providers," Shenoy said.

　　"As service providers look at AI, they need to [determine] how they can take a bigger part of the spend in that space," said Kevin Wollenweber, Cisco's SVP and GM of data center and Internet infrastructure.

　　Though the AI inferencing market is still nascent (most of Cisco's AI orders and revenues come from hyperscalers and AI training), he said a prime example is a recent AI grid collaboration between AT&T, Cisco and Nvidia that aims to bring on-demand AI inference closer to the edge, where the data is generated.

　　Megaport moves on AI inferencing

　　Others are also embracing the trend. For example, Megaport, a network-as-a-service company based in Australia, is raising about $594 million to support the build of its inference cloud strategy. It also announced it has secured four AI infrastructure contracts worth about $329 million.

　　Megaport, which provides a backbone across about 11,000 data centers in about 30 countries and counts AT&T among its customers, believes that its approach will help to fulfill the rising demand for inferencing and access to GPUs.

　　The initial focus for that is in the US, where it has servers already stood up in 20 cities, according to CTO Cameron Daniel. "We'll go where there is demand," he said.

　　And that demand is becoming more distributed as the market is seeing a shift from accessing to hyperscalers to collocated locations, added Matt Simpson, Megaport's EVP of business development.