Agentic AI May Be the Next Major Driver of the Memory Supercycle

 The current artificial intelligence boom, driven by platforms such as ChatGPT, Gemini, and Claude, is associated with rapidly growing demand for GPUs and high-bandwidth memory. But this may represent only one stage of a broader transformation in AI adoption. Over time, agentic AI could become an increasingly important driver. Unlike conventional chatbots that primarily respond to prompts, agentic AI systems can perform multi-step tasks, interact with software, retrieve information, make intermediate decisions, maintain working context, and coordinate with other agents. 

Before proceeding, I would like to note that I wrote this blog post with the assistance of ChatGPT. 

If agentic AI is adopted at scale across enterprises and other organizations, its infrastructure requirements could extend well beyond those of the current chatbot-centric model. For the semiconductor industry, this could broaden AI-related memory demand beyond HBM to server DRAM, enterprise SSDs, NAND flash, CXL-based memory expansion, and potentially new memory architectures.

One important reason is the KV cache used during transformer inference. The KV cache arises from the self-attention mechanism in transformer-based LLMs, which requires extensive matrix computations. As a model processes tokens, it stores the key and value representations of previously processed tokens so they do not need to be recomputed whenever a new token is generated. KV caching therefore reduces redundant computation and accelerates inference, but at the cost of additional memory capacity. KV-cache requirements generally increase with context length and the number of concurrent agent sessions.

This trade-off becomes particularly important for agentic AI. A conventional chatbot interaction may involve a relatively short exchange. An enterprise agent, by contrast, may operate for minutes, hours, or longer while reading documents, querying databases, interacting with applications, invoking tools, evaluating intermediate results, and communicating with other agents. Such activities expand the working context and generate additional state that must be maintained or retrieved throughout a task.

The issue becomes even more significant at scale. A single agent may not require an extraordinary amount of memory, but an enterprise could eventually operate thousands of agents simultaneously, while major cloud platforms could support millions of concurrent agent sessions. The resulting aggregate memory footprint could become very large.

HBM remains essential because it provides the exceptionally high bandwidth required for GPU and AI-accelerator computation. However, keeping every agent's entire working state and KV cache in scarce and expensive HBM may become inefficient as context lengths and agent concurrency increase. This creates a strong incentive for memory tiering.

The most latency-sensitive model operations and active KV-cache data can remain in HBM, while less frequently accessed KV-cache blocks, context data, embeddings, retrieved information, and intermediate states can be moved to larger and less expensive capacity tiers. Server DRAM could provide relatively fast memory for active and semi-active data, while CXL-based memory could expand available memory capacity across inference servers and clusters. Enterprise SSDs could become particularly important as a lower-cost capacity layer beneath DRAM. Agentic workloads may generate large amounts of persistent or semi-persistent data, including inactive KV-cache segments, task histories, embeddings, vector databases, retrieved documents, checkpoints, logs, intermediate files, and long-running agent state. High-performance enterprise SSDs can store this less latency-sensitive data while keeping it sufficiently close to compute resources for rapid retrieval. KV-cache offloading and prefix caching already illustrate how flash storage can participate directly in AI inference rather than serving only as conventional bulk storage.

The emerging AI memory hierarchy could therefore be viewed broadly as follows: HBM for active computation and the most latency-sensitive data; server DRAM for larger pools of active and warm data; CXL-based memory for expandable capacity; and enterprise SSDs and NAND flash for much larger pools of persistent and less frequently accessed data.

This shift could have important implications for the memory industry. The first phase of the generative AI boom has disproportionately benefited HBM because training and high-performance inference require exceptional memory bandwidth. Large-scale agentic AI, however, may require not only bandwidth but also much greater memory capacity, persistence, and efficient movement of data across different memory tiers.

Even continued improvements in model efficiency would not necessarily reduce total memory demand. Lower memory consumption per token could be offset by longer contexts, longer-running tasks, greater agent concurrency, more complex workflows, and a rapidly increasing number of deployed agents.

Why Enterprise Agentic AI Is Different. Enterprise AI also cannot simply be standardized. Unlike a general-purpose chatbot, an enterprise agent must operate within the unique information and operational environment of a particular organization. Every company has its own proprietary data, workflows, software systems, approval processes, security policies, and domain-specific knowledge.

A useful way to think about an enterprise agent is as a combination of a foundation model, proprietary data, tools and APIs, workflows, and organizational rules and permissions.

The foundation model may come from OpenAI, Anthropic, Google, or another provider. Companies therefore do not need to replicate the enormous investment required to develop frontier models. Instead, much of the practical and economic value of enterprise AI may come from the organization-specific layer that connects these models to proprietary data, internal systems, tools, and workflows.

Consider a semiconductor manufacturer. A frontier model may understand semiconductor physics, process engineering, yield optimization, and manufacturing terminology. However, it does not inherently know the real-time conditions inside a particular fabrication plant, wafer histories, equipment performance, proprietary process recipes, defect distributions, customer orders, inventory positions, supplier contracts, or production schedules. An effective manufacturing agent must therefore operate within the company's own information environment. The foundation model provides general reasoning capabilities, while the enterprise supplies the proprietary knowledge, real-time data, tools, permissions, and operational logic required to turn those capabilities into economically useful actions. Two semiconductor companies could consequently use the same foundation model yet develop substantially different agentic capabilities.

The same principle applies to healthcare. Two major hospitals might use the same underlying frontier model, but their clinical agents would differ because their electronic medical records, clinical protocols, formularies, medical equipment, patient populations, administrative systems, and regulatory environments differ. A hospital cannot simply ask a public chatbot to autonomously manage these processes. It needs a secure agentic layer connecting the model to its own clinical and operational infrastructure, generally with appropriate human oversight.

This is why agentic AI may develop very differently from consumer generative AI. As frontier models become more widely accessible, access to the model itself may become less of a differentiating factor. Competitive advantage could increasingly depend on how effectively an organization combines the model with its proprietary data, domain expertise, internal systems, and workflows.

The relevant question may therefore gradually shift from “Which AI model does your company use?” to “How effectively does your company integrate AI with its proprietary knowledge, data, and workflows?”

Corporate AI adoption may consequently extend well beyond purchasing access to ChatGPT, Claude, Gemini, or other AI services. These platforms may instead provide a common technological foundation upon which companies build specialized agents for research, engineering, manufacturing, procurement, finance, logistics, customer service, and administration.

A Broader Memory Supercycle. From the perspective of the memory industry, the structural implication could be significant. Agentic AI does not merely require more computation. It requires increasing amounts of state to be maintained, moved, retrieved, and stored over time. Longer contexts increase KV-cache requirements, longer-running tasks keep that state active for longer periods, multi-agent systems multiply concurrent contexts, and enterprise retrieval systems continually access large proprietary datasets.

The first phase of the AI memory supercycle has therefore been dominated by HBM. A second phase driven by large-scale agentic AI could be considerably broader. If millions of persistent AI agents eventually operate across corporations, hospitals, governments, and cloud platforms, the resulting infrastructure may require not only more HBM but also substantially more server DRAM, enterprise SSDs, NAND flash, and memory-expansion technologies.

In that scenario, agentic AI could become one of the most important long-term demand drivers for the global memory industry.