Every customer conversation I have about AI at the moment seems to start in the same place: GPUs, Kubernetes, inference, models and AI platforms. All important. But there is a question that comes up far less often, and I think it is one of the most important questions in agentic AI. What happens when an AI agent needs to remember?
This article is a collaboration with my Nutanix colleague Leonardo Boscaro, based on his original LinkedIn post on AI memory and the databases behind it. Leonardo kindly gave me permission to build on his thinking here, and I’ve expanded it for the enterprise architects and IT leaders who read this blog.
Disclosure: I work for Nutanix as a Principal Solutions Architect. This is my personal blog, and the opinions expressed here are my own and not necessarily those of my employer.
Models Can Reason. Agents Need to Remember.
Today’s models can reason, generate content and make decisions remarkably well. But a model on its own is largely stateless. Each request starts fresh. That’s fine for a one-off question in a chat window, but it falls apart quickly once you ask an agent to do real work inside a business.
Think about what an enterprise agent actually needs to do. It might need to remember what happened yesterday, or continue a task that was started last week. It may need to recall previous customer interactions, preserve the decisions it made and why, maintain the state of a multi-step business process, or learn from previous outcomes so it does better next time. None of that lives in the model. It lives in AI memory.
And AI memory requires persistence. As Leonardo puts it, persistence creates a database challenge. Whether that memory is conversation history, task and workflow state, vector embeddings for retrieval, a knowledge store, or an audit trail of agent actions, it has to be stored somewhere durable, consistent, secure and recoverable. In other words, it needs a database, and usually more than one kind of database.
I covered the retrieval side of this in my recent article, Bring the AI to the Data – Business Critical Databases Meet Enterprise AI on Nutanix, where the focus was on bringing inference and vectors to the business critical data you already have. AI memory is the other side of the same coin. It’s the new data that the agents themselves create, and need to keep.
From a Handful of Agents to Hundreds or Thousands
Most organisations are starting with a handful of agents, often as pilots. That’s manageable. The persistence layer might be one PostgreSQL instance that someone on the project team stood up, and nobody worries much about it.
But that’s not where this is heading. As enterprises move from a handful of AI agents to hundreds or thousands, those agents will need to maintain context, preserve application state and access enterprise knowledge. That doesn’t necessarily mean one database per agent. Many agents will share stores, and good architecture will consolidate where it makes sense. But it does potentially mean hundreds of persistent data workloads supporting an increasingly complex AI ecosystem, owned by different teams, in different locations, with different data classifications and different recovery requirements.
If that sounds familiar, it should. We’ve been here before with virtual machine sprawl, and again with database sprawl. AI is about to do it again, only faster.
The Challenges We Already Know Very Well
The interesting thing about AI memory is that the hard problems aren’t new. They’re the same problems database and infrastructure teams have been solving for decades, just at a larger scale and a faster pace. Leonardo highlights four.
- Data sovereignty. Where does AI memory live? Which country? Which infrastructure? Who controls it? Agent memory can contain customer interactions, decisions and sensitive business context. For many organisations, especially in regulated industries and in regions with strong data residency requirements, that memory is as sensitive as the systems of record it was derived from, and it needs to be treated that way.
- Database proliferation. How do we manage hundreds of databases without multiplying operational complexity? If every new agent or AI application brings its own hand-built database, the operational burden grows linearly with every project, and so does the risk.
- Lifecycle management. Who provisions, patches, upgrades, protects and recovers all these databases? An agent’s memory being lost or corrupted is not a minor inconvenience if that agent is part of a business process. It needs backup, point-in-time recovery and a tested restore path, like any other production database.
- Governance. How do we ensure consistency, security and operational control at scale? Standard configurations, access controls, audit and clear ownership matter just as much for the database behind an agent as they do for the database behind your ERP.
NAI and NDB – Different Challenges, Complementary Solutions
When people think about AI on Nutanix, they usually think about Nutanix Enterprise AI (NAI), and for good reason. NAI addresses an important part of the AI challenge: inference, model and agent access, and AI governance. It is how you run models privately, control which applications and agents can use them, and govern that access.
But NAI isn’t intended to replace database lifecycle management, and it shouldn’t be the only part of the platform in the AI conversation. This is where Nutanix Database Service (NDB) comes in. NDB addresses the operational challenges behind persistent AI data: database automation, lifecycle management, protection, scalability and consistent operations across locations and supported database engines. Importantly, it can help you maintain control over where your persistent data resides, which is a critical consideration for AI sovereignty.
Or, to put it in one line, as Leonardo did:
NAI helps enterprises run and govern AI. NDB helps enterprises operate the databases that support AI memory.
Different challenges. Complementary solutions. Mapping this back to the four challenges above:
- Sovereignty – keep AI memory on infrastructure you control, in the locations you choose, alongside the models served by NAI, rather than scattered across services you don’t.
- Proliferation – provision databases for new agents and AI applications from standard, automated profiles instead of building each one by hand.
- Lifecycle – apply the same patching, upgrade, protection and recovery processes to the databases behind your agents as you do to the rest of your fleet.
- Governance – consistent operations across engines and sites, so a growing AI estate doesn’t become a growing collection of exceptions.
And a full AI conversation is broader than the inference platform alone. The models, the agents, the data they retrieve and the memory they keep all need to be designed, operated and governed together.
The Question Every AI Plan Should Answer
If you’re an architect or IT leader putting together an agentic AI roadmap, here is a question I’d encourage you to ask, borrowed straight from Leonardo:
“You’re planning to deploy hundreds of AI agents. Who is going to manage, protect and govern the databases behind their memory? And where will that data live?”
It’s a simple question, but it tends to open a very different conversation. It brings in different stakeholders, such as the DBAs, the data protection team, security, risk and compliance, who are often not in the room for early AI discussions. And it surfaces requirements around recovery, residency and audit that are far easier to design in at the start than to retrofit after hundreds of agents are already in production.
A few practical things I’d suggest considering:
- Inventory the memory. For each agent or AI application, identify what state it keeps, where, and how sensitive it is.
- Treat AI memory as production data. Give it the same data classification, backup, recovery objectives and access controls as any other business system.
- Standardise early. Agree a small number of supported database engines and configurations for AI workloads, and automate their provisioning.
- Decide where it lives. Make data residency an explicit design decision for AI memory, not an accident of whichever service a pilot happened to use.
- Plan for scale. Design the operating model for hundreds of persistent data workloads, not just the first few.
Final Word
Agentic AI is only as useful as what the agents can remember, and memory means persistence. Persistence means databases, and lots of them, with all of the sovereignty, proliferation, lifecycle and governance challenges that come with them. The good news is that these are challenges we already know how to solve. Running and governing the AI is one half of the story. Operating the databases that hold its memory is the other. If you’re planning your AI roadmap, make sure both halves are in it. How are you planning to manage the memory behind your agents? I’d love to hear your feedback and comments.
A big thank you to Leonardo Boscaro for the original post and the thinking behind this article, and for letting me collaborate with him on it.
This post first appeared on the Long White Virtual Clouds blog at longwhiteclouds.com. By Michael Webster. Copyright © 2012 – 2026 – IT Solutions 2000 Ltd and Michael Webster. All rights reserved. Not to be reproduced for commercial purposes without written permission.