The Data Layer for the Next Phase of Generative AI

Generative AI is evolving from knowledge-based applications like chatbots and copilots to autonomous AI agents capable of reasoning and performing complex, multistep workflows. These agentic AI systems can automate diverse, high-complexity use cases across industries and business functions. This article explores the different types of memory systems required for agentic AI, identifies the key challenges these systems face, and discusses how to integrate disparate databases into a cohesive memory system.

What is Agentic AI?

Agentic AI opens the door to a new era where AI agents act as collaborators and innovators, fundamentally changing how humans interact with technology. Agentic AI refers to systems designed to autonomously pursue complex goals with minimal human supervision, exhibiting decision-making, planning, and adaptive execution to complete multistep processes. These AI agents aim to operate more like humans—understanding context, setting goals, reasoning through tasks, and adapting actions based on changing conditions.

Agentic AI requires three critical components: large language models (LLMs), memory, and a plan. Each component serves a distinct role, and when integrated, they enable the agent to achieve more than the sum of its parts.

• LLMs: An agent may use LLMs multiple times to break down problems and perform subtasks. It can start by calling an LLM to summarize the current conversation, creating working memory. Another LLM can then plan possible next actions, followed by a third LLM to evaluate the quality of each candidate action. Finally, a fourth LLM call generates the final response for the user. Using specialized LLM calls for different purposes enables agents to achieve significantly higher performance than with a single LLM call.
• Memory: Long-term memory enables agents to answer questions or solve problems more effectively. For example, a retailer’s agent could access detailed knowledge about products, warranty policies, and company history, enabling accurate responses to customer inquiries.
• Plan: To reliably perform complex, multistep tasks, an agent requires a repository of “plans” or workflows. These plans capture procedural knowledge and guide the agent through the necessary steps, ensuring tasks are performed in the correct order and completed successfully.

Types of Memory Systems in Agentic AI

Memory systems are fundamental for AI agents to work. AI agents require memory systems to enhance their functionality, efficiency, and ability to perform complex tasks. They are needed for storing and retrieving information, maintaining context, learning from past experiences, and making informed decisions. A robust memory system is essential for organizing and storing various kinds of information that AI agents can retrieve during inference, ensuring accurate and efficient performance.

The memory system for AI agents can be classified into short-term and long-term memory systems.

• Short-term memory includes the working memory that temporarily holds information that is actively being processed. This is necessary for tasks requiring immediate attention and manipulation. Another use for working or short-term memory is context management, which is crucial for AI agents to maintain the context of ongoing tasks. This capability is necessary for coherent operations and decision-making, allowing agents to process complex instructions over several interactions and workflows.
• Long-term memory encompasses episodic, semantic, and procedural memory:
  • Episodic memory stores specific events and experiences, enabling the AI to recall past interactions and use that knowledge to inform future decisions, which is key for learning and adaptation.
  • Semantic memory holds general knowledge about the world, including facts and concepts, aiding the AI in understanding and reasoning about encountered information for more accurate responses.
  • Procedural memory involves storing knowledge of how to perform tasks, allowing the AI to execute learned procedures automatically, similar to how humans remember how to ride a bike or type on a keyboard. This component maintains the flow and context of interactions with users, ensuring the AI delivers coherent and contextually appropriate responses throughout a conversation.

Challenges with Existing Approaches

The prevalent approach for meeting memory system requirements is the use of special-purpose, standalone database management systems for various data workflows or types:

• In-Memory Databases: Used for caching and providing quick access to frequently needed data.
• Relational and non-relational databases: Used for operational and transactional data.
• Data warehouses and OLAP systems: Used for historical datasets and complex queries.
• Vector databases: Used for managing vectors and essential for tasks involving embeddings and similarity searches.

However, using a complex web of standalone databases can hurt an AI agent’s performance due to:

• Latency issues: Different databases have varying response times, leading to inefficiencies.
• Data silos: Disparate databases hinder comprehensive data analysis.
• Inconsistent data: Variations in data consistency and integrity can lead to errors.

Integrating these disparate databases into a cohesive, interoperable, and resilient memory system for AI agents is its own challenge. Many frequently used database services are not optimal for the speed and scalability that AI agent systems need, and their individual weaknesses are exacerbated in multi-agent systems.

The Future of Agentic AI