Large Language Models have transformed how we approach content creation, code assistance, and data analysis. They can summarize documents, explain complex topics, generate creative content, and write code with impressive accuracy. But despite their capabilities, LLMs alone are not enough to power reliable, production-grade AI systems. The gap between an impressive demo and a dependable business application remains wide. This is where AI agents come in: they provide the structure, control, and integration LLMs lack.
Let’s examine the core limitations of LLMs and how agentic frameworks overcome them and enable agents that solve real-world challenges.
LLMs Need Direction
When you start interacting with an LLM directly, you're having a conversation with a highly capable but directionless system. Without proper guidance, even the most advanced models can produce inconsistent, irrelevant, or even harmful outputs. This is where prompt engineering becomes critical. A well-crafted system prompt serves multiple purposes:
- Baseline Protection: Establishes boundaries and prevents the model from generating inappropriate content
- Role Definition: Focuses the model's responses within a specific domain or expertise area
- Output Structure: Defines expected response formats, schemas, and quality standards
- Context Setting: Provides examples and instructions that guide reasoning patterns
For example, a customer support agent needs different instructions than a code review agent. The support agent requires empathy, company policy knowledge, and escalation procedures. The code reviewer needs technical accuracy, security awareness, and style consistency.
A good agentic framework should support structured instruction management by design. It should enable developers to define reusable prompt templates, inject dynamic context, and enforce consistent behavior across scenarios. These are essential to building reliable agents that adapt to production needs and evolving model capabilities.
LLMs Need Context
The primary interaction method with LLMs is conversational, involving back-and-forth exchanges that build upon previous interactions. Yet LLMs themselves are fundamentally stateless. Each API call starts fresh, with no awareness of prior conversations, learned preferences, or relevant external information. This creates an immediate problem for any practical application. A travel planning agent that can't remember your preferences or access current flight prices is barely more useful than a search engine. A coding assistant that forgets your project's architecture with each interaction becomes frustrating rather than helpful.
Agentic applications like ChatGPT or Gemini add memory (and many other capabilities such as tool calling, memory, preferences) on top of LLMs to turn them into useful applications. These layers manage context persistence and user preferences over time, compensating for the underlying statelessness of the model.
A well-designed agentic framework should include support for memory management tailored to different needs. This allows agents to maintain relevant context, adapt over time, and deliver coherent interactions. Depending on the task and tools involved, different memory types may be required:
- Session Memory: Captures short-term conversation history during a single interaction.
- User Memory: Stores long-term preferences and learned behaviors across sessions.
- Knowledge Memory: Retains evolving domain-specific information useful for reasoning or decision-making.
Memory management is part of the broader topic of context engineering, which involves filling the context window with just the right information at each step of an agent's execution.
As agents engage in longer conversations and accumulate tool feedback, they quickly hit context limits and performance degradation. Context engineering addresses this through four key aspects:
- writing context outside the window (scratchpads, memories),
- selecting relevant context (RAG, memory retrieval),
- compressing context (summarization, trimming), and
- isolating context (multi-agent systems, sandboxing).
Effective context engineering is critical because poorly managed context leads to hallucinations, distractions, and confused responses, turning capable agents into unreliable systems.
LLMs Need Environment Access
Despite training on vast datasets, LLMs have fundamental knowledge gaps:
- Temporal Limitations: No awareness of events after their training cutoff
- Private Data: No access to your company's internal processes, customer data, or proprietary systems
- Real-time Information: No ability to fetch current data or respond to live events
This is why agents need environmental access to retrieve data and take action in the real world. A well-architected agentic framework should support this through structured tool integration, including:
- Function Calling: Letting models invoke tools with clearly defined parameters.
- Model Context Protocol (MCP): Standardizing how agents connect to internal systems and data sources, external systems and APIs.
These capabilities enable agents to interact with databases, APIs, and third-party services, bridging the gap between language understanding and real-world execution.
LLMs Need Human Oversight
LLMs can reason about complex problems and generate sophisticated plans, but they lack the judgment to distinguish between routine tasks and decisions that require human approval. An LLM can be highly confident about sending an email to thousands of customers announcing a product recall or price change, without understanding the potential business impact or timing sensitivity. Without human oversight, these systems can make costly or dangerous decisions with complete confidence.
Modern frameworks address some of these concerns through automated guardrails using additional LLMs to validate inputs and outputs before execution, as demonstrated in OpenAI's Agents SDK guardrails. However, human oversight remains essential for high-stakes decisions where automated validation isn't sufficient.
This is where human-in-the-loop (HITL) patterns become essential. Critical decisions such as financial transactions above certain thresholds, irreversible actions, or operations affecting sensitive systems need human validation before execution. The challenge isn't just detecting when human input is required, but managing the flow to interruption gracefully while waiting for human review.
A powerful agentic framework must provide mechanisms to pause execution at designated approval points, route requests to appropriate human reviewers, and resume workflows seamlessly once approval is granted or denied. This includes handling timeout scenarios where humans don't respond within expected timeframes, maintaining workflow state during potentially long approval cycles, and providing clear audit trails of human decisions. The goal is to ensure that AI systems enhance human decision-making rather than bypass it entirely, particularly for actions with significant business impact or risk.
LLMs Need to Fit the Problem
LLMs are evolving rapidly, models become cheaper, more capable, and the ecosystem becomes more diverse. Today’s landscape includes models that reason deeply, models that don’t, small models optimized for latency and cost, open-source and on-premise models, and offerings with different context sizes, caching behavior, and APIs. This diversity means there’s no single best model or provider. In fact, organizational constraints such as regulatory requirements and cost often dictate what can or cannot be used.
A well-designed agentic system must support selecting the right model for the specific task or environment. Different parts of an agent workflow may require different models, for example, one for summarization, another for classification, a third for reasoning. Development and production environments may use different providers. Deployment targets (cloud, edge, or air-gapped environments) may demand different configurations.
Agents should be designed to swap models easily, without major code changes or redeployments. This flexibility is essential for adapting to evolving models, pricing, capabilities, and compliance needs without breaking the core logic of the system.
Functional Requirements for Agent Frameworks
An agent is a cohesive unit that combines the right model with tailored instructions, memory, execution, and access to its environment and humans. A good agentic framework should enable developers to easily pick, use, and change every aspect of the agent:
- Tailored Instructions: Domain-specific guidance and structured examples
- Memory: The ability to retain conversation state and broader context
- Relevant Tools: Only what’s needed for the agent’s defined function
- Human-in-the-Loop: Allow human oversight and course correction when needed
- Appropriate Model: A model matched to task complexity, cost, and latency
The purpose of an agentic framework is to let developers create and orchestrate such agents with ease, while enabling operations teams to deploy and manage them securely and reliably in production.
At the same time, real-world needs can rarely be addressed by a single agent. Typically, agentic systems are built from multiple specialized and narrowly scoped agents, each focused on a clear task. These agents can be combined into multi-agent systems (MAS) that support both structured orchestration through workflows or autonomous collaboration across evolving, complex tasks.
Agentic frameworks should enable developers to compose systems from multiple specialized agents, each with distinct roles, tools, and expertise. As problems grow in complexity, frameworks must support different orchestration patterns: manager-style coordination where a central agent delegates to specialists, or decentralized handoffs where peer agents transfer control based on their expertise.
How Does Dapr Address Agentic Needs?
The GenAI space is in rapid change with models evolving, pricing shifts, and new capabilities emerging unpredictably. Building directly against specific providers creates brittle dependencies that degrade over time. A robust agentic system must adapt to several dynamics:
- Provider Competition: As new models emerge and pricing improves, teams must evaluate alternatives without rewriting their agents.
- Model Evolution: Prompting strategies that work today may break with future model versions.
- Tool Ecosystem: New APIs and integrations appear regularly. Agents should be able to adopt new tools without architectural change.
- Business Requirements: As organizations grow, agent interactions must evolve. Frameworks should support discovery, connection, and coordination with new agents.
- Infrastructure Changes: Agents must work across dev, staging, and production environments, adapting to backing services without code modification.
- Production Integration: At scale, agents must connect to identity providers, observability stacks, compliance systems, and enterprise infrastructure.
To manage this change, systems must enforce a clean separation of concerns. AI engineers should be able to iterate on agent logic independently, while operations teams manage infrastructure, configuration, and compliance.
Dapr Agents is created to address these challenges. It is built on top of Dapr by combining stateful workflow coordination with agentic needs. Rather than reinventing the wheel and creating yet another AI application stack from scratch, it provides Dapr's proven APIs and infrastructure abstractions as an easy-to-use Python library.
Dapr Agents’ primary goal is to express business needs with a fluent Pythonic domain-specific language (DSL), isolated from the rapidly evolving AI infrastructure. The DSL combines prompts, storage, models, and orchestration logic, while Dapr’s YAML-defined components handle the underlying infrastructure such as LLM providers, storage, state management, observability, resiliency, and more.
This approach offers several key advantages:
- Familiar Programming Model: Developers use standard Python APIs to define agents, tools, and workflows without reinventing distributed system building blocks.
- Infrastructure Abstraction: The Dapr sidecar model separates agent logic from backing services. Operations teams can swap Redis for PostgreSQL, change LLM providers, or update observability stacks without touching agent code.
- Production-Ready Reliability: Built on top of Dapr, a trusted enterprise framework that covers observability, security, and resiliency at scale used in governments and thousands of companies worldwide.
The agent development ecosystem spans a wide spectrum. Some frameworks target business users with no-code interfaces like Bubble.io's visual agent builder. Others use semi-technical approaches like CrewAI's higher-level abstractions for role-based agent collaboration. At the technical end, frameworks like LangGraph require graph-based programming expertise to build complex state machines and execution flows. Dapr Agents takes a different approach: it extends proven distributed systems capabilities with lightweight AI abstractions, enabling developers to build production-ready agentic systems without reinventing infrastructure primitives. This allows teams to focus on agent logic while inheriting enterprise-grade reliability, security, and observability from the battle-tested Dapr ecosystem.
Ready to explore more? Dive into the free Dapr Agents University Course or try the Dapr Agents quickstarts.
