AI Agents Need More Than Guardrails: Engineering Secure Agentic Systems

AI agents can read untrusted content, call tools, access data, delegate work, and take actions across connected systems. That combination creates security risks that are different from those of a chatbot that only generates text.

Prompt instructions and model-level guardrails are useful, but they are not security boundaries. A robust agent system must remain constrained when a model misunderstands a task, follows malicious instructions embedded in retrieved content, selects an unsafe tool, or behaves in an unexpected way.

The engineering goal is not to make the model perfectly trustworthy. It is to design the system so that a model mistake or compromise cannot automatically become an unauthorized action.

This guide brings together practical controls from the OWASP Top 10 for Agentic Applications, the OWASP AI Agent Security Cheat Sheet, and security guidance from NVIDIA, alongside risk-management guidance from NIST.

1. Separate model behavior from security enforcement

A model can be instructed to avoid sensitive data, but the instruction itself does not prevent a tool from returning that data. A model can be asked to use approved APIs, but the prompt does not stop the runtime from reaching an unapproved endpoint.

Enforce critical restrictions outside the model:

Authorization: Check the agent identity, task, requested operation, target resource, and current permissions before each tool call.
Data access: Enforce access controls in the database, API, retrieval layer, and file system, not only in the prompt.
Execution: Run code and tools in isolated environments with explicit resource and capability limits.
Network access: Apply default-deny egress where practical, then allow only required destinations and protocols.
Human approval: Require explicit, action-specific approval for consequential operations when the risk warrants it.

Guardrails can detect or shape model behavior. Independent policy enforcement limits what the system can actually do. Use both, and do not treat one as a replacement for the other.

2. Apply least privilege to every task

An agent should receive only the permissions required for its current task, not everything its application or user could access.

For example, an agent summarizing support tickets may need read-only access to a defined ticket collection. It should not automatically receive permission to export the customer database, change account roles, or send messages to every customer.

Practical controls include:

Give each agent or workload a distinct identity.
Scope permissions to specific resources and operations.
Prefer read-only access unless a write operation is necessary.
Issue short-lived, task-scoped credentials where supported.
Separate permission to propose an action from permission to execute it.
Revoke access when the task ends or the workflow is cancelled.

Avoid broad shared service accounts. A compromised agent should not inherit the permissions of every other agent in the application.

3. Treat prompt injection as an input trust problem

Prompt injection can arrive through web pages, emails, documents, code comments, issue descriptions, retrieved passages, or tool output. The content may tell an agent to ignore prior instructions, disclose secrets, or call a tool on the attacker’s behalf.

There is no single prompt pattern that reliably eliminates this risk. Instead, treat external content as untrusted data and reduce the impact if an agent follows malicious instructions.

Separate trusted instructions from retrieved or user-supplied content.
Give tools narrow schemas and validate all arguments outside the model.
Check tool calls against policy before execution.
Keep secrets out of model context unless access is necessary.
Limit the actions available to agents processing untrusted content.
Test direct and indirect prompt injection paths, including instructions returned by tools.

Content filters and guardrail frameworks can add useful detection. For example, NVIDIA NeMo Guardrails agentic security documents controls for agentic workflows. Such controls should be one layer in a defense-in-depth design, not the only barrier protecting privileged operations.

4. Isolate execution and control network access

Tool execution is where model decisions can become system-level consequences. Running agent code with the same privileges and network access as the host application increases the blast radius of a mistake or exploit.

Use isolation appropriate to the risk:

Execute generated code in disposable sandboxes or isolated workers.
Set CPU, memory, execution-time, file-system, and process limits.
Mount only the files required for the task.
Keep host credentials and sensitive environment variables outside the execution context.
Restrict outbound network connections and block access to cloud metadata endpoints unless explicitly required.
Separate development, staging, and production identities and environments.

A sandbox is not a complete security solution by itself. Combine it with access controls, network policy, monitoring, and a clear boundary between the agent runtime and sensitive systems.

Vendor implementations can help illustrate these patterns. NVIDIA’s Four Ways to Deploy More Secure AI Agents and OpenShell describe approaches to controlled execution and runtime security. Evaluate any implementation against your own threat model and deployment environment rather than assuming a vendor-specific mechanism is universally required.

5. Protect credentials, memory, and inter-agent communication

Agents can expose sensitive information through prompts, logs, tool results, persistent memory, or delegated tasks. Memory and agent-to-agent communication should be treated as security-sensitive data paths.

Credentials

Store secrets in a secrets manager or equivalent protected service.
Avoid embedding credentials in prompts, source code, logs, or persistent memory.
Retrieve credentials only when needed and scope them to the smallest practical set of operations.
Rotate credentials and audit access.

Memory and retrieved data

Enforce authorization on every retrieval, not just when data is first written.
Track provenance where it helps distinguish trusted records from untrusted content.
Define retention and deletion policies.
Prevent one user’s or tenant’s memory from being exposed to another.
Treat stored agent notes and summaries as data, not as trusted instructions.

Multi-agent workflows

Authenticate agents and services to each other.
Authorize delegation and constrain what context and tools are passed downstream.
Preserve the original task’s permission limits when work is delegated.
Record which agent initiated an action and which component executed it.

Delegation must not become a way to bypass permissions. A sub-agent should not receive broader access merely because another agent asked it to perform the work.

6. Put explicit limits on autonomy

An agent can loop, repeatedly call an expensive tool, delegate too broadly, or continue acting after the original task is no longer valid. Define limits in the runtime and orchestration layer.

Useful limits include maximum execution time, tool-call count, delegation depth, token or cost budget, retry count, and the set of permitted actions. Add cancellation and timeout behavior, and fail closed when authorization or policy checks cannot be completed.

For sensitive operations, use a controlled action boundary:

The agent proposes an action with validated parameters.
A policy layer checks the actor, target, scope, and risk.
A human approves the specific action when required.
The system revalidates authorization immediately before execution.
The result is recorded and the approval cannot be reused for unrelated actions.

A general approval such as “you may manage accounts” is not equivalent to approving a specific account change. Approval should be understandable, scoped, and time-bound.

7. Make monitoring and audit part of the design

Security teams need enough evidence to reconstruct what happened without logging unnecessary sensitive content.

Capture events such as the task and agent identity, tool requested, policy decision, resource accessed, approval reference, execution result, and relevant timestamps. Protect logs against tampering and restrict access to them. Redact credentials and sensitive payloads.

Monitoring should detect unusual behavior such as unexpected tool sequences, repeated policy denials, abnormal data access, unexpected network destinations, and unusual delegation patterns. Alerts are most useful when connected to a response process that can suspend a workflow, revoke credentials, or isolate a runtime.

NVIDIA’s Sentry is an example of a hardware-based runtime security approach in a particular NVIDIA deployment context. It illustrates the value of enforcement and monitoring beyond model prompts, but it should not be interpreted as a generic feature available in every agent stack.

8. Test security as a release requirement

A successful demonstration does not show that an agent is secure. Test expected behavior and failure cases before release, and repeat those tests when models, tools, prompts, permissions, dependencies, or orchestration logic change.

Build a test suite around realistic threats:

Direct and indirect prompt injection.
Attempts to access data outside the current user’s authorization.
Malicious or malformed tool arguments.
Secret exposure through responses, logs, and memory.
Unauthorized writes and high-impact actions.
Excessive retries, tool calls, cost, or delegation.
Cross-tenant memory and retrieval leakage.
Network access outside the approved destinations.
Failures of the policy service, credential provider, or approval workflow.

For each test, define the expected security outcome and preserve evidence. Include regression cases from incidents and near misses. Where possible, automate these tests in CI and run additional adversarial exercises against deployed workflows.

The NIST AI Risk Management Framework and its Generative AI Profile can help teams organize governance, measurement, and risk treatment. They complement technical controls; they do not replace system-specific threat modeling and testing.

A practical baseline

Before deploying an agent with access to real systems, verify that:

Conclusion

Secure agentic systems are built by constraining capabilities, enforcing policy independently of the model, and testing how the system behaves under hostile or unexpected conditions.

Guardrails remain useful, but they are not the security boundary. The boundary is the combination of identity, authorization, isolation, network policy, credential protection, runtime limits, human oversight, monitoring, and repeatable security testing.

Start with the tools and data that create the greatest potential impact. Limit what the agent can reach, validate every consequential action, and collect evidence that those controls work. That is a stronger foundation for production agents than relying on the model to always make the right decision.