How to Secure Agentic AI During Development: Turning a Digital Insider Into a Useful Tool

Key takeaways:

• With non-human identities outnumbering humans more than 80:1 in corporate environments, the identity security challenge is already massive. AI agents amplify it by introducing machine identities that can act fully autonomously.
• Agents are susceptible to common AI and LLM attacks like prompt injection. On top of that, there are attacks that uniquely target agentic systems.
• Agentic solutions require special security measures on the architectural, agency, and communication levels.

Company-wide adoption of AI agents and multi-agent systems necessitates changes to corporate security posture. With traditional AI, the main business risk is a faulty output. AI agents, whether operating independently or within a multi-agent system, act autonomously. As a result, they may carry out undesirable actions instantaneously and independently.

That’s why agentic AI requires a special approach to security. In this article, we cover key controls for agentic AI security: built-in human oversight, zero trust for non-human identities, sandboxing agents so a compromised model can’t reach your infrastructure, and more.

This article is for security professionals and architects building or evaluating custom AI agent systems.

The state of agentic AI security

Enterprise adoption of AI agents is outpacing security readiness. 

By 2026, one in four organizations have moved beyond prompt-based AI toward autonomous systems chasing performance and automation benefits. At the same time, 92% of security professionals are concerned with the security implications of AI agents.

Most organizations have learned how to secure their chatbots and LLMs from simple attacks, but agentic AI is not just an LLM. 

Agentic AI combines a reasoning model with orchestration, memory, tools, and an execution environment. This architecture allows agents to autonomously plan and carry out multi‑step tasks across internal and external systems rather than respond to isolated prompts.

As a result, agents operate with delegated privileges similar to employees or internal services. They access APIs, manipulate data, and trigger workflows, effectively acting as non‑human identities (NHI) inside your infrastructure.

Treating agents like chatbots creates a blind spot in enterprise security. The security risks of AI are well-documented. But agents add a new level of risk: not just access to your sensitive records but the ability to act autonomously with them. Without supervision, AI agents can become digital insiders.

Vadim Nevidomy,
Head of AI at Apriorit

10 key security threats of agentic AI

Organizations are already struggling with identity security as non-human identities outnumber human users by more than 80 to 1. The rise of AI agents adds a new level of complexity, introducing identities that think, decide, and act with the same authority as humans.

This shift is reflected in emerging security frameworks. The OWASP Top 10 for Agentic Applications for 2026 highlights direct consequences of giving AI systems the ability to make decisions and act independently.

Table 1. Key agentic AI security risks

Threat	What it looks like	What it leads to
Agent goal hijack	An attacker injects hidden or malicious instructions so the agent silently adopts malicious objectives and action plans	– Confidential data leakage -Unauthorized transactions  – Manipulation of business decisions
Tool misuse & exploitation	Agents are tricked into using legitimate tools in unsafe ways (deleting data, calling over‑privileged APIs, following malicious links, etc.)	– Data loss or corruption – Exfiltration via legitimate APIs – Bill spikes – DoS from overuse
Identity & privilege abuse	Weak authentication lets attackers discover and call exposed agents, inherit their credentials, or exploit agent‑to‑service trust boundaries	– Unauthorized access  – Account takeover – Privilege escalation
Agentic supply chain vulnerabilities	The agent trusts malicious or compromised components in the agent supply chain	– Poisoned code or configurations in production – Systemic risk inherited from third‑party tooling
Unexpected code execution	Prompt injection or tool design flaws cause the agent to pass untrusted content into shells, CLIs, or code execution tools	– Remote compromise of endpoints and servers – Malware deployment- Infrastructure takeover
Memory & context poisoning	Attackers inject malicious content into long‑term memory, RAG indices, or context stores	– Long‑term data integrity issues – Recurring data leaks or bad decisions
Insecure inter‑agent communication	Agents communicate over unauthenticated or poorly secured channels; malicious agents or services spoof identities, intercept messages, or inject commands into multi‑agent protocols	– Cross‑agent data leakage – Unauthorized access to agents – Cascading compromise across workflows – Loss of trust in multi‑agent orchestration
Cascading failures	Small misconfigurations or prompt injections in one agent propagate through connected agents, tools, and systems	– System‑wide outages – Mass data leaks – Organization-wide corruption of business processes
Human–agent trust exploitation	Manipulated agents hallucinate data, hide mistakes, or present convincing but harmful recommendations that humans accept	– Bad business decisions – Security incidents
Rogue agents	Misconfigured, misaligned, or intentionally malicious agents operate with excessive autonomy, deviating from intended goals without active attacker control	– Unauthorized data access – Policy violations – Large‑scale process changes

While some of these attacks and exploits work on both generative AI applications and agentic AI, agentic systems provide malicious actors with more opportunities to steal data or disrupt a business. 

Securing agentic solutions from the attacks OWASP describes requires a combination of organizational and development practices. 

On the organizational side, this starts with treating AI agents the way you’d treat any privileged insider. That means:

• Defining what each agent is allowed to access and why
• Enforcing company-wide governance policies for agentic AI, including allowed and prohibited use cases, procedures to safely deploy new agents, and ways to report AI misconduct
• Conducting regular reviews for AI decision-making processes and internal reasoning

As a cybersecurity-focused vendor, we pay close attention to agentic AI security during design and development. Based on our AI development projects, we’ve figured out several must-have practices for building agents that remain secure over the long term.

Apriorit’s approach to building secure AI agents

Our methodology for securing AI agents during development and after release is based on secure SDLC principles and cybersecurity best practices regarding agentic AI. 

When we work on an AI agent from scratch, we bake security mechanisms into the architecture and keep an eye on the product’s security during development. If a client involves us in improving an existing product, we check its security features and focus on enhancing its integrations, access restrictions, and protection from specific attacks.

Here’s how we secure agentic AI:

1. Design for human-in-the-loop (HITL) and human-on-the-loop (HOTL)

Most discussions of human oversight treat it as binary: either a human approves every action or the agent runs autonomously. In practice, agentic systems require both patterns for processes with different levels of risk.

To ensure this control, we design agentic AI systems to allow for:

• Human-in-the-loop processes where an agent can’t act until a human explicitly authorizes the action. The agent drafts a decision and provides its reasoning, but it’s up to a human to assess whether that decision is what’s best for the business. This is the right approach for high-risk actions: financial transactions, data deletion, production deployments.
• Human-on-the-loop allows an agent to act autonomously while a human monitors and can intervene or override AI decisions. HOTL suits high-volume low-risk operations like routine data lookups, status updates, and standard API calls within pre-authorized scopes.

The timing of intervention is a crucial factor for both approaches. We prefer implementing pre-execution gates over post-hoc review, layered shutdown mechanisms over single kill switches, and real-time scope enforcement over static permission boundaries. This framework allows human reviewers to catch AI mistakes and security risks early and minimize their effects down the chain.

HITL checkpoints should be supported by both an agent’s architecture and UI. A simple “Approve?” prompt would be useless, as it simply nudges a human towards clicking on it. Instead, add a checklist covering intent, data lineage, permissions chain, and expected impact so that the approver acknowledges each item. 

To improve HITL and HOTL efficiency, teams that oversee agentic AI should be trained to recognize complacency cues like unusually large values or sudden scope expansion.

Vadim Nevidomy,
Head of AI at Apriorit

2. Build based on zero trust architecture

In agentic software, we implement zero trust not as a one-time authentication event but as continuous verification that every request is consistent with the established policy, the current context, and the assessed risk level. 

Compared to implementing zero trust for human users, implementing it for AI agents requires the following additional controls: 

• Continuous action-level verification. Every tool call, API request, and data access event is continually evaluated against the security policy. The agent’s trust posture can change mid-task based on what the agent does.
• Deterministic controls outside the reasoning loop. Because prompt injection, excessive agency, and model hallucination remain risks even in well-designed systems, we add controls that sit outside the LLM’s reasoning processes. Policy enforcement, scope boundaries, and permission checks must be handled by deterministic code. 
• Credential isolation. Agents should never hold backend credentials directly. Instead, they should authenticate to a gateway, and the gateway should handle actual credentials on the agent’s behalf.
• Confused deputy problem. Unlike traditional systems, where the deputy is a service with fixed permissions, here the deputy is an LLM that can be convinced to act against its rules through natural language. The zero trust approach helps prevent this issue. It allows us to treat inter-agent communication as an untrusted channel and enforce policy checks at every delegation boundary.

3. Implement the principle of least agency for NHIs

Only 8% of companies are sure their AI agents never exceed their intended permissions according to a survey conducted by the Cloud Security Alliance. 

Most agents today inherit human credentials or operate under shared service accounts. Yet, NHIs have more opportunity to abuse their access rights than humans: they move faster, have less security overhead, and don’t run their decisions by security officers unless specifically instructed.

Fixing this issue requires treating agents with the same rigor as human identities. OWASP’s Top 10 for Agentic Applications formalizes this through the principle of least agency — an extension of least privilege that limits both an agent’s access rights and degree of autonomy.

We implement this through layered access controls:

• Role-based access control to scope permissions to the agent’s specific task
• Attribute-based access control for dynamic permissions based on agent context and real-time risk level
• Short-lived, just-in-time credentials scoped to the task lifetime that are revoked automatically when the task is completed
• An MCP gateway that enforces consistent, least-privilege policies across all agent interactions with MCP servers and tools

Another risk of unchecked agent access is stale context. Without explicit boundaries, an agent may apply privileges obtained in one context to a completely different scenario. This way, it can expose sensitive data or trigger unintended workflows. 

To prevent this, we re-authenticate the user each time the agent calls an external system on their behalf.

Vadim Nevidomy,
Head of AI at Apriorit

4. Add sandboxing and restricted execution environments

Sandboxing lets agents execute code, access files, and use the network without compromising your credentials, local files, or host system. This additional level of isolation is essential when agents run autonomously, as it protects the rest of your environment from potential malicious activity.

Application-level controls alone are insufficient for agentic sandboxing. As the NVIDIA AI Red Team found, once an agent passes control to a subprocess, the application has no visibility into what that subprocess does. Attackers exploit this by calling restricted tools through approved ones. Sandbox enforcement must happen at the operating system or hypervisor level, beneath the application layer, so that every process in the sandbox is constrained regardless of how it was spawned.

That’s why we implement AI agent sandboxing across several layers:

• Network egress controls block all outbound connections by default and whitelist only required API endpoints. This prevents data exfiltration and reverse shell establishment.
• Filesystem boundaries block write operations outside the active workspace and do not allow changes to configuration files. Such boundaries stop persistence mechanisms, sandbox escapes, and exploitation of hooks or local MCP configurations.
• Secrets isolation allows us to define tools that run in the host environment and handle authentication there. This way, an agent never knows credentials. Per-session credential rotation ensures that even if a token is exfiltrated, it expires within minutes.
• Isolation primitive. Standard containers share the host kernel and are not sufficient for agentic workloads. We default to microVMs for untrusted or adversarial code and gVisor for compute-heavy multi-tenant deployments; we relax to hardened containers only where the threat model justifies it.
• Resource limits. CPU, memory, and wall-clock time caps per session prevent a compromised agent from starving the host even when it cannot escape the sandbox.

We also phase agent deployments to contain the possible area of an incident. Before any launch, we run a pre-mortem to map potential failures and their impact, then roll out in sections to internal sandbox, limited beta, and full production. Each deployment has automatic kill switches tied to error rate thresholds to quickly localize incidents and roll back to a stable agent version.

5. Prevent prompt injection and adversarial inputs

Prompt injection and adversarial inputs are two of the most common types of attacks on generative AI systems and LLMs. In agentic systems, the risk escalates because a successful injection hijacks an autonomous actor with valid permissions and creates an insider threat. 

In an attack on GitHub’s MCP server, researchers showed how malicious code posted to a public repository could inject hidden instructions into an AI agent. The agent followed the injected prompt, accessed the user’s private repositories, and exfiltrated sensitive data through an automatically created pull request.

Our practice shows that preventing prompt injection requires a layered strategy that includes:

• Input and output filtering to scan all content entering the agent’s context for known injection patterns
• Prompt partitioning to separate system instructions from external data at the architectural level
• Inter-agent validation that prevents successful injection of one agent from compromising other agents

On top of that, we recommend HITL controls for high-risk actions, actions with a large “blast radius”, and irreversible actions like sending an email, executing code, or deleting data.

6. Secure multi-agent communication

Multi-agent communication creates an extended attack surface for prompt injection, agent goal hijacking, and cascading attacks. 

To mitigate these risks, you should secure agentic systems across several layers:

• Mutual TLS (mTLS) for transport. Unlike API keys or OAuth tokens, mTLS binds agent identity to a private key that never leaves the agent. Both the calling and receiving agent present X.509 certificates before any data is exchanged.
• Signed capability discovery. Agents find each other at runtime by reading signed capability cards. Without cryptographic signing and pinned keys, an attacker can publish a lookalike card, but the agent will validate the signature instead of dispatching tasks straightaway.
• Third-party component validation. We extend security requirements to third-party tools through contracts and due diligence that include auditability, kill switches, and red-team testing. Governance policies are managed so that when legal and regulatory requirements change, enforcement updates propagate automatically across all agents.

7. Keep immutable audit logs

Detailed and immutable logging of AI activity and decisions is a must for security investigations and compliance audits. It’s also a part of making your AI agents explainable, which becomes a product requirement from both regulatory bodies and most customers.

We configure logging for all agent actions, including decisions, tool calls, code execution, and API requests. Logs should be stored in tamper-resistant, cryptographically signed files. 

On top of logging, anomaly detection monitors for unexpected network connections, excessive API calls, unusual resource consumption, and failed access attempts as indicators of potential compromise.

We recommend capturing structured audit logs after every kill switch event to support investigation, accountability, and rollback decisions.

How Apriorit helps secure agentic AI solutions

In production systems, risk usually appears where agents connect to real infrastructure. That’s why securing an agentic AI solution requires control over architecture, permissions, integrations, and runtime behavior.

With 20+ years of experience in cybersecurity and system engineering, we help teams design agentic systems with clear access boundaries, controlled autonomy, and security checks built into the delivery process. 

Relevant Apriorit expertise includes:

• Security-first development. We add security checks and measures to every stage of your AI agent development process. Secure SDLC principles help us comply with high security standards and simplify future product maintenance.
• AI consulting. With real experience in delivering AI agents, LLM-based systems, chatbots, AI-powered data processing, and other types of AI and ML solutions, we can assess your project plan, identify risks, and suggest practical improvement steps.
• Design and development for agentic AI. We design architecture for your AI agents with a focus on trust boundaries, autonomy limits, privilege scoping, and security mechanisms to prevent common exploits like prompt injection.
• API and integration security — We secure authentication, authorization, data flows, and tool connections across agent workflows.
• AI-focused security testing — We test for prompt injection, unsafe execution paths, misclassification, and unintended agent behavior.

FAQ