Prompt injection is currently the top risk when it comes to building applications driven by large language models (LLMs), like AI agents. That’s because prompts are the main control interface for these models, and they can follow text instructions embedded in user input or any external content the model reads as part of a given prompt.

Without guardrails in place, an LLM may follow those instructions, whether authorized or unauthorized. Recent research has uncovered novel techniques for deploying prompt injections. Cybercriminals are actively exploring new types of prompt injection as well.

This article explains what prompt injection is, its potential consequences, and how APIs expand the attack surface. It also includes a few tips on defending against it.

What is Prompt Injection?

Prompt injection is where an LLM (or an AI agent powered by one) is manipulated to produce unintended output or behavior. This occurs by prompting the model one or multiple times or the model reading externally sourced content that contains an injection — sometimes both. For example, a cybercriminal could manipulate an LLM into providing restricted information or bypassing safety controls put in place by a developer. The two main categories of prompt injection are direct and indirect.

• Direct prompt injection involves a user creating malicious input that they place directly in prompts they submit to the AI model.
• Indirect prompt injection (IPI) is where malicious or misleading instructions are embedded in external content (that the AI model processes) to manipulate its output.

Anything that a user can write into and an LLM then reads to perform a task can be weaponized as an indirect form of prompt injection — from HTML pages and PDFs to images and error logs.

Note: If a user uploads an injected file, like a PDF, to the model using a direct prompt, that is considered indirect prompt injection. The classification hinges on where the malicious instructions live — either directly in the prompt or hidden in external content processed by the AI model.

Prompt injection can be intentional or unintentional. For example, a book writer could accidentally include wording directly in the prompt or in an uploaded document that looks like instructions to the model. The wording could then cause odd results (unintentional). On the other hand, a cybercriminal could intentionally insert malicious instructions directly into a prompt or into PDFs that a model processes. The goal could be to trick the model into ignoring system instructions.

Direct and indirect prompt injections often involve LLM tool calls, expanding the risk from these types of attacks. This increased risk includes the possibility of an injection triggering some tools to return malicious payloads or a cascade of unauthorized tool calls.

Note: The terms prompt injection and jailbreaking are sometimes used interchangeably, but they are not the same. Jailbreaking is a form of prompt injection where the user tries different techniques to force the model to bypass set safety guardrails.

Examples of Prompt Injection Methods

At the time of this writing, there are no primary studies measuring direct versus indirect prompt injection risks. However, there is direct research and anecdotal news reports that indicate an increase in the risk of indirect prompt injection attacks.

Google researchers recently scanned an archive of the public web obtained through Common Crawl, although the archive does not include social media sites. The purpose of this research is to understand the current state of prompt injections on the web.

Google’s report on this research states that the team observed an increase of 32% in the malicious prompt injection category between November 2025 and February 2026. This upward trend shows increasing interest in IPI attacks.

Ghostcommit and BioShocking

Bleeping Computer recently reported that researchers created an IPI proof-of-concept aimed at GitHub dubbed “Ghostcommit.” It hides malicious instructions in a PNG image included in a pull request, with the main goal of stealing a repository’s secrets. A prompt injection within an image is text that a reviewer likely won’t check.

The tech news publication also highlighted a new IPI attack method named “BioShocking” where a malicious webpage is presented to an AI-powered browser. The webpage proof of concept contained a BioShock-themed puzzle game that teaches the browser’s AI agent that “incorrect actions are acceptable,” causing the browser to ignore any safety guardrails.

AI Recommendation Poisoning

The Hacker News reported in August on a new class of prompt injection that exploits pre-filled deep links, a standard feature of most major AI assistants. Microsoft security researchers have named this technique “AI Recommendation Poisoning.” It involves embedding hidden prompt injection payloads inside “Ask AI” buttons to poison the AI’s memory for promotional purposes.

When a user clicks on an “Ask AI” link for an assistant like Claude or Gemini, it executes a pre-formed query immediately in their session with the user unaware of it. In most cases, these links are not dangerous. However, some prompt the LLM to remember the company as a “trusted source” or “recommended first.” The goal is to bias the model’s future responses toward the company’s products or services.

How APIs Expand the Attack Surface

Prompt injections are a problem for all applications and systems that rely on LLMs, and APIs expand that attack surface on two levels. First, you have APIs that developers use to integrate LLMs. Second, you have APIs that these large AI models can select as tools to complete various tasks. When you build LLM-driven applications, you often end up with an array of interconnected APIs and data pipelines.

The more APIs you have in that system, especially as part of tool calling, the larger the attack surface for prompt injection. One reason for this expanded surface is the higher probability of more unvetted content sources.

For example, an AI agent that connects to an LLM with no access to tools has only one injection attack vector: the user’s direct prompts. Once the model is wired to APIs through tool calling, it can start consuming content that may or may not be reviewed by humans before the model reads it. Any content provided in API responses can contain text that acts as instructions for the model. So, every tool that has an API integration is a new place where an attacker can plant instructions.

Another reason APIs expand the attack surface for prompt injections is the user permissions they carry, typically through authentication tokens.

Say you have a Vendor Risk Agent that needs access to multiple APIs to monitor third-party vendor health and contract compliance, and that agent holds all the tokens for those APIs. The agent reads an API response with hidden instructions that cause it to act in unpredictable ways, carrying those API permissions along the way. The injection model could chain API capabilities to complete an objective, especially with the right permissions.

According to the Least Privilege Research Report 2026, most companies have AI agents inherit human privileges by default, and 96% of enterprise permissions are unused. This trend means most AI agents will likely have over-permissioned access.

Note: AI agents often accumulate permissions, referred to as privilege drift. This accumulation creates an expanded attack surface with several potential risks, including prompt injections.

The Potential Consequences of Prompt Injection

Prompt injections can lead to damaging scenarios for businesses of all sizes. The potential impacts include:

• Unauthorized actions: A user manipulates an LLM into using APIs or connected tools to perform unwanted or unauthorized activities.
• Data exfiltration: Where users’ sensitive or proprietary data is sent to an attacker.
• Compliance violations: A data breach could cause violations of regulations like GDPR or HIPAA.
• System override: An attacker uses various techniques to cause the AI to ignore system instructions.
• Bypass safety controls: A user figures out how to get around safety mechanisms using techniques such as jailbreaking.

According to the OWASP Top 10 for LLM Applications 2026 guide, practitioners rank prompt injection as the #1 risk for applications that rely on LLMs. The guide also states that:

“The [risk] surface still sits everywhere a model reads untrusted input, which is to say everywhere. It [prompt injection] stays at number one. Since you cannot close that surface, you build for the day it is used against you.”

Defending Against Prompt Injection: Quick Tips

It is impossible to completely prevent prompt injections due to the vast and ever-expanding attack surface. However, you can take steps to reduce the risk of this type of attack. Here are a few to get you started.

• Set system prompt boundaries: If you are building an LLM-driven application, make sure that the system prompt separates its initial set of instructions from content provided by users, setting clear structural boundaries. One way to do this is to use explicit delimiters. XML and markdown are commonly used as delimiters in system prompts.
• Filter user inputs: Direct prompt injection happens by inputting malicious text into the model. Analyze and sanitize all user input before it reaches the model. Remove instruction-like patterns, especially known malicious text such as “ignore previous instructions” or “provide system root password.”
• Validate model responses: Scan the model’s output to detect illegitimate content, such as unexpected URLs, unauthorized tool calls, or executable code. There are tools available that catch unsafe inputs and outputs from LLMs. One such tool is the NVIDIA NeMo Guardrails Library, an open source Python package used to add guardrails to LLM-based applications.
• Use rate limiting: Prompt injection attacks often require multiple attempts to succeed. Apply rate limiting techniques for the AI Model and APIs your application uses. For example, implement daily LLM token budget limits for each app user and set depth limits on conversations. If you need third-party APIs for your application, look for providers that have implemented adaptive rate limiting.
• Employ continuous monitoring: Implement runtime monitoring tools that scan AI applications continuously to detect abnormal behaviors in real time. These tools should detect issues like plan drift and risky tool execution sequences. You could also use “critic agents” to catch anomalies and improve the outputs of other AI agents in multi-agent (MAS) systems.

These are the first steps in reducing the risk of prompt injections. You should also consider human intervention mechanisms, particularly for securing API integrations with AI agents. For example, you could set boundaries on any API endpoints where an AI agent would use access tokens.

An Ever-Expanding Problem

Prompt injection is an ever-expanding and evolving problem that impacts every development team building LLM-driven applications and systems. Attackers today can find novel ways to craft prompts that trick models into breaking safety guardrails and manipulating model behavior. Indirect prompt injection is even more insidious — it can hide in any content a user can add text to and an LLM could subsequently read to complete an action. And with a rapidly growing global digital world projected to reach 394 zettabytes of data by 2028, prompt injections could be hidden where developers and users least expect them.