Poisoning the Library, or Hijacking the Reader?

There are two completely different things in AI security that both get called “injection,” and confusing them will cost you. One is an attack on the conversation. The other is an attack on the library the AI reads from. They have different attackers, different lifespans, and almost entirely different defences — and the second one is the one most people building with AI right now have not thought about.

This article is about that distinction. It is worth being precise, because the security literature is precise, and the difference between “someone tricked the chatbot” and “someone edited the chatbot’s sources” is the difference between a bad afternoon and a problem that persists until you find the document.

The word “injection” is doing two jobs

Borrowed from SQL injection, the term describes a general shape of attack: hostile input reaches a system that cannot tell the difference between the data it was handed and the instructions it is supposed to follow. In AI there are two places that can happen.

Both end with untrusted text in the model’s context. The difference is how it got there and how long it stays, and that difference drives everything else.

What prompt injection actually is

The term has a precise origin. A company called Preamble found the flaw in GPT-3 and disclosed it privately to OpenAI on 3 May 2022. [1] Riley Goodside demonstrated it publicly that September. Simon Willison named it on 12 September 2022, writing: “This isn’t just an interesting academic trick: it’s a form of security exploit. I propose that the obvious name for this should be prompt injection.” [2]

He proposed a fix in the same post — “parameterized prompts,” by analogy with the fix that actually solved SQL injection. He retracted it nineteen months later, in an update appended to that same post: “It’s becoming increasingly clear over time that this ‘parameterized prompts’ solution to prompt injection is extremely difficult, if not impossible, to implement on the current architecture of large language models.” [2] That retraction is worth more than most of what has been written on the subject since.

Why it has not been fixed

The UK’s National Cyber Security Centre published the clearest explanation in December 2025, and it is worth quoting directly because it is the whole problem in one sentence:

That is the architectural fact everything else follows from. SQL injection was solved because a database engine can be told, structurally, that a piece of input is data and must never be executed as a command. Parameterised queries work because that boundary genuinely exists. In a language model there is no such boundary to enforce. Your system prompt, the user’s question, and the contents of a retrieved web page all become one sequence of tokens.

The NCSC’s conclusion is blunt: prompt injection attacks “may never be totally mitigated in the way that SQL injection attacks can be”, and every current defence is “trying to overlay a concept of ‘instruction’ and ‘data’ on a technology that inherently does not distinguish between the two.” [4]

This is not a fringe view. OWASP’s 2026 edition states that no reliable prevention mechanism exists today and that defence must therefore be architectural rather than interceptive. [5] OpenAI’s CISO called prompt injection “a frontier, unsolved security problem” in October 2025. [6] Anthropic, publishing genuinely improved defences in November 2025, wrote that “a 1% attack success rate — while a significant improvement — still represents meaningful risk” and that “no browser agent is immune.” [7]

Prompt injection has been the number one entry in OWASP’s Top 10 for LLM Applications for three consecutive editions. It is LLM01:2026 in the current one. [5]

Direct, and indirect — the distinction that matters here

Early prompt injection meant a user typing something clever into a box. That is the direct form, and it is largely a problem for whoever owns the box.

In February 2023, Greshake and colleagues described the form that matters for retrieval systems. Their paper is titled — and this remains the best summary anyone has managed — “Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection.” [8]

Indirect injection means the attacker never touches your application. They put the hostile text somewhere your application will later go and read: a web page, a support ticket, a shared document, an email. The paper’s formulation is that this lets adversaries “remotely (without a direct interface) exploit LLM-integrated applications by strategically injecting prompts into data likely to be retrieved.”

They demonstrated it against real production systems, and closed with a line that has aged well: “Despite the increasing integration and reliance on LLMs, effective mitigations of these emerging threats are currently lacking.”

What retrieval changes

Here is the asymmetry, and it is the reason this article exists.

To prompt-inject a bare model, an attacker needs to get text into a specific conversation. To poison a RAG system, an attacker needs only to get text into a corpus your system trusts — and then wait. They do not need to know who will ask, or when, or what your prompt looks like.

Ask yourself which of your sources an outsider can write to. The public web, obviously. But also: wikis, support ticket queues, shared drives, customer-submitted content, code repositories, email. Anywhere a document can be created by someone who is not you is a place a poisoned document can be created.

The researchers behind ConfusedPilot, working against Copilot for Microsoft 365, put the structural point precisely: with a bare model “the entity that writes the prompt is usually the same one who sees the response” — so the damage is contained. But “unlike traditional LLMs where the information is embedded in the model weights through training, in RAG, the information resides in a database. This provides an attack vector in addition to the prompt itself” — and documents inside an organisation get “created, shared, and edited to different entities within the enterprise, sometimes even without the entities’ awareness.” [9]

How well does corpus poisoning actually work?

The reference result is PoisonedRAG, published at USENIX Security 2025. [10] The authors deliberately gave their attacker almost nothing: it cannot read the knowledge database, cannot query the model, and in the black-box setting cannot even query the retriever.

Their abstract reports a 90% attack success rate from injecting five malicious texts per targeted question into a knowledge database containing millions of texts. On one specific configuration — the Natural Questions dataset, black-box, against PaLM 2 — they report 97% against a corpus of 2,681,468 clean documents.

Two findings matter more than the headline.

Poisoning the weights vs poisoning the index

These get conflated constantly, and they are different attacks with different remedies.

Training-time poisoning corrupts the data a model learns from. Carlini and colleagues showed this is practical and cheap: they calculated they could have poisoned 0.01% of the LAION-400M dataset for about $60. [12]

A 2025 study from the UK AI Security Institute, Anthropic and the Alan Turing Institute found something more counterintuitive. Poisoning appears to require a near-constant number of documents regardless of model size — roughly 250 documents backdoored models from 600M to 13B parameters, which at the largest size was 0.00016% of training tokens. [13] The intuition that a bigger model is harder to poison because your poison is a smaller fraction of its diet appears to be wrong.

Their own caveat deserves equal billing, and is usually dropped: the backdoor they induced makes the model produce gibberish, which they describe as “unlikely to pose significant risks in frontier models,” and they state plainly that it is unclear whether the finding extends to more complex behaviours.

Retrieval-time poisoning is the RAG case, and in almost every respect it is the easier attack and the easier fix.

That last column is the good news in this article. RAG widens the attack surface, but it also puts the attack somewhere you can see it and delete it. A poisoned model is a much worse position to be in.

This is not hypothetical any more

In March 2026, Palo Alto’s Unit 42 published telemetry analysis with a specific claim: prior research had focused on theoretical risk, but “our analysis of large-scale real-world telemetry shows that IDPI is no longer merely theoretical but is being actively weaponized.” [14] They catalogued 22 distinct techniques, and found one page carrying 24 separate injection attempts, with the attackers having labelled their own layers in the HTML.

Three cases worth knowing:

What actually defends — and what the measurements say

This is where most coverage becomes useless, because it lists mitigations without telling you whether any of them were tested. So here is the uncomfortable summary first, and then the detail.

In October 2025, fourteen researchers from OpenAI, Anthropic and Google DeepMind published a paper testing twelve recent prompt-injection and jailbreak defences against adaptive attackers — attackers who know what the defence is and try to route around it. They bypassed most of them with attack success above 90%, and noted that “the majority of defenses originally reported near-zero attack success rates.” [18]

The clearest single example is spotlighting, a Microsoft technique that marks untrusted text so the model can distinguish it. When published in March 2024 it reduced attack success from over 50% to below 2%. [19] Under adaptive attack nineteen months later: 99%. A human red-teaming competition with 500 participants defeated every defence tested.

On the RAG side the picture is similar. The independent benchmark tested seven defences against thirteen attacks. [11] Perplexity filtering and embedding-norm filtering performed within noise of no defence at all — norm filtering took one attack from 99% to 100%. Query paraphrasing made six of the thirteen attacks more successful, not less. PoisonedRAG’s own authors found that deduplicating the corpus by hash had precisely zero effect, because each malicious document is generated independently.

What does help:

And one thing that sounds like a defence and is not: OWASP notes that human-in-the-loop approval degrades in two specific ways — invisible-character smuggling can make the displayed action differ from the executed one, and approval fatigue degrades reviewer judgement at volume. A human clicking approve on their four-hundredth request is not a control.

What this means if you are building something

Not everyone reading this runs a RAG system. But a lot of people are about to, often without calling it that — every “chat with your documents” feature is one.

And the hardest one, also from the NCSC: “If the system’s security cannot tolerate the remaining risk, it may not be a good use case for LLMs.” [4] That sentence is unpopular and it is the honest position. Some things should not be built this way yet.

Sources & References