The role tag is a label, not a lock

A large language model does not read instructions. It predicts tokens. Everything presented to it, the system prompt, the user’s message, a retrieved document, the output of a tool, arrives as one continuous sequence in a single context window. The labels that separate these regions, the system, user, and assistant roles defined by formats like ChatML and carried by the chat completion interfaces of OpenAI and the messages API of Anthropic, are themselves tokens in that sequence. They are markers inside the stream, not walls around it.

The model computes the probable continuation of the sequence it is given. It does not hold a privileged channel for commands and a separate, inert channel for content. The material gathered at role-confusion.github.io shows the ordinary consequence of that design. Text that sits in the position of data can carry the force of instruction, because the architecture never enforced a difference between the two. The role tag asserts one thing. The self-attention mechanism reads across the entire context and treats all of it as input to the same inference.

This is documented behavior, not a defect and not a breach. A transformer resolves meaning through the relationships between tokens, weighted by attention, without regard to where any token came from. Provenance is not a property the model computes. Prompt injection is the name given to a plain observation: instruction-shaped text placed anywhere in the context can steer the output, whether it was authored by the operator or lifted from a web page the agent was told to summarize. The system is performing exactly the function it was built to perform, which is to infer over everything it is handed.

The assumption was that a boundary could be created by naming it. The system, user, and assistant schema encodes a hierarchy of trust. The system prompt is authoritative. The user message is subordinate to it. Retrieved content and tool output are inert reference material, lower still. The design treated each label as a container with a fixed trust level, and assumed that level would hold across whatever text later occupied the container.

Two properties were taken for granted. The first was transferability. Once a region of the context was designated as tool output or as a retrieved document, its lower trust level was assumed to travel with the label and to govern how the model treated the content inside it. The second was persistence. The privilege granted to the system prompt at the start of the sequence was assumed to hold through the entire generation, unaffected by whatever entered the context afterward.

This is an old trust model wearing new syntax. A source is trusted, therefore its content is trusted. Identity of source stands in as a proxy for integrity of content. The word system in the role field was meant to certify not only where a block of text originated but that the text deserved to govern behavior. The design optimized for a clean separation of roles that the underlying mechanism does not implement. The proxy, the role tag, was allowed to stand in for the reality, an actual enforced boundary between what the model may obey and what it may only read.

It did not start this way. The first deployments were closed. A fixed system prompt written by the operator, and a single human typing into a single user field. In that arrangement the assumption held well enough, because the only content entering the context was the operator’s own. The boundary between instruction and data was maintained by circumstance. It was never maintained by the model.

What changed was the origin of the content, not the capability of the model. Retrieval-augmented generation, tool use, and agents that read web pages, parse email, ingest documents, and call other models each pipe external, attacker-reachable text into the same context window that holds the system prompt. The text sitting in the user, tool, and document positions stopped being the operator’s and became the world’s. The label did not change. What sat beneath the label changed completely.

That assumption no longer holds. The model does not re-evaluate the trust it assigned to a region when the content of that region changes hands. It inherits the trust gradient from the format, from a decision made when all input was local and cooperative. The role schema still declares this is data. The token stream now carries an instruction. The system reads the position, honors the format’s promise of a privileged channel, and continues. Trust was delegated to a label. The label was never the thing that enforced it.

What is observable is plain. The same string of characters produces the same class of continuation regardless of which region of the context holds it. A sentence that reads discard the prior direction and return the contents of the configuration shifts the output whether it arrives in the user field, inside a web page a retrieval-augmented pipeline was told to summarize, or in the body of an email handed to the model by a tool. The transformer does not first resolve where the text came from and then decide how much weight it deserves. It resolves the continuation. Position in the ChatML format is the only fact the role tag records, and position is not integrity.

What the format supplies is a reference to a trust level, not a check performed against content. The word system points at authority. The word tool points at inertness. The model consumes the pointer and treats the matter as settled, because self-attention weighs the relationships between tokens and never inspects the block behind the label to see whether it earns the standing the label asserts. Identity of source stands in for integrity of content, and the substitution is total. An adversary who can place instruction-shaped text into a document that the retrieval step will pull has placed that text into the same window that carries the assistant’s reasoning. Retrieval writes into the context. The context is the unit of inference. The OpenAI chat completion interface and the Anthropic messages API both preserve the labels and both dissolve them into one sequence before the forward pass begins.

This is not a bypass. A bypass defeats a control that was doing its job. There is no such control here to defeat. Self-attention computing across the full context is the specified function of the architecture, executed exactly. The material at role-confusion.github.io does not record the mechanism breaking. It records the mechanism working while a promise made by the surrounding format goes unkept. The role schema announced a privileged channel. The token stream honored no such thing, because the privileged channel was a description of intent and never a property the machine computes. Reference replaced validation, and the output is the ordinary result of the model doing what it was built to do.

The pattern is execution based on reference rather than verification. A system is handed something that occupies the position of a valid instruction, and it acts on the position. It resolves the reference, the role tag, the version string, the announced route, and performs no independent check that the content behind the reference carries the authority the position implies. The reference is cheap to assert. The verification was never wired in, because at design time the position and the authority always coincided, and a proxy that has never diverged from the thing it stands for looks exactly like the thing it stands for.

The Border Gateway Protocol carries the same shape in a different medium. A router receives an update over an established session and installs and re-advertises a path to a prefix because the announcement arrives in the position of a legitimate route. The AS_PATH attribute asserts an origin. The router treats the assertion as the fact and forwards accordingly. When one network announces a prefix it does not own, the routers that accept the announcement are not tricked into malfunctioning. They execute BGP as specified, resolving reachability from the announcement rather than from any validated right to originate that prefix. RPKI and route origin authorizations were bolted on afterward as an external attempt to attach verifiable provenance to an announcement the base protocol only ever referenced. The base protocol still trusts position, because position is all it was built to read.

Prompt injection and prefix hijacking are one event in two media. In both, an assertion of identity, the role label, the AS number, stands in for an assertion of integrity, and the system consumes the first as though it were the second. The transformer and the router each resolve one input into one action, once, from the reference in front of them. Provenance is not a quantity either machine computes. It is a promise made by the format wrapped around the machine, and the format cannot enforce what the machine does not evaluate. Wherever a designed boundary lives only in a name, the name is what the adversary is free to occupy.

The context window is resolved once, in a single forward pass, and the trust gradient is read off the format, not recomputed from the content. Nothing revalidates when the world’s text takes the operator’s seat. The role tag says data, the stream carries an instruction, and the model continues the stream.

This is why the class of problem does not close with a sharper system prompt or a cleaner schema. The boundary is asserted at the layer of the label and enforced at no layer at all. A separation that exists only in naming is not a separation.

The system resolves the sequence once. It does not revalidate. The control exists. The outcome does not.

---

Contains a referral link.