The chatbot answered the door for attackers

Meta confirmed that thousands of Instagram accounts were compromised through abuse of its AI assistant features. The vector was not a buffer overflow. It was not a memory corruption primitive. It was a trust boundary failure where the chatbot - wired into account support and identity-adjacent flows - accepted attacker-controlled natural language as if it were authenticated user intent. The bug class is prompt injection. CWE-77 in spirit, CWE-1039 in mechanism. No CVE has been assigned because Meta does not treat product features as a CVE-bearing surface. The exposure is operational. The mechanism is structural.

The failure mode is the confused deputy problem rendered in natural language. An LLM deployed in front of a user-facing support flow is, by construction, a parser that converts unstructured input into structured action. The parser has privilege the input does not. When the model is the only thing standing between an attacker’s text and a downstream call to an identity, recovery, or support API, the model becomes the authorisation boundary. Models are not authorisation boundaries. They are probabilistic functions that sample tokens conditional on context. Treating a sampling function as an access control decision is the root defect.

The exploit primitive is indirect prompt injection combined with tool-use abuse. The chatbot in Meta’s stack - like every production LLM assistant - operates with a system prompt, a tool schema, and a set of backend functions it can invoke. Account lookup. Recovery initiation. Session metadata retrieval. Identity verification escalation. The attacker does not need to read the system prompt. They need to push the model into a state where it calls a sensitive tool with attacker-chosen arguments while believing it is acting on behalf of the rightful account holder. The mechanism is well documented since Greshake’s work in 2023 and the Simon Willison taxonomy that followed. Hidden instructions inside attacker-controlled fields. Encoded directives that pass through input filters. Role confusion that makes the model treat injected content as system-level guidance. None of this is novel. All of it is reproducible.

The specific weakness in a social platform context is that the chatbot has access to context the attacker should not have. Profile metadata. Recovery contact hints. Linked device lists. Login geography. When a model is allowed to read this state and also allowed to act on it - for example, by initiating a recovery flow, sending a verification code to a chosen channel, or modifying a recovery email - the chain from injected instruction to account takeover is one model call long. The attacker writes a message. The model parses it. The model picks the tool. The tool executes with platform privilege. The account changes hands. There is no malware. There is no credential theft in the classical sense. There is a parser executing the wrong intent.

Maps to MITRE ATT&CK cleanly. T1566 for the initial social engineering surface where attackers reach the chatbot through profile messages, support entry points, or business inbox flows. T1078 for the valid accounts outcome - the attacker ends up operating as the victim, not bypassing authentication but acquiring it. T1539 where session material is the artefact extracted. T1556 where the authentication mechanism itself is modified through recovery flow manipulation. Sub-techniques under T1199, trusted relationship, apply when the chatbot is integrated with third-party identity providers that inherit the trust without re-verifying. The technique matrix already covers this. The platforms have not closed the gaps the matrix describes.

The second-order weakness is AI-generated content as a vector inside the platform. Meta has invested heavily in generative features that produce text, images, and synthetic personas across Instagram and Facebook surfaces. The output of these systems flows back into the same input channels that drive moderation, recommendation, and - critically - automated support triage. When a model produces content that another model consumes as authoritative context, you have model-to-model trust without a human in the loop. An attacker who can influence what the generative system produces - through manipulated training-adjacent inputs, prompt steering on public surfaces, or exploitation of personalisation features - can shape what the downstream model sees as ground truth. This is the inverse of the supply chain compromise model. The compromise is in the inference path, not the build path.

The identity verification implications are direct. Many platforms now use AI-driven liveness checks, document analysis, and behavioural signal scoring as part of account recovery. These systems are themselves models. They consume images, text, and metadata, and they output a trust score. An attacker who can produce synthetic media tuned against the verification model’s decision boundary can pass verification without holding the credential. This is not theoretical. Public research on liveness bypass against major IDV vendors has demonstrated the technique repeatedly through 2024 and 2025. When the same platform uses a generative model on one surface and a verification model on another, and both are tuned on overlapping data distributions, the attacker has gradient information they should not have.

What defenders see in telemetry for this class of compromise is the problem. Account takeover via chatbot abuse does not produce the signatures the SOC was built to catch. There is no anomalous login geography if the attacker uses the recovery flow to change the trusted device before authenticating. There is no credential stuffing pattern because no credential was guessed. There is no impossible-travel alert because the session is established cleanly after the recovery succeeds. The MFA event log shows a successful enrolment, not a bypass. From the platform’s perspective, a legitimate-looking support interaction concluded with a legitimate-looking account change. The chatbot logs show a conversation. The conversation is in natural language. The conversation does not parse as malicious to any keyword-based detection.

The detection gap is the conversation transcript itself. Most platforms do not run their own chatbot transcripts through prompt injection classifiers in production. The transcripts are stored, sometimes redacted, often retained only for short windows. The signals that would identify an injection attempt - anomalous token patterns, instruction-like phrases inside user input fields, base64 or homoglyph-encoded directives, role confusion markers - are extractable but not extracted at scale. The defenders looking for this need to instrument the LLM inference path. Log the full prompt as constructed. Log the tool calls the model selected. Log the arguments. Correlate tool-call sequences against the user’s prior behaviour. None of this is standard. All of it is required.

The network and EDR layers contribute nothing here. The attack happens entirely inside the platform’s API surface, between an authenticated session and an internal tool-call dispatcher. Sysmon has no visibility. The customer’s endpoint sees nothing because the customer is not involved. The attacker’s endpoint sees an HTTPS session to the platform, indistinguishable from any other user of the service. The detection has to live inside the platform. Outside-in monitoring will not catch this.

Real-world exploitation context is consistent with the actor profile that has run credential-harvesting campaigns against Meta surfaces for years. The same operators who ran phishing kits against Instagram business accounts in 2023 have migrated to chatbot abuse because the conversion rate is higher and the friction is lower. There is no need to host a phishing page. There is no need to evade Safe Browsing. The attack runs inside the platform’s own UI. Threat intel reporting through 2025 has tracked the shift. The disclosed compromise scale - thousands of accounts - is consistent with semi-automated abuse where the operator scripts the conversation flow and runs it across a target list.

The residual exposure after Meta’s response is the part worth tracking. A patch that adds keyword filters to the chatbot input does not close prompt injection. A patch that adds output filters to the chatbot response does not close tool-use abuse. The structural fix is to remove sensitive tool access from the model’s reachable function set, require deterministic human-or-MFA confirmation for any state change the model proposes, and treat every model decision as untrusted input to a separate authorisation layer that does not consume natural language. Until the architecture separates the parser from the privilege, the same class of compromise remains available. The next variant will not look exactly like this one. It will exploit the same trust boundary.

The technical reality is that LLM-fronted support is a new attack surface with old failure modes. Confused deputy. Input validation against a privileged action. Trust placed in a component that cannot enforce trust. The vulnerability is not in Instagram’s authentication stack. It is in the decision to let a probabilistic text generator make authorisation-adjacent calls without a deterministic gate behind it. Patch boundary on this kind of issue is fuzzy because the fix is architectural, not a version bump. Defenders auditing their own LLM deployments should assume the same primitive is present in their stack until proven otherwise. The platforms that ship these features faster than they instrument them will continue to produce incidents of this shape.