Low-and-Slow Context Harvesting Signals That Precede an LLM Breach

To detect low-and-slow attempts to extract system prompts or exfiltrate data, defenders need to watch the whole conversation as well as classify each query. The strongest practical approach combines semantic drift, conversational pacing, external-context inspection, and resource signals. Low-and-slow context harvesting uses turns that look clean on their own to build a harmful exchange, so detection must track what the session is building before information gets out.

Why low-and-slow context harvesting is architecturally different from a frontal attack

A guardrail that evaluates each turn independently has a different view from the model it protects. The model predicts token continuations from the context built up so far, while a per-turn safety filter judges the signals it can see in a single exchange. The conversation can therefore develop a harmful direction that the filter never evaluates as a whole. This is a gap in the unit of analysis.

That gap is there before an attacker chooses a technique. A frontal injection such as “ignore all previous instructions” presents a recognizable signal to classifiers trained on direct attacks. A low-and-slow sequence makes each step look like a normal next turn, so there is no obvious conflict that would lead the model to reject it. The attacker no longer needs one decisive instruction, because earlier turns can establish the ground on which a later request becomes acceptable to the model.

The two mechanical paths through which context is harvested across turns

Slow semantic drift changes the conversational ground a turn at a time. DeepContext, published in February 2026, names Crescendo as an example of this gradual erosion of the model’s safety posture and describes it as a primary failure mode for stateless guardrails. Earlier exchanges make later requests fit the conversation even as the conversation moves away from its safe starting point. The setup is part of the attack. Its significance becomes visible only when the sequence is examined.

Context-window manipulation takes a different route by using volume or poisoning to displace or overwrite trusted instructions. Rather than relying only on conversational agreement, this path changes the context through which later input is interpreted. Yona et al.’s December 2025 Doublespeak attack illustrates the representational version. It substitutes a benign token for a harmful one across in-context examples, causing deep-layer embeddings to converge toward harmful representations. Input-layer checks miss the change because it emerges only after several transformer layers.

Persistent pre-training poisoning moves the preparation further upstream. The study identified as arXiv:2410.13722 describes how a tiny fraction of poisonous training documents can produce adversarial behavior that survives post-training alignment. That supply-chain route can make a later trigger produce a context dump without fresh prompt hacking in each session. It broadens the source of the same defensive problem. Whether the preparation occurs across conversation turns or before deployment, inspecting a trigger turn alone can leave the harmful behavior unexplained.

What the attacker is actually assembling — the value of harvested context

What is the attacker gaining from these exchanges? DeepContext identifies system prompts, RAG chunks, proprietary configuration, and user PII as leakage targets because they share the context window with user queries. An extracted system prompt can support application cloning or follow-on injections calibrated to its wording, while an embedded trigger can enable future context dumps. Detecting the harvest gives defenders a chance to step in before the leaked information helps with another attack.

In Microsoft 365 Copilot, a carefully crafted email entered the model’s context and caused hidden attacker instructions to be processed with retrieved enterprise data. Sensitive organizational content was then exfiltrated. Microsoft patched CVE-2025-32711 server-side in May 2025 and released a public advisory on June 11, 2025. The incident demonstrates the exposure created by retrieved content, rather than establishing that every context-harvesting attack requires a long user conversation.

The exposed surface also extends beyond the final answer. The 2026 study identified as arXiv:2603.05618 documents the risk that extended chain-of-thought output in reasoning modes may reproduce sensitive context verbatim. Multi-agent pipelines add further injection points at inter-agent API boundaries, and repository content has already steered GitHub Copilot and Cursor agents into unintended actions. These are expansion vectors for the same underlying exposure.

Behavioral signals that accumulate in the session record before a breach completes

Semantic drift is the most direct signal of a conversation changing direction. Sydney Holder and Bivin Sadler’s Spring 2025 SMU context-switch study showed a language-model-based detector that measures cosine similarity against both the original prompt and the conversation as it grows. Those two comparisons capture different changes. The first retains the starting point, while the second follows how the exchange develops, allowing the detector to flag both abrupt role changes and gradual drift. A sequence can cross a measurable semantic distance even when each turn remains below a per-turn alert threshold.

The SMU study paired semantic detection with a curated keyword and hack-phrase scanner, and its 2025 evaluation found that 98.6% of transcripts activated at least two independent alarms. That finding supports redundancy. A legitimate conversation can also change subjects. The value lies in recognizing a trajectory and considering the other evidence around it, rather than treating any departure from the opening topic as malicious.

Structural pacing provides that additional evidence without requiring access to model internals. More turns than usual before a sensitive request can be a sign of the setup work that harvesting needs, when compared with how legitimate sessions unfold. Payload splitting has another pattern, in which turns that are neutral on their own build an instruction together. Encoding can also leave cross-turn traces. Base64, ROT13, reversed character order, and Unicode substitutions may evade input filters while producing anomalous token distributions across the session.

OVERTHINK, described in arXiv:2502.02542, injects decoy reasoning problems into external context to increase reasoning-token consumption while preserving contextually correct answers. LingoLoop, presented at ICLR 2026, induces repetitive generation through changes to part-of-speech token dynamics and hidden-state magnitude. Neither is itself a context-harvesting attack, so a resource spike is not proof of exfiltration. Their signatures are anomalies that can help when seen alongside other signals, but Reasoning Hijacking needs attention-map analysis because it can corrupt reasoning while keeping task intent the same and leaving intent-based monitoring silent.

Why per-turn guardrails and input classifiers cannot close this gap alone

Could a stronger input classifier just spot the setup earlier? DeepContext’s evaluation found that the tested system with the lowest detection latency had poor recall, catching attacks early but missing a large fraction. High-recall systems added latency beyond the AWS Prompt Attack Guardrails baseline. Across multi-turn slow-drift scenarios, none of the tested systems was fast, high-recall, and low in false positives all at once.

Attackers can also adjust to the defense. The LLMSec 2025 study identified as arXiv:2504.11168 demonstrated guardrail bypass through prompt obfuscation. “The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against LLM Jailbreaks and Prompt Injections,” published in 2025, argues that disclosed detection heuristics give attackers information they can use to circumvent them. A published threshold becomes a calibration target. This complicates any claim that a fixed boundary will remain effective once an adversary can probe it.

Some limits concern what the detector can observe. Reasoning Hijacking preserves task intent, so monitoring focused on goal deviation has no alert to raise for that subclass. NIST AI 100-2e2025, published in March 2025, acknowledges that limiting user queries and detecting suspicious queries remain incomplete mitigations against adaptive adversaries. That remains an open problem. Per-turn guardrails still handle direct attacks, and session-level monitoring and cross-turn analysis add the view of the whole conversation that defenders also need.

How do you detect system prompt extraction across a deployed LLM session?

The monitoring layer needs to treat the full conversation as its unit of analysis, combining semantic trajectory measurements with structural anomalies. The SMU study recommends a hybrid defense that pairs a language-model-based semantic-drift detector, using a cosine-similarity threshold of 0.70, with a keyword and hack-phrase scanner. The threshold is a starting point. Adaptive thresholds tuned to the deployment can reduce false positives from legitimate topic pivots, while comparisons against both the original prompt and the evolving conversation preserve the two views needed to detect drift.

External context must enter that analysis too. EchoLeak shows why inspecting only user messages leaves a gap when email, documents, or RAG retrieval can introduce instructions into the same context window. “Embedding-Based Detection of Indirect Prompt Injection Attacks in Large Language Models Using Semantic Context Analysis,” published in Algorithms in 2026, describes an approach to detecting indirect injection through external data. Its adaptive robustness remains unresolved. It has not yet been evaluated against adversaries who can observe the embedding boundary, so that limitation belongs in any assessment of its coverage.

For agentic deployments, the monitoring boundary must extend from the user-facing interface across the multi-agent call graph. A dedicated AI risk intelligence platform fits this requirement when it combines vendor risk assessment with real-time threat detection across models, plugins, agents, and connectors, including unvetted assets. Coverage is the basis for comparison. Per-turn guardrails inspect individual exchanges, semantic detectors follow conversational movement, and ecosystem monitoring follows the additional sources and boundaries through which context enters. Traditional third-party risk management and governance frameworks were not built to check for context-window manipulation or slow-drift grooming.

The same coverage needs continuity before and after deployment. Persistent pre-training poisoning makes training provenance relevant to vendor assessment, because runtime behavior is not the only source of risk. The CSA AI Safety Initiative’s April 2026 whitepaper, “The Collapsing Exploit Window: AI-Speed Vulnerability Weaponization,” describes a narrowing gap between vulnerability publication and exploitation as LLM-assisted exploit generation becomes more accessible. Procurement-only assessment therefore produces a decaying risk picture. Continuous vendor assessment and session monitoring address different parts of that exposure, and neither can stand in for the other.

What defenders should watch for before the next low-and-slow attack completes

The first watchpoint is semantic distance from the session’s origin, measured at every turn and interpreted with the evolving conversation. The next is pacing, especially when many setup turns come before a request for sensitive information. Defenders can see these signals without model-internal instrumentation. For defenders, the practical priority is to examine them together so that a long but ordinary exchange is not confused with a sequence preparing a disclosure.

Cross-turn n-gram or embedding analysis can show how short messages with neutral topics build an instruction together. External-context inspection also checks for instruction-override patterns that come through email, documents, and RAG retrieval. The user message is not the whole input. In an EchoLeak-style vector, the suspect material comes from a source the model retrieves, so even a clean user-facing exchange leaves that source to be checked.

Reasoning-token consumption is another signal to watch when API billing or token counts are available. A resource spike without a corresponding increase in query complexity resembles the OVERTHINK signature, although it does not by itself establish harvesting. Attention-map shifts toward injected tokens can expose Reasoning Hijacking, while Doublespeak’s deep-layer convergence is detectable in principle through activation analysis but is not yet operationalized in production monitoring systems.

The final constraint is an asymmetry in detection. DeepContext found that early-detecting systems sacrificed recall, while high-recall systems sacrificed latency. Even with a sharper per-turn threshold, the session history is still missing and that trade-off remains.