Confidential Computing Controls for LLM Inference in Enterprise Environments

Confidential computing gives enterprises a way to run LLM inference on shared cloud infrastructure without handing prompts, model weights, or outputs to the cloud provider in plaintext. This piece walks through how that protection actually works, where it has been built and deployed, what it costs, where it can still fail, and what a team needs to check before it trusts a vendor's claim.

Enterprises cannot send sensitive data to vendor LLM APIs under today's default infrastructure

When a prompt reaches an LLM, more than the text itself is at risk. All of this sits briefly unencrypted in GPU memory, on hardware the customer using it does not own or control.

TLS and disk encryption do not touch this problem. Neither one covers the moment a model actually runs a computation. The instant inference happens, the data is plaintext in memory, and anyone with access to a compromised host OS, hypervisor, or cloud operator account can read it.

This is why signing a contract does not fix the architecture. A signed contract does not change where the plaintext actually lives in memory during inference. That's an architectural fact, not a legal one.

The regulatory pressure keeps building on top of this. Once an LLM enters a data processing chain, you run into questions about lawful basis for processing, transparency around automated decisions, and how data crosses borders when inference happens outside the data subject's own jurisdiction.

None of this means enterprises can simply walk away from the cloud. Enterprises face a structural dependency on shared GPU infrastructure, so the fix has to live inside that infrastructure.

How a Trusted Execution Environment protects data during LLM inference

Confidential computing answers that architectural gap directly. It is a hardware-enforced security model that isolates prompts, model weights, tokenizer state, the KV-cache, and evaluator code from the host OS, the hypervisor, and the cloud provider, by running the AI workload inside a Trusted Execution Environment, or TEE.

The mechanism starts on the CPU side. Hardware firewalls sit around that region, and they physically block the host OS and cloud administrators from reaching in.

Encryption alone would not be enough, since a customer still needs proof that the environment doing the computing is the one it claims to be. That proof comes from cryptographic attestation. When a confidential VM boots, the hardware produces a signed quote that includes hashes of the kernel, the initrd, and the command line. That measurement identifies the exact software build running inside the TEE, and the host has no way to forge it.

The Confer model shows what this looks like as an actual exchange between a client and an inference endpoint. Only after all of that clears does any sensitive data get sent.

What falls inside that protected boundary matters as much as the mechanism itself. A properly built confidential inference system protects the model weights, the KV-cache, and any retrieval results pulled in through a RAG pipeline, not just the prompt and the response. The protection is meant to cover the full inference lifecycle, start to finish, inside the enclave.

Three deployed architectures for confidential inference in production

The theory is one thing. What organizations have actually built reveals a real split in how the root of trust gets established, and that split has direct consequences for how an enterprise can verify what it's being told.

Apple's Private Cloud Compute, known as PCC, runs as the inference backbone behind Apple Intelligence, with the inference service running inside an attestable environment under strict security policies. In June 2026, Apple announced an expanded version of PCC on Google Cloud, built together with Google and NVIDIA, aiming for a serving platform that meets Apple's own security, confidentiality, and transparency standards. The limitation sits in who controls the root of trust: PCC's security is rooted inside Apple's own chip, so Apple is both the operator and the party vouching for the system's trustworthiness. Because the system runs on proprietary software and hardware, you cannot adopt it on commodity infrastructure elsewhere.

Google's approach shows a different version of the same tension. Gboard now runs English and Japanese next-word prediction models on that system as of October 2026. But Google's Private AI Compute carries assurances that are hard to verify independently, because the implementation is closed-source and there is no binary available for outside inspection.

OpenPcc takes the opposite approach. It is the first open, end-to-end confidential LLM inference system built entirely on commodity CPU and GPU TEEs, specifically Intel TDX and NVIDIA H100. Its composite attestation design moves the root of trust away from a single vertically integrated vendor and puts it in the hands of parties the user can check directly. You can clone the repository, rebuild it, and confirm the measurements match what's claimed, so trust here does not depend on trusting one company's chip.

Anthropic's case rounds this out, and its value is in what it doesn't yet do. In June 2025, Anthropic published a whitepaper on confidential inference systems, working with Pattern Labs (now called Irregular) and building on RAND's SL4 and SL5 security levels. The paper lays out a high-level framework but stops there. There's no deployable system behind it yet. A whitepaper describes an architecture; it does not produce an attestation quote. What these systems, the ones that do run in production, actually cost to operate is the next question.

The real performance cost of TEE-protected inference

Security controls only matter if they're operationally workable, so the performance cost deserves a straight answer.

Benchmarking across Intel TDX and NVIDIA H100 confidential computing setups has found throughput penalties under 7 to 9% on GPU TEEs in some configurations, and that gap tends to shrink as model size and batch size grow. A separate engineering estimate puts confidential GPU instances at a 4 to 9% performance overhead compared to non-confidential equivalents when running inference servers like vLLM or TensorRT-LLM inside a TEE.

The spread between these numbers, roughly 4 to 8% in some studies and 10 to 25% in others, comes down to workload characteristics: batch size, model size, and whether the bottleneck is memory bandwidth or raw compute. If you're evaluating this for your own workload, run your own benchmark rather than lean on a number from someone else's paper.

Alternatives to TEEs exist on paper but aren't ready for production use at this scale. Fully Homomorphic Encryption (FHE) and secure multi-party computation (MPC) both score low to medium on deployability and don't come close to near-plaintext latency. THOR, a 2025 system from Moon and colleagues, runs GPU inference under homomorphic encryption but manages only about 0.2 tokens per second, several orders of magnitude below what any interactive production system needs.

The real engineering decision is not whether you adopt TEEs across the board. A well-designed confidential AI architecture isolates the genuinely sensitive components, patient notes, financial records, legal documents, inside the TEE, while running everything else on standard infrastructure. That selective approach is what keeps the performance cost proportional to the risk being managed.

Side-channel attacks and attestation weaknesses that TEEs do not automatically prevent

The real risk to confidential inference rarely comes from breaking the TEE's encryption directly. It comes from channels that operate entirely outside that encrypted boundary: side channels, attestation weaknesses, and flaws in how these systems get implemented.

The TDXRay attack, presented at IEEE S&P 2026, reconstructed user prompts from an encrypted Intel TDX virtual machine by watching cache-line-level memory access patterns during LLM tokenization and embedding lookups. No attestation check catches this category of attack, because the TEE's encryption was never actually broken.

A more direct threat to the trust model itself appeared in October 2025, when the TEE.Fail demonstration used a $1,000 DDR5 bus interposer to pull attestation keys out of Intel TDX, which made attestation forgery possible, and also bypassed ciphertext hiding on AMD SEV-SNP. Once attestation keys can be forged, a client checking an attestation quote has no reliable way to tell a real TEE from a spoofed one. That's the deepest kind of failure this trust model can have, because it undermines the one step everything else depends on.

Other vulnerabilities have targeted the guest environment directly. Separately, a 2026 result found that a common performance optimization, precomputing static secret bases to speed up cryptographic operations in partially-shielded TEE inference, introduces a key-reuse vulnerability. That optimization is exactly the kind of shortcut enterprises reach for to cut overhead, so the fix for one problem opened a different one.

Underneath all of these specific attacks sits a structural issue. A vendor's confidential inference claim has to be measured against what the customer can check independently, not against what the provider simply asserts.

Agentic workloads expand the confidential inference threat surface beyond what single-request TEEs cover

A single LLM request running inside a TEE is a meaningfully simpler security problem than an LLM agent that plans steps, calls tools, and coordinates a multi-step workflow. The agent case opens up attack surfaces that live between TEE boundaries, outside any one of them.

A 2026 survey of agentic AI systems identifies a threat surface that looks different from standalone inference.

A few specific points of exposure fall outside what TEE memory protection covers. Tool invocations can carry prompt content out past the TEE boundary to external APIs or retrieval systems. Plugin and connector integrations create another gap: a TEE-protected inference node wired up to an unvetted plugin leaks data right at that plugin interface, no matter how well the enclave itself is protected.

Secure RAG pipelines raise a specific version of the same issue.

The lesson for enterprise architecture is that confidential computing can't be bolted onto a single inference node and called done. It has to cover the entire data flow, model loading, the tokenizer, the KV-cache, the retrieval pipeline, evaluators, and trace exporters, as a system-wide property. A TEE at the core of the pipeline is necessary. For agentic workloads, it is not sufficient on its own.

Verifying a Vendor's Confidential Inference Claim

Everything above points toward a concrete question: what should a TPRM, Privacy, or Legal team actually ask before signing off on a vendor's confidential inference claim? The gap between marketing language and what's cryptographically enforced can be wide, and the attacks and limitations already covered show exactly where that gap tends to open up.

Start with attestation itself. A node labeled "confidential" in Kubernetes metadata is just a label, not a security guarantee, so the real question is whether every inference request carries an attestation receipt the customer can verify independently, and whether that verification runs against the hardware vendor's own attestation service, Intel Trust Authority for TDX or AMD KDS for SEV-SNP, rather than resting on the cloud provider's word alone.

Next, check who actually controls the root of trust. Google's Private AI Compute carries assurances that are hard to verify, because its implementation is closed-source and no inspectable binary exists. Apple's PCC roots its trust in Apple's own chip, so the operator and the verifier are the same party. The real benchmark here is reproducibility: can the customer clone the repository, rebuild it, and confirm the resulting measurements match the attestation. The Confer model, for instance, uses nix and mkosi to produce fully reproducible builds and publishes every release to a public transparency log.

Confirm the boundary covers the whole pipeline, not just the model. For any agentic deployment, confirm tool invocations and retrieval results sit inside that same protected boundary, not just the prompt and the response.

Finally, you need to tie regulatory evidence to technical proof, not just contractual language. Under HIPAA, a BAA is a legal instrument, necessary but not a technical control by itself, so attestation receipts that attach to audit-log entries give regulators the technical evidence that backs up what the BAA promises.

Continuous monitoring of attested inference versus one-time procurement assessment

A vendor that clears every item on that checklist at procurement time can fail the same evaluation six months later. Firmware updates, drift in TEE configuration, newly disclosed CVEs like StackWarp, TDXRay, and TEE.Fail, and changes somewhere in the attestation supply chain can all shift a vendor's security posture without ever triggering a new procurement review.

The hardware itself keeps moving underneath these systems. The accelerator landscape is fragmented: Google's TPUs don't yet support confidential computing, and NVIDIA's own confidential computing capabilities differ meaningfully across the H100, H200, and B200 generations. A vendor's infrastructure can change on its own timeline, with no obligation to flag the change to a customer.

Traditional governance tools were not built to catch changes like these. A SOC 2 report will not surface a tokenizer side-channel mitigation that quietly got removed in a firmware update. A standard TPRM questionnaire will not catch a vendor sliding from hardware-attested TDX to a software-emulated mode somewhere in its stack.

The checklist from the previous section asks the right questions. Answering them once, at the moment of signing a contract, only confirms the posture a vendor held on that particular day. Confidential inference, by its nature, sits inside a hardware and firmware landscape that keeps shifting, and an answer that was true in procurement is not guaranteed to stay true afterward.