Confidential Containers on Kubernetes: A 2026 Guide

Confidential Containers on Kubernetes: A 2026 Guide

Confidential Containers on Kubernetes in 2026: CoCo Architecture, Attestation, GPUs and Cloud Options

Encryption at rest protects disks and TLS protects the wire, but the moment a pod loads plaintext into RAM, the host kernel, the hypervisor and anyone with root on the node can read it. Confidential containers kubernetes deployments close that gap by running each pod inside a hardware-encrypted confidential virtual machine and releasing secrets only after the hardware proves what booted. The topic changed a lot in 2026. The CNCF Confidential Containers project (CoCo) reached incubation in July. NVIDIA shipped a generally available confidential GPU stack for Kubernetes in April. AWS launched Nitro Sandboxes with a per-pod confidential runtime class in November. Microsoft, meanwhile, announced the retirement of its original AKS preview. Most guides written before this year are now wrong on install method, runtime class names and cloud availability.

What this covers: the threat model, the pod-in-a-confidential-VM architecture, SEV-SNP and TDX evidence, the Trustee attestation and key-release protocol, init-data and agent policy, encrypted images, confidential GPUs, peer pods, a 2026 cloud decision matrix, capacity math, failure modes and a rollout checklist.

Context and Background

For two decades, defense in depth had a hole in the middle. Data in use, the plaintext in memory while the CPU works on it, was simply trusted to the platform underneath, including a hypervisor, host kernel and control plane you do not own.

Confidential computing changes the assumption. A Trusted Execution Environment (TEE) is hardware that encrypts a workload’s memory with keys the host software never sees. It also produces signed evidence of what was loaded at launch, so a remote party can verify the environment before trusting it. AMD SEV-SNP and Intel TDX do this for entire virtual machines, which is what makes them usable for unmodified containers. IBM Secure Execution does the same on s390x mainframes.

The Confidential Containers project started in 2021 as a collaboration between Red Hat, Intel, IBM and others. According to its CNCF project page, it was accepted into the CNCF on 8 March 2022 and moved to incubating on 8 July 2026. The CNCF announcement cites more than 150 active contributors, over 1,200 merged pull requests across 26 repository groups, and backing from Microsoft Azure, Intel, AMD, IBM, NVIDIA, Alibaba and Red Hat. CoCo does not invent a new runtime. It builds on Kata Containers, which already runs each pod in a lightweight VM, and adds attestation, guest-side image handling and secret delivery.

If the hardware layer is new to you, start with our comparison of Intel TDX, AMD SEV-SNP and Arm CCA confidential VMs and their remote attestation architectures. This post sits one layer up, where those VMs meet the Kubernetes scheduler.

Adoption is driven by regulated industries keeping records from the operator, AI teams protecting model weights and prompts on rented GPUs, and SaaS vendors answering “can your staff read our data?”

Confidential computing does not fix application bugs, protect data your code sends out, stop a supply-chain attacker who poisons your image before signing, or guarantee availability. The guarantee is narrow and precise, which is why it is useful.

How Confidential Containers on Kubernetes Work: The Pod-in-a-Confidential-VM Architecture

Confidential containers on Kubernetes run each pod inside its own hardware-encrypted VM on a TEE-capable node. Kata Containers launches the VM; AMD SEV-SNP or Intel TDX encrypts its memory. Inside, an attestation agent proves the boot state to a Trustee Key Broker Service, which releases image keys and secrets only after verification succeeds.

Confidential containers kubernetes architecture with kubelet, containerd, Kata shim and an encrypted confidential VM holding the Kata agent, attestation agent and workload
Figure 1: The CoCo node architecture. Everything inside the encrypted guest is protected from the host; Trustee runs in a separately trusted domain.

Figure 1 traces a pod from the API server to the encrypted guest. The kubelet and containerd run on the untrusted host and call the Kata shim selected by the pod’s runtime class. The shim asks QEMU to launch a confidential VM. Inside the encrypted guest sit the Kata agent, which manages containers on the host’s behalf, the Attestation Agent, the Confidential Data Hub, and the image-pull machinery. The workload containers start only after these components have attested and fetched what they need from the Trustee Key Broker Service.

The point is to remove the host from the trusted computing base

The trusted computing base (TCB) is everything that must behave correctly for your security claim to hold. In an ordinary pod, the TCB includes the host kernel, the hypervisor, the container runtime, the node’s root users and the cloud provider’s management plane. That is a large surface owned by other people.

CoCo shrinks it to the CPU silicon and firmware, the guest firmware and kernel, the in-guest agents and your own image. The host still orchestrates the VM, but it cannot read the guest’s memory or silently alter its contents. Every other design decision follows from this one. The image is pulled inside the guest so the host never sees it unpacked. Secrets arrive over an attested channel so they never pass through Kubernetes Secret objects. The Kata agent enforces a policy so the host cannot use the management channel as a backdoor.

The hardware: SEV-SNP, TDX and Secure Execution

AMD SEV-SNP (Secure Encrypted Virtualization with Secure Nested Paging) encrypts each guest with a per-VM key held by the AMD Secure Processor. Its key addition over earlier SEV generations is integrity. A Reverse Map Table tracks which guest owns each physical page, so the hypervisor cannot remap, replay or alias guest memory. Attestation reports are signed by a chip-unique key (the VCEK) that chains to AMD’s root certificates. Each report carries the launch measurement, the reported TCB version of the firmware and microcode, and a 32-byte HOSTDATA field that the host sets at launch.

Intel TDX (Trust Domain Extensions) runs each guest as a trust domain. A small Intel-signed component called the TDX Module mediates every transition between host and guest, and memory is encrypted and integrity-checked per domain. Evidence is a quote containing MRTD, the build-time measurement of the initial guest image, plus four runtime measurement registers (RTMRs) that firmware and the guest kernel extend as they boot. A field called MR_CONFIG_ID plays the same role as SNP’s HOSTDATA.

IBM Secure Execution for Linux protects guests on IBM Z and LinuxONE. CoCo v0.23.0 lists attestation as supported and tested on all three: TDX, SEV-SNP and Secure Execution.

Unlike Intel SGX, which protected a slice of a process and required an enclave SDK, SNP and TDX protect a whole VM, so unmodified images run inside. CoCo’s process-based runtime, enclave-cc, is now archived.

Runtime classes and installation in 2026

Many older tutorials, including the earlier version of this article, describe installing CoCo with an operator and selecting a runtime class called kata-cc. Both details are out of date. The confidential-containers/operator repository is archived, and the project documentation now installs CoCo with a Helm chart published to GHCR:

helm install coco oci://ghcr.io/confidential-containers/charts/confidential-containers \
  --namespace coco-system \
  --create-namespace

The chart installs Kata binaries on the nodes and registers runtime classes. On x86_64 the CoCo installation docs list kata-qemu-snp, kata-qemu-tdx, kata-qemu-nvidia-gpu-snp and kata-qemu-nvidia-gpu-tdx for real hardware, plus kata-qemu-coco-dev for testing without a TEE. On s390x you get kata-qemu-se. Peer pods, covered later, use kata-remote.

The coco-dev class exercises the full software path with no hardware protection. It is excellent for CI, but it is not a security boundary, so block it from production namespaces with an admission policy.

The current release at the time of writing is CoCo v0.23.0, published 24 September 2026. It is based on Kata Containers 4.2.0 and Trustee v0.22.0, and its guest and Trustee components speak KBS protocol v0.4.0. Kata 4.0, released in July 2026, made the Rust runtime (runtime-rs) the default and deprecated the Go runtime. The CoCo v0.23.0 notes state that CoCo still uses the Go shim and will move later. If you read a guide that mixes those two facts up, trust the release notes. The project ships roughly every eight weeks, so check the version again before you deploy.

Why the Kata agent policy matters as much as encryption

In standard Kata, the host talks to the in-guest Kata agent over a vsock channel. It can create containers, mount volumes, stream logs and run exec. If the host is untrusted, that channel is a backdoor: a malicious host could ask the agent to exec a shell inside your container and print your secrets, and memory encryption would never notice.

CoCo closes this with an agent policy written in Rego, the Open Policy Agent language. The agent evaluates every request from the host against the policy and refuses anything not allowed. A production policy typically pins the exact container images, commands, environment variables and mounts for the pod, and denies ExecProcessRequest and log streaming. Tools such as Kata’s genpolicy can generate a starting policy from a pod manifest, which is much safer than writing Rego by hand.

How the guest knows the policy was not swapped by the host is the job of init-data, explained next.

Attestation, Init-Data and Key Release with Trustee

Attestation turns isolation into trust. Encrypted memory alone says nothing about whether the guest runs your firmware, kernel and policy or something the host swapped in.

Confidential containers kubernetes attestation sequence between the attestation agent, Trustee KBS, Attestation Service and RVPS
Figure 2: The KBS request, challenge, attestation and response exchange, followed by a resource request that is gated by policy.

Figure 2 shows the exchange. The Attestation Agent opens a session with the Key Broker Service and receives a fresh nonce. It asks the hardware for a report that binds that nonce and an ephemeral public key generated inside the guest. The KBS forwards the evidence to the Attestation Service, which checks signatures and compares measurements against reference values supplied by the Reference Value Provider Service. If the evidence passes, the KBS issues a token. When the guest later requests a resource, the KBS evaluates its resource policy against the token’s claims and returns the secret encrypted to the guest’s ephemeral key.

The Trustee components and how they map to RATS

Trustee is the CoCo project’s attestation and secret-delivery stack, and it maps closely onto the IETF RATS architecture (RFC 9334). The Trustee repository describes four pieces. The Key Broker Service (KBS) is the relying party and the only endpoint guests talk to. The Attestation Service (AS) is the verifier. The Reference Value Provider Service (RVPS) stores the known-good measurements the verifier compares against. A KBS client tool configures policies and uploads resources.

Inside the guest there are two matching components. The Attestation Agent (AA) gathers hardware evidence and runs the protocol. The Confidential Data Hub (CDH) is what workloads and the image-pull code call when they need a secret or a decryption key. Resources are addressed with a three-part path of the form repository, type and tag, so a database password might live at prod/db/password.

Trustee must run somewhere you trust more than the nodes it attests. That might be a separate cluster in your own data centre, a confidential VM you control, or a managed key service. Putting the KBS on the same untrusted cluster as the workloads defeats the purpose, because the host that you are trying to exclude could then tamper with the verifier. The project recommends the Helm chart for Kubernetes deployments, and v0.23.0 adds Redis or Valkey as a session store for running several KBS replicas.

Evidence, reference values and the TCB floor

The verifier does three separate checks, each failing differently.

First, the signature chain. An SNP report must chain from the chip’s VCEK through AMD’s intermediate to AMD’s root. A TDX quote must chain through Intel’s provisioning certificates. This proves the evidence came from genuine silicon, but nothing more.

Second, the TCB version. Every report carries the security version numbers of the firmware and microcode that produced it. When AMD or Intel patch a vulnerability, those numbers increase. If your policy does not enforce a minimum, Trustee will release secrets to a node running microcode with a known flaw. Treat the minimum as a moving floor that you raise when an advisory ships, exactly as you track CVEs in base images.

Third, the measurements. The launch measurement on SNP, and MRTD plus RTMRs on TDX, are hashes over the guest firmware, kernel, command line and initial ramdisk. If you do not load the expected values into RVPS, attestation proves only that the hardware is real, not that your software booted. Reference-value management is therefore not housekeeping. It is the core of the security argument, and it needs the same release discipline as your container images.

Init-data: binding policy and configuration to the hardware

Init-data solves the substitution problem from the previous section. It is a small TOML document that holds the Kata agent policy (policy.rego), the Attestation Agent configuration (aa.toml) and the CDH configuration (cdh.toml), including the KBS URL. You attach it to the pod as a gzip-compressed, base64-encoded annotation:

apiVersion: v1
kind: Pod
metadata:
  name: claims-scorer
  annotations:
    io.katacontainers.config.hypervisor.cc_init_data: "<output of: cat initdata.toml | gzip | base64 -w0>"
spec:
  runtimeClassName: kata-qemu-snp
  containers:
  - name: scorer
    image: registry.example.com/claims/scorer@sha256:<digest>
    resources:
      limits:
        cpu: "2"
        memory: 4Gi
version = "0.1.0"
algorithm = "sha384"

[data]
"aa.toml" = '''
[token_configs.kbs]
url = "https://kbs.trust.example.com:8080"
'''

"cdh.toml" = '''
[kbc]
name = "cc_kbc"
url = "https://kbs.trust.example.com:8080"
'''

"policy.rego" = '''
# generated by genpolicy, then reviewed
'''

The binding is the clever part. According to the CoCo init-data documentation, the runtime hashes the init-data and writes the digest into a hardware register at launch: HOSTDATA on SNP and MR_CONFIG_ID on TDX. The Attestation Agent checks that the init-data it received matches that digest and refuses to start the pod otherwise. The Attestation Service then includes the init-data claims in the token, so a KBS resource policy can require a specific policy hash before releasing anything.

If the host swaps in a permissive policy, the digest in the evidence changes and Trustee withholds the secrets. The host can start a pod with a bad policy, but it gets a pod without keys.

Two policies, two questions

Trustee has two policy layers, and conflating them is a common design mistake. The attestation policy in the Attestation Service decides whether the evidence itself is acceptable: right TEE type, TCB above the floor, measurements match. The resource policy in the KBS decides whether an acceptable environment may read a particular secret.

Keep them separate: attestation rules change with hardware and guest images, resource rules with teams and secrets, so a platform team can own one and application teams the other. By default, Trustee denies resources to any client that has not presented valid TEE evidence, which is the right default.

A confidential pod boots useless, by design

When the VM launches, the hardware records the launch measurement and init-data digest, but the guest holds no keys and, if the image is encrypted, no usable image. Only after the exchange in Figure 2 does it pull and decrypt the image, unlock storage and fetch secrets. A normal pod has everything the instant it starts; a confidential pod earns its capabilities through attestation.

This is why a slow or unreachable KBS shows up as a pod stuck in ContainerCreating, not a crash. Surfacing attestation state as a metric and an alert is one of the most valuable pieces of operational glue you can build.

Encrypted Images, Sealed Secrets, Confidential GPUs and Peer Pods

The memory boundary is only half the story. The other half is everything that has to cross into the guest: the image, the secrets, and in 2026 increasingly a GPU.

Guest pull, signed images and encrypted layers

In a normal Kata pod, containerd on the host pulls and unpacks the image, so the host sees every binary and config file. CoCo moves the pull inside the guest. The confidential VM fetches the layers itself, and the host sees only the bytes in transit.

Confidential containers kubernetes encrypted image pipeline from CI signing and ocicrypt layer encryption to in-guest decryption after attestation
Figure 3: Image protection end to end. The registry holds ciphertext, the layer key lives in Trustee, and decryption happens only inside an attested guest.

Figure 3 shows how the pieces compose. Your CI pipeline signs the image, for example with cosign, and can also encrypt the layers with ocicrypt, the OCI layer-encryption scheme used by containerd and skopeo. The layer key is uploaded to Trustee rather than distributed to nodes. At runtime the guest pulls ciphertext layers, the CDH requests the key after attestation, and the guest verifies the signature against a policy before starting the container.

Signing answers “is this our image?” while encryption answers “can the host see inside it?” Regulated workloads usually need both. If you already run a signing pipeline, our guide to Trivy, Grype and cosign for image scanning and signing covers the CI half. CoCo adds the requirement that verification happens inside the guest rather than in an admission controller on the host.

There is one sharp edge here. The v0.23.0 release notes state plainly that credentials for authenticated registries are exposed to the host. If the registry password itself is sensitive, encrypt the layers so that a host who steals the credential gains only ciphertext.

Sealed secrets instead of Kubernetes Secrets

Kubernetes Secret objects are readable by the control plane and anyone with the right RBAC. CoCo offers sealed secrets: the Secret object holds only a reference, and the CDH resolves it to the real value from the KBS after attestation, inside encrypted memory. Cluster administrators and CI systems can list Secret objects all day and see only those references.

The same path delivers keys for encrypted storage. A LUKS-encrypted volume can be unlocked inside the guest with a key from Trustee. The v0.23.0 notes add a caveat worth repeating: this LUKS integration provides integrity but not replay protection, so a host could roll a volume back to an older valid state.

Confidential GPUs for AI inference

The biggest change in 2026 is GPU support. NVIDIA introduced confidential computing mode with Hopper, where the GPU establishes an encrypted session with the confidential VM and data crossing PCIe is encrypted. The GPU produces its own attestation report, so the verifier must check composite evidence: the CPU TEE and the GPU together.

NVIDIA’s Confidential Containers reference architecture release notes give the current state. Early access shipped with GPU Operator v25.10 in December 2025. Version 1.0.0 became the first GA release on 29 April 2026, supporting single-GPU passthrough on H100, H200, B200 and RTX Pro 6000 Blackwell Server Edition, and multi-GPU passthrough on H100 and H200 Protected PCIe systems and B200. Version 1.1.0, released 11 August 2026, added the HGX B300 with single and multi-GPU passthrough, moved to Kata Containers 4.0.0 and Trustee v0.21.0, and added Ubuntu 26.04 as a host OS. Tested CPU platforms include AMD Genoa and Milan for SEV-SNP and Intel Emerald Rapids and Granite Rapids for TDX.

The constraints are specific. Only containerd is supported as the container runtime. Every GPU on a host must be configured for confidential computing, and for multi-GPU passthrough, all GPUs on the host must go to a single confidential VM. That rules out the sharing models most inference platforms rely on today, which our post on GPU sharing with MIG, MPS, time-slicing and vGPU compares. In a confidential deployment, plan on whole-GPU or whole-node allocation.

CoCo’s guest components now include an NVIDIA attester in all x86 builds. Red Hat’s OpenShift sandboxed containers 1.13, announced in August 2026, made confidential containers with confidential GPUs on bare metal generally available and integrated GPU attestation through Trustee with NVIDIA’s Remote Attestation Service. Red Hat lists confidential GPU support on Azure as a Technology Preview, not GA.

Peer pods: confidential containers without bare metal

Cloud worker nodes are usually VMs without nested confidential virtualization. The cloud-api-adaptor project solves this with peer pods. It implements Kata’s remote hypervisor interface: instead of launching a local VM, the shim asks the cloud provider’s API to create a separate confidential VM for the pod, and the agent connects back to the worker node over a tunnel.

Peer pods use the kata-remote runtime class. The trade-off is latency and cost. Pod start now includes cloud VM provisioning, which typically takes tens of seconds rather than a few seconds, and each pod is billed as its own instance. In exchange you get confidential pods on standard managed Kubernetes worker nodes. Red Hat documents peer pods for Azure, AWS and Google Cloud in its OpenShift sandboxed containers guides.

The 2026 Cloud Landscape: What Each Provider Actually Offers

The table below summarises what is verifiable as of 16 November 2026. “Per-pod” means each pod has its own TEE boundary and the node administrator is outside the TCB. “Per-node” means the whole node is a confidential VM, which protects against the cloud operator but not against your own cluster administrators.

Option Boundary TEE Status as of Nov 2026 Who is excluded from TCB
Upstream CoCo on bare metal Per pod SEV-SNP, TDX, Secure Execution Open source, CNCF incubating, v0.23.0 Host OS, hypervisor, node admins
OpenShift sandboxed containers 1.13 Per pod SEV-SNP, TDX, IBM SE, plus GPUs GA on bare metal incl. confidential GPU; Azure GPU in tech preview Host, node admins
AWS Nitro Sandboxes, confidential mode Per pod or task Nitro Hypervisor and Nitro Security Module Launched 10 Nov 2026 Cluster admins, platform team, AWS per AWS
Azure AKS Confidential Containers preview Per pod SEV-SNP via Kata Retirement announced; KataCcIsolation pools supported until 2 Mar 2026 Host, node admins
Azure AKS confidential VM node pools Per node SEV-SNP GA with Azure Linux Azure operator, not cluster admins
Azure Confidential Container Instances Per container group SEV-SNP Available; serverless, not Kubernetes Azure operator
GKE Confidential GKE Nodes Per node SEV, SEV-SNP, TDX Available; SNP and TDX on Autopilot need 1.35.2-gke.1485000 or later Google operator, not cluster admins
Peer pods via cloud-api-adaptor Per pod Cloud confidential VMs Open source; productised by Red Hat Host worker node, node admins

Confidential containers kubernetes decision tree for choosing between confidential nodes, bare-metal CoCo, AWS Nitro Sandboxes, Azure options and peer pods
Figure 4: Choosing a deployment model. The first question is whether your own cluster administrators must be outside the trust boundary.

Figure 4 encodes the decision. If you only need protection from the cloud operator, confidential nodes on GKE or AKS are the simplest answer, because ordinary pods run on them unchanged. If your own administrators must be excluded, you need a per-pod boundary, and the choice depends on whether you control bare-metal TEE hosts, which cloud you use and whether you need GPUs. For managed-cluster differences beyond confidential computing, see our EKS vs GKE vs AKS comparison for 2026.

Azure: the preview is ending, the docs lag

Microsoft’s message centre notice AZ502044, published 16 October 2025, announced that the Confidential Containers preview on AKS is being retired. Node pools using the KataCcIsolation runtime were to keep running until 2 March 2026, after which the feature is unsupported and new pools cannot be created. Microsoft said it is focusing on “supported and production-ready” confidential offerings but did not name a direct replacement.

Confusingly, the AKS Confidential Containers overview was still published as a preview page with a September 2026 revision date when we checked. Treat the retirement notice as authoritative and do not start new projects on that preview. On Azure today, the realistic options are confidential VM node pools, which are GA with Azure Linux and protect against the operator; Confidential Container Instances for serverless container groups with SEV-SNP and a policy generated by the az confcom extension; or peer pods on Azure confidential VMs, which Red Hat supports through OpenShift.

Google Cloud: confidential nodes, not confidential pods

GKE offers Confidential GKE Nodes backed by AMD SEV, AMD SEV-SNP or Intel TDX. You can enable it for a whole cluster, which is irreversible, for a node pool, or per workload through a ComputeClass that provisions dedicated tainted nodes. On Autopilot, SEV-SNP and TDX require GKE 1.35.2-gke.1485000 or later; on Standard with Container-Optimized OS, 1.32.2-gke.1297000 or later.

Sole-tenant nodes are not supported, only Container-Optimized OS and Ubuntu are available, and on machine types without live migration a host maintenance event marks the node NotReady. Because the boundary is the node, anyone with node or cluster admin rights is still inside your TCB.

AWS: Nitro Sandboxes change the conversation

AWS never offered SEV-SNP or TDX confidential containers on EKS. Its confidential story was Nitro Enclaves and, more recently, SEV-SNP on selected EC2 instances, which in July 2026 extended to Dedicated Hosts. That changed on 10 November 2026, when AWS announced Nitro Sandboxes. Each EKS pod or ECS task runs in its own hardware-isolated VM with its own kernel. You opt in by setting runtimeClassName to isolated-sandbox or confidential-sandbox.

In confidential mode, sandboxes produce Nitro Security Module attestation documents in the style of Nitro Enclaves. AWS KMS key policies can require specific measurements before releasing a key, and image loading is gated by a Cedar policy. AWS states that neither the cluster administrator, the platform team nor AWS can read pod memory. The isolation is enforced by the Nitro Hypervisor rather than by SEV-SNP or TDX, so the trust root is AWS’s Nitro hardware rather than AMD or Intel silicon. AWS says sandboxes work on every instance type that supports Nitro Enclaves, at no extra charge beyond the instance and any EKS Auto Mode fees.

This is a different trust model from CoCo: with SNP or TDX you trust AMD or Intel and verify with Trustee anywhere; with Nitro Sandboxes you trust AWS hardware and verify through KMS. The launch post does not cover GPU support, kubectl exec behaviour or log handling in confidential mode, so test those before you commit.

Capacity, Performance and Cost: What to Budget For

Vendors rarely publish neutral overhead numbers for confidential containers, and the ones that exist depend heavily on workload, kernel version and I/O pattern. The figures in this section are illustrative planning estimates drawn from first principles, not benchmarks. Measure your own workload before you size a cluster.

The first cost is memory. Every confidential pod is a VM with its own guest kernel, agent and page cache. Plan for a fixed overhead of a few hundred megabytes per pod on top of the container’s own limit (illustrative). On a node running 40 small pods, that can add 8 to 16 GB that would otherwise be shared through the host kernel. Confidential VMs also cannot share memory pages with the host through deduplication, and memory ballooning is constrained, so overcommit is largely off the table. This is why the AKS preview documentation told users to set resource limits rather than relying on requests: the VM must be sized up front.

The second cost is start-up time. A confidential pod boots firmware and a guest kernel, validates its memory (SEV-SNP requires each guest page to be accepted before use), attests, then pulls the image inside the guest without the host’s shared layer cache. On bare metal, expect several seconds more than a runc pod and much more for large images, because every pod pulls its own layers (illustrative). With peer pods, add cloud VM provisioning time on top. Workloads that scale from zero on demand will feel this; long-running services will not.

The third cost is I/O. A confidential guest cannot let devices read its encrypted memory directly, so virtio I/O passes through bounce buffers in shared memory, with an extra copy on every transfer. CPU-bound work typically shows small overhead, while storage-heavy and network-heavy workloads show more (illustrative). For confidential GPUs, data crossing PCIe is encrypted and staged through bounce buffers, so transfer-heavy pipelines suffer more than compute-heavy inference on large batches.

A simple worked example helps set expectations. Suppose a fraud-scoring service runs 30 replicas at 2 GB each on runc today, using 60 GB. Moving it to CoCo with an assumed 300 MB of per-pod VM overhead adds 9 GB, a 15 percent increase in memory alone, before any loss of overcommit headroom (illustrative). If the service also scales from zero during traffic spikes, the slower start-up may force you to keep more warm replicas, which is often the larger cost.

Finally, count new infrastructure: a highly available Trustee, its session store, a reference-value pipeline and attestation monitoring.

Trade-offs, Gotchas, and What Goes Wrong

The most common failure is attesting the hardware but not the software. Teams deploy Trustee, see attestation succeed, and never load reference values or a TCB floor. The result proves the chip is genuine and nothing else. Make reference values part of the release pipeline, generated from the same build that produces the guest image.

The second is operational access. A secure agent policy blocks kubectl exec and, depending on configuration, log streaming. CoCo v0.23.0 says so explicitly. Developers will ask for an exception the first time a pod misbehaves. Granting it quietly reintroduces the host into the TCB. Design observability into the workload instead: export metrics and structured logs over an authenticated channel you control.

The third is the image path. The v0.23.0 notes warn that guest pull with the nydus snapshotter sometimes fails, that containerd older than 2.3.x may have guest-pull problems, that CRI-O support is still evolving, and that SELinux must be permissive on hosts. These are not show-stoppers, but they mean you should pin your node OS, containerd and CoCo versions together and upgrade them as a set.

The fourth is the limits of the hardware threat model. In October 2025, researchers disclosed TEE.Fail, a DDR5 memory-bus interposer built from parts costing under 1,000 US dollars. It extracted secrets, including attestation keys, from Intel SGX and TDX and from AMD SEV-SNP with ciphertext hiding enabled, and the researchers showed knock-on effects for NVIDIA GPU confidential computing. Earlier attacks, Battering RAM and WireTap, targeted DDR4. AMD and Intel both state that physical attacks are out of scope. The practical reading is that confidential computing protects you from remote and software-level attackers, including a malicious hypervisor, but not from someone with physical access and a logic analyser in the data centre. If that threat matters, you are relying on the provider’s physical security.

The fifth is availability and side channels: the host can always kill or starve your VM and still observes timing, page faults and network metadata.

The last is confusing per-node with per-pod: if a requirement says “no administrator can access the data,” confidential nodes alone do not satisfy it.

Practical Recommendations

Start by writing down the threat model in one sentence. “Protect from the cloud operator” leads to confidential nodes, which are cheap and nearly transparent. “Protect from the cloud operator and our own platform team” leads to per-pod CoCo, Nitro Sandboxes or peer pods, with all the operational change that implies.

Run Trustee outside the cluster it attests, and treat it as tier-zero infrastructure with the same care as your root CA. Then build the reference-value pipeline before you onboard a single workload. Use kata-qemu-coco-dev in CI to catch policy and image-pull problems early, and block it in production namespaces.

For AI inference, begin with one model on whole GPUs and measure end-to-end latency, including attestation and model-weight decryption at start-up. Pair confidential containers with conventional controls: Pod Security Standards, admission policies and seccomp hardening still matter inside the guest.

Checklist before production:

  • Threat model written down, with per-node versus per-pod decided explicitly.
  • Runtime class pinned to kata-qemu-snp, kata-qemu-tdx or a GPU variant; coco-dev blocked by admission policy.
  • Agent policy generated with genpolicy, reviewed, and delivered through init-data.
  • KBS resource policies require the init-data hash and a minimum TCB version.
  • Reference values generated by the same pipeline that builds guest images.
  • Images signed, and encrypted if registry credentials or contents are sensitive.
  • Secrets delivered as sealed secrets, not plain Kubernetes Secrets.
  • Attestation latency and failures exported as metrics with alerts.
  • containerd, Kata, CoCo and node OS versions pinned and upgraded together.

Frequently Asked Questions

What is the difference between Kata Containers and Confidential Containers?

Kata Containers runs each pod in a lightweight VM to isolate it from other pods and from the host kernel, but the host and hypervisor can still read that VM’s memory. Confidential Containers builds on Kata and launches the VM as a hardware-encrypted SEV-SNP, TDX or Secure Execution guest. It also moves image pulls inside the guest, enforces a Rego agent policy against host requests, and gates secrets on remote attestation through Trustee. Kata protects the host from the pod; CoCo also protects the pod from the host.

Is Confidential Containers production-ready in 2026?

The upstream project is a CNCF incubating project, which signals maturity and adoption, not a vendor support contract. Commercial support exists: Red Hat’s OpenShift sandboxed containers 1.13 offers GA confidential containers on bare metal, including confidential GPUs, and NVIDIA’s reference architecture reached GA in April 2026. The v0.23.0 release notes still list real limitations, including registry credentials visible to the host and evolving CRI-O support. Production-ready for a well-scoped workload with a capable platform team, yes; a drop-in default for every pod, no.

Can I run confidential containers on AKS, EKS or GKE today?

On AKS, the original Confidential Containers preview was announced for retirement, with existing pools supported until 2 March 2026; use confidential VM node pools, Confidential Container Instances or peer pods instead. On EKS, AWS Nitro Sandboxes launched on 10 November 2026 with a confidential-sandbox runtime class for per-pod confidentiality. On GKE, Confidential GKE Nodes protect whole nodes with SEV, SEV-SNP or TDX, which excludes Google but not your cluster administrators. Per-pod CoCo on any managed cluster generally means peer pods.

Do I need to change my application to use confidential containers?

Usually not. SEV-SNP and TDX protect the whole VM, so an unmodified container image runs inside. The changes are in deployment and operations: a runtime class, an init-data annotation carrying the agent policy and KBS address, image digests instead of mutable tags, and sealed secrets in place of plain Kubernetes Secrets. You will also lose easy kubectl exec access under a strict policy, so build logging and metrics into the application. Code changes are only needed if the application itself must request secrets from the Confidential Data Hub.

How do confidential GPUs work with Kubernetes?

The GPU is passed through to the confidential VM and switched into confidential computing mode, which encrypts traffic between the guest and the GPU over PCIe. The GPU produces its own attestation report, so verification covers both the CPU TEE and the GPU. NVIDIA’s reference architecture supports H100, H200, B200, RTX Pro 6000 Blackwell Server Edition and, since August 2026, HGX B300. CoCo provides kata-qemu-nvidia-gpu-snp and kata-qemu-nvidia-gpu-tdx runtime classes. All GPUs on a host must be in confidential mode, which rules out MIG-style sharing.

Does confidential computing stop all attacks on data in use?

No. It stops software on the host, including a malicious hypervisor or a root user, from reading or tampering with guest memory, and attestation proves what booted. It does not fix application bugs, protect data your code sends out, or guarantee availability. Physical attacks are explicitly out of scope for AMD and Intel, and 2025 research such as TEE.Fail showed that a cheap DDR5 memory interposer could extract keys from SEV-SNP and TDX. Side channels are narrowed, not eliminated.

Further Reading

By Riju — about

2 Comments

Leave a Reply

Your email address will not be published. Required fields are marked *