Confidential GPU infrastructure: architecture, workloads, and buyer checks

The confidential boundary is a system

GPU execution is only confidential if plaintext does not fall out of the boundary on its way to, inside, or back from the accelerator. NVIDIA’s design extends a CPU confidential VM to supported GPUs. The platform must establish the VM, place the device in the intended mode, protect transfers, validate device state, and restrict key/data release to an approved attested configuration.NVIDIA-WP,NVIDIA-DEPLOY

Dedicated bare metal is useful—but not sufficient

Bare metal gives one tenant a physical server and can improve topology control, performance consistency, and separation from other customers. It does not inherently stop a privileged infrastructure operator from inspecting host memory, reconfiguring firmware, attaching a debugger, or observing an unprotected device path. A buyer should treat “dedicated” and “confidential” as separate fields.CCC-TECH

Workloads that benefit most

• Sensitive inference: prompts, retrieved documents, agent state, KV cache, credentials, and outputs should not be visible to the host operator.
• Proprietary model deployment: weights, fine-tuned adapters, or specialized kernels run in another party’s facility.
• Regulated analytics and AI: health, finance, legal, government, or personal data needs an additional data-in-use control.
• Multi-party computation workflows: organizations contribute data to a joint job without granting the infrastructure operator plaintext access.
• Sovereign or delegated operations: infrastructure is locally operated while a separate workload owner controls attestation policy and keys.
• High-value training or fine-tuning: training examples and resulting checkpoints are sensitive enough to justify a larger protected cluster.

The weakest fit is a workload whose main risks are malicious inputs, model behavior, output misuse, or an already-trusted application administrator. Confidential execution can complement controls for those risks, but it does not directly solve them.

Constraints and performance considerations

NVIDIA’s Hopper design uses encrypted and integrity-protected bounce buffers for traffic crossing between CPU-protected memory and the GPU. Workloads that move data frequently across that boundary can pay more overhead than long-running, compute-dense kernels. On-package HBM handling and the physical attack assumptions are platform-specific; use the vendor threat model rather than a blanket “all memory encrypted” statement.NVIDIA-WP

• Benchmark end-to-end throughput and tail latency, not just kernel execution.
• Include model load, attestation, encrypted artifact release, autoscaling, and node replacement in startup measurements.
• Check whether peer-to-peer, collective communication, live migration, device management, profiling, and debugging are supported in the intended secure mode.
• Treat development or diagnostics modes separately; NVIDIA’s CC-DevTools mode is not the same security state as CC-On.
• Validate the whole topology for multi-GPU and multi-node jobs; one attested accelerator does not cover an unverified peer.

Hardware capability, vendor support, and cloud availability

Procurement checklist

1. Record exact GPU SKU, form factor, count, topology, host CPU, system firmware, GPU firmware, driver branch, and confidential-computing mode.
2. Require a primary-source support reference for that combination and a named owner for security updates.
3. Obtain sample CPU and GPU evidence. Confirm the customer can verify it directly or through an explicitly trusted verifier.
4. Document protected and unprotected paths, including storage, NICs, host management, telemetry, logs, snapshots, crash dumps, and customer support.
5. Test that secrets are withheld when debug or development mode is active, a device is missing, firmware is below policy, or evidence is stale.
6. Measure the actual workload and topology under secure mode, then price delivered throughput rather than nominal GPU-hours.