Confidential LLM inference

Run models inside hardware-isolated CPU and GPU environments.

API format

OpenAI-compatible

POST /v1/chat/completions

{
  "model": "deepseek-v4-flash",
  "messages": [
    { "role": "user", "content": "..." }
  ]
}

Operator-private

Infrastructure administrators cannot inspect prompts or responses inside the confidential runtime.

Hardware-isolated

CPU and GPU confidential computing isolates data while the model runs.

Attested runtime

Cryptographic evidence identifies the hardware and software serving the endpoint.

Traditional inference vs confidential inference

See who can access your data. See whether the runtime can prove its identity.

Traditional AI API

Privacy depends on provider policies.
Infrastructure admins can access request systems.
The cloud operator shares the execution boundary.
Clients get no hardware-backed proof of the runtime.

Confidential Nodes API

Prompts and responses stay inside the confidential runtime.
Infrastructure operators stay outside the runtime.
Models run in hardware-isolated CPU and GPU environments.
Attestation verifies the hardware and software serving the API.

Available models

Chat, multimodal, and speech models through one API format.

Model	Capability	Context	Price
Gemma 4 31B Google · `google/gemma-4-31b-it`	Text, image, video → text	262,144 tokens	Input$0.09/MOutput$0.34/MCache$0.05/M
Whisper large-v3-turbo OpenAI · `openai/whisper-large-v3-turbo`	Audio → text	Not token-based	Audio$0.04/hr
DeepSeek V4 Flash 0731 DeepSeek · `deepseek/deepseek-v4-flash-0731`	Text → text	1,310,720 tokens	Input$0.08/MOutput$0.18/MCache$0.016/M
GLM-5.3-Flash Z.ai · `z-ai/glm-5.3-flash`	Text, image, video → text	1,310,720 tokens	Input$0.15/MOutput$0.50/MCache$0.03/M

Questions about confidential inference

How the privacy boundary, verification, and API integration work.

What makes this API confidential?

Inference runs inside hardware-isolated CPU and GPU environments. Prompts and responses are processed inside the confidential runtime rather than the operator-controlled host.

Can infrastructure operators read prompts and responses?

No. Prompts and responses stay inside the confidential runtime while they are processed, keeping infrastructure operators outside the execution boundary.

How is confidential inference different from TLS?

TLS encrypts data while it travels between systems. Confidential computing also isolates the data while the model is running, covering data in use.

What does runtime attestation verify?

Runtime attestation provides cryptographic evidence identifying the hardware and software serving the endpoint, so the client can check the confidential environment.

Will the API work with existing OpenAI clients?

The API uses an OpenAI-compatible request format. Integrations can change the endpoint and model name while keeping the familiar request structure.

Which models are included?

The current model list includes Gemma 4 31B, Whisper large-v3-turbo, DeepSeek V4 Flash 0731, and GLM-5.3-Flash. Waitlist demand will help prioritize additional models.