Chat Completions

Create OpenAI-compatible chat completion responses with Confidential AI models.

Endpoint

POST https://inference.phala.com/v1/chat/completions

Creates a response for a chat conversation. Use the same OpenAI-compatible request shape you already use with the OpenAI SDK, then set the base URL to https://inference.phala.com/v1.

Request Body

model · string · required

Model ID to use for completion.

Examples: phala/qwen3.5-27b, phala/gemma-3-27b-it, z-ai/glm-5, openai/gpt-oss-120b.

messages · array · required

Conversation messages. Each message includes role and content.

[
  {"role": "system", "content": "You are a helpful assistant."},
  {"role": "user", "content": "Explain GPU TEE in one paragraph."}
]

temperature · number

Sampling temperature. Typical range is 0 to 2.

max_tokens · integer

Maximum number of output tokens for most open models and GPU TEE models.

max_completion_tokens · integer

Maximum output tokens for newer OpenAI reasoning models that do not accept max_tokens.

stream · boolean

Set to true to receive server-sent event chunks.

tools · array

Function/tool definitions that supported models can call.

tool_choice · string | object

Controls whether the model may call tools. Common values are auto, none, or a specific tool selection object.

response_format · object

Requests structured output from supported models, including JSON schema mode.

provider · object

Routing constraints for this request. Fields are applied together, and the request is served only by a route that satisfies all of them.

zdr (boolean): set true to route only to providers that operate under zero data retention, meaning they do not retain prompt or completion content after serving the request. false, or omitting the field, applies no retention constraint.

{"provider": {"zdr": true}}

If no route for the model satisfies the constraint, the request returns 404 and no prompt is sent. The constraint is enforced for the whole request, including failover, so a retry cannot fall back to a route that does not satisfy it.

Find models that can satisfy zdr ahead of time with GET /v1/models?zdr=true. See Zero Data Retention.

aci_verified (boolean): set true to route only to an upstream the gateway verified inside the TEE before sending the prompt. false, or omitting the field, applies no attestation constraint.

{"provider": {"aci_verified": true}}

If no upstream for the model is both classified as attested and currently verified, the request returns 503 and no prompt is sent. Nothing in the request can relax this constraint once it is set.

aci_session_ids (array of strings): restrict the request further to specific attested channels, identified by the session ids from GET /v1/aci/sessions. Any one of the listed ids may serve the request; if none matches a currently verified channel, the request returns 503.

{"provider": {"aci_verified": true, "aci_session_ids": ["as_2a44...6290"]}}

Session ids imply aci_verified. Passing them with "aci_verified": false returns 400, as does an empty array or a non-string entry. Both fields are read by the gateway and removed before the request reaches a provider. See Attested Routing.

Examples

cURL

curl https://inference.phala.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <API_KEY>" \
  -d '{
    "model": "phala/qwen3.5-27b",
    "messages": [
      {"role": "user", "content": "What privacy guarantees does GPU TEE provide?"}
    ]
  }'

Response

{
  "id": "chatcmpl-123",
  "object": "chat.completion",
  "created": 1677652288,
  "model": "phala/qwen3.5-27b",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "GPU TEE protects inference by..."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 16,
    "completion_tokens": 48,
    "total_tokens": 64
  }
}

Raw HTTP responses include verification headers. Use x-receipt-id with Get Receipt when you need cryptographic proof for this specific response. The response id can also be used as a receipt lookup id.

Header	Meaning
x-receipt-id	Signed receipt id for this response.
x-aci-identity	Attested gateway workload identity.
x-aci-keyset-digest	Digest of the attested gateway keyset.

Feature Notes

• Streaming uses the same stream: true option as the OpenAI API.
• Vision models accept multimodal content arrays with image_url entries.
• Tool calling uses OpenAI-compatible tools, tool_choice, assistant tool_calls, and tool response messages.
• Structured output uses response_format on supported models.

Next Steps

List Models

Discover available Confidential AI models and capabilities

Verify Responses

Fetch the receipt for a chat completion response