Streaming

Learn how to use streaming in Confidential AI API

This feature works for both API Access and Dedicated Models.

Streaming in Confidential AI API

Confidential AI API supports streaming, enabling you to receive responses in a streaming fashion. This is particularly useful for applications that require real-time data processing or integration with other systems.

Example of Streaming

Replace <API_KEY> with your actual API key in the examples below.

Python

from openai import OpenAI

client = OpenAI(
    api_key="<API_KEY>",
    base_url="https://inference.phala.com/v1",
)

stream = client.chat.completions.create(
    model="phala/qwen3.5-27b",
    messages=[
        {
            "role": "user",
            "content": "say `Hello` 2 times fast, no other output",
        },
    ],
    stream=True,
)
for chunk in stream:
    content = chunk.choices[0].delta.content
    if content:
        print(content, end="")

Sample output of streaming request

HelloHello

Supported Models

All models support streaming.