Streaming Responses

Token101 supports real-time streaming for all text generation endpoints. Streaming delivers tokens as they are generated, giving users a faster, more interactive experience than waiting for the full response.

Streaming is implemented using Server-Sent Events (SSE) — the same standard used by the Anthropic and OpenAI APIs.

How Streaming Works

When you set stream: true (or "stream": true in JSON), the server sends partial response chunks over a persistent HTTP connection. Each chunk is a JSON object prefixed with data: . The stream ends with data: [DONE].

data: {"type":"content_block_delta","delta":{"text":"Hello"}}
data: {"type":"content_block_delta","delta":{"text":", world"}}
data: {"type":"message_stop"}
data: [DONE]

Anthropic Messages API

Python — Streaming

import anthropic
import os

client = anthropic.Anthropic(
    api_key=os.environ.get("TOKEN101_API_KEY"),
    base_url="https://token.ppthub.shop/api",
)

with client.messages.stream(
    model="gpt-5.2",
    max_tokens=512,
    messages=[{"role": "user", "content": "Write a haiku about the ocean."}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
print()  # newline at end

Python — Low-level streaming (manual event handling)

import anthropic
import os

client = anthropic.Anthropic(
    api_key=os.environ.get("TOKEN101_API_KEY"),
    base_url="https://token.ppthub.shop/api",
)

with client.messages.stream(
    model="gpt-5.2",
    max_tokens=512,
    messages=[{"role": "user", "content": "Write a short poem."}],
) as stream:
    for event in stream:
        if hasattr(event, 'delta') and hasattr(event.delta, 'text'):
            print(event.delta.text, end="", flush=True)

Node.js / TypeScript — Streaming

import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic({
  apiKey: process.env.TOKEN101_API_KEY,
  baseURL: 'https://token.ppthub.shop/api',
});

const stream = await client.messages.stream({
  model: 'gpt-5.2',
  max_tokens: 512,
  messages: [{ role: 'user', content: 'Write a haiku about the ocean.' }],
});

for await (const event of stream) {
  if (event.type === 'content_block_delta' && event.delta.type === 'text_delta') {
    process.stdout.write(event.delta.text);
  }
}
console.log(); // newline at end

curl — Raw SSE

curl https://token.ppthub.shop/api/v1/messages \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN101_API_KEY" \
  -d '{
    "model": "gpt-5.2",
    "max_tokens": 512,
    "stream": true,
    "messages": [
      {"role": "user", "content": "Write a haiku about the ocean."}
    ]
  }'

OpenAI Chat Completions API

Python — Streaming

from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ.get("TOKEN101_API_KEY"),
    base_url="https://token.ppthub.shop/api/v1",
)

stream = client.chat.completions.create(
    model="gpt-5.2",
    messages=[{"role": "user", "content": "Write a haiku about the ocean."}],
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta
    if delta.content:
        print(delta.content, end="", flush=True)
print()

Node.js / TypeScript — Streaming

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.TOKEN101_API_KEY,
  baseURL: 'https://token.ppthub.shop/api/v1',
});

const stream = await client.chat.completions.create({
  model: 'gpt-5.2',
  messages: [{ role: 'user', content: 'Write a haiku about the ocean.' }],
  stream: true,
});

for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content;
  if (content) {
    process.stdout.write(content);
  }
}
console.log();

curl — Raw SSE

curl https://token.ppthub.shop/api/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN101_API_KEY" \
  -d '{
    "model": "gpt-5.2",
    "stream": true,
    "messages": [
      {"role": "user", "content": "Write a haiku about the ocean."}
    ]
  }'

Streaming in Web Applications

React — Custom hook

import { useState, useCallback } from 'react';

export function useStream() {
  const [content, setContent] = useState('');
  const [isStreaming, setIsStreaming] = useState(false);

  const streamMessage = useCallback(async (prompt: string) => {
    setContent('');
    setIsStreaming(true);

    const response = await fetch('https://token.ppthub.shop/api/v1/chat/completions', {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        Authorization: `Bearer ${process.env.NEXT_PUBLIC_TOKEN101_KEY}`,
      },
      body: JSON.stringify({
        model: 'gpt-5.2',
        stream: true,
        messages: [{ role: 'user', content: prompt }],
      }),
    });

    const reader = response.body!.getReader();
    const decoder = new TextDecoder();

    while (true) {
      const { done, value } = await reader.read();
      if (done) break;

      const lines = decoder.decode(value).split('\n');
      for (const line of lines) {
        if (line.startsWith('data: ') && line !== 'data: [DONE]') {
          const data = JSON.parse(line.slice(6));
          const text = data.choices?.[0]?.delta?.content;
          if (text) setContent((prev) => prev + text);
        }
      }
    }

    setIsStreaming(false);
  }, []);

  return { content, isStreaming, streamMessage };
}

Never expose your Token101 API key in client-side code. Use a server-side API route to proxy requests and keep the key secret.

Next.js — Server-side streaming API route

// app/api/chat/route.ts
import { NextRequest } from 'next/server';

export async function POST(req: NextRequest) {
  const { messages } = await req.json();

  const response = await fetch('https://token.ppthub.shop/api/v1/chat/completions', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      Authorization: `Bearer ${process.env.TOKEN101_API_KEY}`,
    },
    body: JSON.stringify({ model: 'gpt-5.2', stream: true, messages }),
  });

  // Pass the SSE stream directly to the client
  return new Response(response.body, {
    headers: {
      'Content-Type': 'text/event-stream',
      'Cache-Control': 'no-cache',
    },
  });
}

Handling Stream Events

Anthropic Messages API event types

Event Type	Description
`message_start`	Contains initial message metadata
`content_block_start`	Start of a content block (text or tool_use)
`content_block_delta`	Incremental text or tool input
`content_block_stop`	End of a content block
`message_delta`	Final message stats (stop_reason, usage)
`message_stop`	End of message

OpenAI Chat Completions event types

Field	Description
`choices[0].delta.content`	Text content chunk (may be null)
`choices[0].delta.role`	Role (only in first chunk)
`choices[0].finish_reason`	Reason stream ended (null until last chunk)

Tips and Best Practices

Always flush output — when printing streamed tokens in a terminal, use flush=True (Python) or process.stdout.write() (Node.js) to avoid buffering.

Handle stream errors — wrap streaming code in try/catch. If the connection drops mid-stream, the SDK will throw an error.

Set appropriate timeouts — streaming responses can take longer than non-streaming ones for long outputs. Configure your HTTP client with a generous read timeout (60+ seconds).

Token counting — streaming responses include usage metadata in the final event (message_delta for Anthropic, last chunk's usage for OpenAI).

Next Steps

Tool Calling — use tools and function calling with streaming
Supported Models — see which models support streaming
API Reference — full endpoint documentation

Streaming Responses

Streaming is implemented using Server-Sent Events (SSE) — the same standard used by the Anthropic and OpenAI APIs.

How Streaming Works

data: {"type":"content_block_delta","delta":{"text":"Hello"}}
data: {"type":"content_block_delta","delta":{"text":", world"}}
data: {"type":"message_stop"}
data: [DONE]

Anthropic Messages API

Python — Streaming

import anthropic
import os

client = anthropic.Anthropic(
    api_key=os.environ.get("TOKEN101_API_KEY"),
    base_url="https://token.ppthub.shop/api",
)

with client.messages.stream(
    model="gpt-5.2",
    max_tokens=512,
    messages=[{"role": "user", "content": "Write a haiku about the ocean."}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
print()  # newline at end

Python — Low-level streaming (manual event handling)

import anthropic
import os

client = anthropic.Anthropic(
    api_key=os.environ.get("TOKEN101_API_KEY"),
    base_url="https://token.ppthub.shop/api",
)

with client.messages.stream(
    model="gpt-5.2",
    max_tokens=512,
    messages=[{"role": "user", "content": "Write a short poem."}],
) as stream:
    for event in stream:
        if hasattr(event, 'delta') and hasattr(event.delta, 'text'):
            print(event.delta.text, end="", flush=True)

Node.js / TypeScript — Streaming

import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic({
  apiKey: process.env.TOKEN101_API_KEY,
  baseURL: 'https://token.ppthub.shop/api',
});

const stream = await client.messages.stream({
  model: 'gpt-5.2',
  max_tokens: 512,
  messages: [{ role: 'user', content: 'Write a haiku about the ocean.' }],
});

for await (const event of stream) {
  if (event.type === 'content_block_delta' && event.delta.type === 'text_delta') {
    process.stdout.write(event.delta.text);
  }
}
console.log(); // newline at end

curl — Raw SSE

curl https://token.ppthub.shop/api/v1/messages \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN101_API_KEY" \
  -d '{
    "model": "gpt-5.2",
    "max_tokens": 512,
    "stream": true,
    "messages": [
      {"role": "user", "content": "Write a haiku about the ocean."}
    ]
  }'

OpenAI Chat Completions API

Python — Streaming

from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ.get("TOKEN101_API_KEY"),
    base_url="https://token.ppthub.shop/api/v1",
)

stream = client.chat.completions.create(
    model="gpt-5.2",
    messages=[{"role": "user", "content": "Write a haiku about the ocean."}],
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta
    if delta.content:
        print(delta.content, end="", flush=True)
print()

Node.js / TypeScript — Streaming

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.TOKEN101_API_KEY,
  baseURL: 'https://token.ppthub.shop/api/v1',
});

const stream = await client.chat.completions.create({
  model: 'gpt-5.2',
  messages: [{ role: 'user', content: 'Write a haiku about the ocean.' }],
  stream: true,
});

for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content;
  if (content) {
    process.stdout.write(content);
  }
}
console.log();

curl — Raw SSE

curl https://token.ppthub.shop/api/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN101_API_KEY" \
  -d '{
    "model": "gpt-5.2",
    "stream": true,
    "messages": [
      {"role": "user", "content": "Write a haiku about the ocean."}
    ]
  }'

Streaming in Web Applications

React — Custom hook

import { useState, useCallback } from 'react';

export function useStream() {
  const [content, setContent] = useState('');
  const [isStreaming, setIsStreaming] = useState(false);

  const streamMessage = useCallback(async (prompt: string) => {
    setContent('');
    setIsStreaming(true);

    const response = await fetch('https://token.ppthub.shop/api/v1/chat/completions', {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        Authorization: `Bearer ${process.env.NEXT_PUBLIC_TOKEN101_KEY}`,
      },
      body: JSON.stringify({
        model: 'gpt-5.2',
        stream: true,
        messages: [{ role: 'user', content: prompt }],
      }),
    });

    const reader = response.body!.getReader();
    const decoder = new TextDecoder();

    while (true) {
      const { done, value } = await reader.read();
      if (done) break;

      const lines = decoder.decode(value).split('\n');
      for (const line of lines) {
        if (line.startsWith('data: ') && line !== 'data: [DONE]') {
          const data = JSON.parse(line.slice(6));
          const text = data.choices?.[0]?.delta?.content;
          if (text) setContent((prev) => prev + text);
        }
      }
    }

    setIsStreaming(false);
  }, []);

  return { content, isStreaming, streamMessage };
}

Never expose your Token101 API key in client-side code. Use a server-side API route to proxy requests and keep the key secret.

Next.js — Server-side streaming API route

// app/api/chat/route.ts
import { NextRequest } from 'next/server';

export async function POST(req: NextRequest) {
  const { messages } = await req.json();

  const response = await fetch('https://token.ppthub.shop/api/v1/chat/completions', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      Authorization: `Bearer ${process.env.TOKEN101_API_KEY}`,
    },
    body: JSON.stringify({ model: 'gpt-5.2', stream: true, messages }),
  });

  // Pass the SSE stream directly to the client
  return new Response(response.body, {
    headers: {
      'Content-Type': 'text/event-stream',
      'Cache-Control': 'no-cache',
    },
  });
}

Handling Stream Events

Anthropic Messages API event types

Event Type	Description
`message_start`	Contains initial message metadata
`content_block_start`	Start of a content block (text or tool_use)
`content_block_delta`	Incremental text or tool input
`content_block_stop`	End of a content block
`message_delta`	Final message stats (stop_reason, usage)
`message_stop`	End of message

OpenAI Chat Completions event types

Field	Description
`choices[0].delta.content`	Text content chunk (may be null)
`choices[0].delta.role`	Role (only in first chunk)
`choices[0].finish_reason`	Reason stream ended (null until last chunk)

Tips and Best Practices

Always flush output — when printing streamed tokens in a terminal, use flush=True (Python) or process.stdout.write() (Node.js) to avoid buffering.

Handle stream errors — wrap streaming code in try/catch. If the connection drops mid-stream, the SDK will throw an error.

Set appropriate timeouts — streaming responses can take longer than non-streaming ones for long outputs. Configure your HTTP client with a generous read timeout (60+ seconds).

Token counting — streaming responses include usage metadata in the final event (message_delta for Anthropic, last chunk's usage for OpenAI).

Next Steps

Tool Calling — use tools and function calling with streaming
Supported Models — see which models support streaming
API Reference — full endpoint documentation

Streaming

Table of Contents

Streaming

Table of Contents