How do I stream a reply in JavaScript?

Set stream: true and use for await. Text is in choices[0].delta.content.

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.gateway.clatterlabs.xyz/v1",
  apiKey: process.env.CLATTER_API_KEY,
  defaultHeaders: { "X-Clatter-Account": process.env.CLATTER_ACCOUNT_ID },
});

const stream = await client.chat.completions.create({
  model: "supercosmos-flash",
  messages: [{ role: "user", content: "Explain recursion in two sentences." }],
  stream: true,
  stream_options: { include_usage: true },
});
for await (const chunk of stream) {
  const t = chunk.choices[0]?.delta?.content;
  if (t) process.stdout.write(t);
  if (chunk.usage) console.log("\n", chunk.usage);
}

With include_usage, the final chunk has usage and an empty choices array, hence the ?..

Every JavaScript example uses the same client (base URL, key and the X-Clatter-Account header); see Set up the JavaScript SDK. Full parameter list: Create a chat completion.

Good to know#

• Your key's account is responsible for your end users' text under the Abuse policy. What it means for apps
• Every call shows in Console › Logs and Usage as Model API, with its price. See what you spent
• Every response has an X-Request-Id; log it so a request can be found later. Why log it

Still need help?#

• General support: support@clatterlabs.com or team@clatterlabs.xyz
• Infrastructure questions, or permanent deletion of your data (your whole account or one service): infrastructure@clatterlabs.xyz

We usually reply within the first hour, and always within 24 hours. A full support team is with you.