How do I call the Model API asynchronously in Python?

AsyncOpenAI takes the same settings. Run several requests at once with asyncio.gather, keeping your key's per-minute limit in mind.

import asyncio, os
from openai import AsyncOpenAI

client = AsyncOpenAI(
    base_url="https://api.gateway.clatterlabs.xyz/v1",
    api_key=os.environ["CLATTER_API_KEY"],
    default_headers={"X-Clatter-Account": os.environ["CLATTER_ACCOUNT_ID"]},
)

async def ask(q):
    r = await client.chat.completions.create(model="supercosmos-flash", messages=[{"role": "user", "content": q}])
    return r.choices[0].message.content

async def main():
    sem = asyncio.Semaphore(5)          # cap concurrency
    async def one(q):
        async with sem:
            return await ask(q)
    print(await asyncio.gather(*(one(q) for q in ["2+2?", "Capital of Egypt?", "Spell 'cat'"])))

asyncio.run(main())

A semaphore keeps bursts under your key's limit (10, 30, 60 or 120 requests per minute).

The client setup above (base URL, key and the X-Clatter-Account header) is the same in every Python example; see Set up the Python SDK. Full parameter list: Create a chat completion.

Good to know#

• Keep keys on your server, never in a browser or mobile app. Why
• Every response has an X-Request-Id; log it so a request can be found later. Why log it
• Each key has its own limit: 10, 30 (default), 60 or 120 requests per minute. Rate limits

Still need help?#

• General support: support@clatterlabs.com or team@clatterlabs.xyz
• Infrastructure questions, or permanent deletion of your data (your whole account or one service): infrastructure@clatterlabs.xyz

We usually reply within the first hour, and always within 24 hours. A full support team is with you.