Streaming

How it works

With stream: true the answer arrives as text/event-stream (server-sent events, SSE). Each data: line holds a chat.completion.chunk JSON object whose choices[0].delta.content is the next piece of text, and the stream ends with the line data: [DONE]. Lines are separated by a blank line, so read the response body line by line, keep a partial line in a buffer, and ignore lines that do not start with data: .

import jsonimport os import requests payload = {    "model": "MODEL_ID",    "messages": [{"role": "user", "content": "Write a two-line poem."}],    "stream": True,    "stream_options": {"include_usage": True},}with requests.post(    "https://api.hoveydai.ir/v1/chat/completions",    headers={"Authorization": f"Bearer {os.environ['HOVEYDAI_API_KEY']}"},    json=payload,    stream=True,    timeout=120,) as reply:    reply.raise_for_status()    for line in reply.iter_lines(decode_unicode=True):        if not line.startswith("data: "):            continue        data = line[len("data: "):]        if data == "[DONE]":            break        chunk = json.loads(data)        if "error" in chunk:            raise RuntimeError(chunk["error"]["message"])        if chunk["choices"]:            print(chunk["choices"][0]["delta"].get("content") or "", end="")

With include_usage the last chunk before [DONE] carries usage and its choices is empty; the code above handles that. Without it, no usage is sent in the stream.

Errors during a stream

• Errors before the first chunk (key, model, credit) come back as a normal HTTP error status with the JSON body described on the Errors page.
• A failure after the first chunk cannot change the 200 status. It arrives as an event data: {"error": {...}} and the stream ends without [DONE]. Keep the text received so far and decide whether to send the request again.
• If the connection drops with no error event, treat the answer as unfinished: only data: [DONE] marks a complete stream.