Build a Streaming Streamlit Chat App with CometAPI

Last reviewed: 2026-09-01

Direct answer

You can build a CometAPI Streamlit streaming chat app by configuring the OpenAI Python client with CometAPI’s base URL, sending the current message list through client.chat.completions.create(..., stream=True), and passing the returned stream directly to st.write_stream. Keep the API client on the server, collect prompts with st.chat_input, display each turn with st.chat_message, and append an assistant turn only after the stream completes successfully.

This approach follows CometAPI’s OpenAI-compatible quickstart , which documents the Python client pattern, the POST /v1/chat/completions route, and incremental Server-Sent Events when stream is true. Streamlit’s chat elements reference defines the input and message containers. Its st.write_stream reference says the function accepts an OpenAI Stream, renders string chunks progressively, and returns the complete response.

Create a virtual environment and install the two application dependencies:

python -m venv .venv
. .venv/bin/activate
python -m pip install streamlit openai

Before launch, provide three process environment variables: COMETAPI_KEY, COMETAPI_MODEL, and COMETAPI_BASE_URL. Choose a current text-capable CometAPI model ID. Set the base URL to the /v1 value shown in the CometAPI quickstart . Keep the key outside the repository and never print its value.

Save this as app.py:

import logging
import os
import time
import uuid

import streamlit as st
from openai import OpenAI

logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
logger = logging.getLogger("cometapi_streamlit_chat")

required_config = ("COMETAPI_KEY", "COMETAPI_MODEL", "COMETAPI_BASE_URL")
missing_config = [name for name in required_config if not os.environ.get(name)]

st.set_page_config(page_title="CometAPI Chat")
st.title("CometAPI Chat")

if missing_config:
    st.error("Missing required environment configuration: " + ", ".join(missing_config))
    st.stop()

model = os.environ["COMETAPI_MODEL"]
client = OpenAI(
    api_key=os.environ["COMETAPI_KEY"],
    base_url=os.environ["COMETAPI_BASE_URL"],
)

if "messages" not in st.session_state:
    st.session_state["messages"] = []

for message in st.session_state["messages"]:
    with st.chat_message(message["role"]):
        st.write(message["content"])

prompt = st.chat_input("Send a message")

if prompt:
    event_id = uuid.uuid4().hex[:12]
    st.session_state["messages"].append(
        {"role": "user", "content": prompt}
    )

    with st.chat_message("user"):
        st.write(prompt)

    started = time.monotonic()

    try:
        stream = client.chat.completions.create(
            model=model,
            messages=st.session_state["messages"],
            stream=True,
        )

        with st.chat_message("assistant"):
            answer = st.write_stream(stream)

        if not isinstance(answer, str) or not answer.strip():
            raise RuntimeError("The completed stream did not return text")
    except Exception as exc:
        latency_ms = round((time.monotonic() - started) * 1000)
        status = getattr(exc, "status_code", None)
        safe_status = status if isinstance(status, int) else "unavailable"
        logger.error(
            "event=chat_request_failed event_id=%s error_type=%s "
            "status_code=%s message_count=%d latency_ms=%d",
            event_id,
            type(exc).__name__,
            safe_status,
            len(st.session_state["messages"]),
            latency_ms,
        )
        st.error(f"The request failed. Reference: {event_id}")
    else:
        st.session_state["messages"].append(
            {"role": "assistant", "content": answer}
        )
        latency_ms = round((time.monotonic() - started) * 1000)
        logger.info(
            "event=chat_request_complete event_id=%s "
            "message_count=%d latency_ms=%d output_chars=%d",
            event_id,
            len(st.session_state["messages"]),
            latency_ms,
            len(answer),
        )

Run the app with:

streamlit run app.py

The request includes the existing conversation plus the new user turn. st.write_stream consumes the SDK stream and returns the assembled text. The else branch then records that completed answer for the next Streamlit rerun. If streaming raises an error or returns no text, the code shows a short reference to the reader and does not record a completed assistant turn.

Who this is for

This tutorial is for Python developers who want a small browser chat interface without maintaining a separate JavaScript frontend. It is a useful starting point for internal demonstrations, model evaluations, support prototypes, and low-volume tools where one Streamlit process owns both the interface and the CometAPI call.

It assumes you can run Python 3.10 or newer, install packages, configure process environment variables, and select a text-capable CometAPI model. It does not attempt to add user authentication, durable cross-session storage, or an unbounded production retry system. Those concerns should be added deliberately after the basic request and stream contract works.

Key takeaways

• Configure the Python SDK with the CometAPI base URL and a current CometAPI model ID.
• Send messages through the documented Chat Completions method and set stream=True.
• Use st.chat_input for the prompt and st.chat_message for user and assistant turns.
• Pass the SDK stream to st.write_stream, then store only its completed nonempty string result.
• Keep prompt text, response text, request headers, keys, and other sensitive configuration values out of logs.
• Treat a failed or interrupted stream as incomplete instead of silently adding it to conversation history.

Sources checked

• The CometAPI OpenAI-compatible API quickstart , last modified June 30, 2026, documents the base URL, Python client configuration, Chat Completions route, messages, model IDs, and SSE streaming behavior.
• The Streamlit chat elements documentation documents how st.chat_input accepts a message and how st.chat_message creates a user or app message container.
• The Streamlit st.write_stream documentation , shown for Streamlit 1.62.0 when checked, documents accepted stream types, progressive string rendering, and the completed return value.
• The OpenAI Python library README confirms that the current Python library still exposes client.chat.completions.create and typed synchronous and asynchronous clients. CometAPI’s quickstart supplies the provider-specific base URL and model requirements used here.

Contract details to verify

First, verify the client boundary. base_url must point to CometAPI’s documented /v1 base, while the SDK method supplies the Chat Completions route. If the base URL is absent or points elsewhere, the same SDK call can reach the wrong service. The key and model must also belong to the CometAPI runtime configuration, not to another provider’s setup.

Second, verify the request family. This article uses Chat Completions, so the body is a messages array and the application reads a stream from client.chat.completions.create. Do not mix a Responses payload into this call. The CometAPI quickstart describes the route as synchronous by default and streamed only when stream is true.

Third, verify the selected model before troubleshooting the UI. Use a current text-capable CometAPI model ID rather than copying an old identifier from an unrelated example. The model-catalog prepublish checks provide a related validation workflow.

Fourth, preserve the stream return contract. Streamlit documents that st.write_stream returns a string when the streamed output contains only text, but it can return a list when other object types appear. The example therefore accepts only a nonempty string as a completed assistant turn. That guard prevents a non-text result from being written back into a text-only messages history.

Fifth, keep the example’s security boundary intact. The application reads its key from the process environment and never sends it to the browser, writes it into the page, or places it in a URL. Follow the same boundary described in the repository key-handling guide .

Finally, decide which optional parameters the chosen model accepts before adding them. The CometAPI quickstart lists temperature, max_completion_tokens, and response_format, but it also notes model-dependent behavior. Start with model, messages, and stream; add one optional control at a time and test its happy and error paths.

Failure modes

A useful app needs an operator path for both success and failure. The example emits structured, sanitized fields: event, event_id, error_type, status_code, message_count, latency_ms, and output_chars on success. It intentionally omits the configured model, prompts, responses, request headers, keys, other sensitive configuration values, and exception messages. An exception message can contain upstream details that are inappropriate for routine logs, so the reader sees only the generated reference.

Happy-path operator workflow

1. Confirm that all three environment variable names are present in the Streamlit runtime without displaying their values.
2. Launch streamlit run app.py and submit: Reply with exactly: stream check.
3. Verify that the user turn appears once, the assistant text renders progressively, and no error reference appears.
4. Submit a second prompt asking which phrase was requested. This confirms that the application sends the accumulated messages list.
5. Check for one chat_request_complete log event per successful turn. Confirm that it includes latency and output length but no message content or configuration values.

Error-path operator workflow

1. In a non-production test environment, temporarily set COMETAPI_MODEL to model-not-available while leaving the key unchanged.
2. Restart the app and submit a short prompt.
3. Confirm that the UI displays a request reference, the log contains chat_request_failed, and no completed assistant turn is added to history.
4. Verify that logs contain only the sanitized fields listed above. Do not add the configured model, raw exception, request body, or streamed chunks to diagnose the test.
5. Restore a current model ID, restart, and repeat the happy-path check. Add an explicit client timeout before wider deployment using the client-side timeout pattern .

FAQ

Can st.write_stream consume the SDK result directly?

Yes, for the object used here. Streamlit documents native parsing for OpenAI Chat Completions streams, and the CometAPI quickstart documents using the OpenAI Python SDK with CometAPI’s compatible Chat Completions route. The call returns the SDK stream object, which is passed directly to st.write_stream.

Why append the assistant message after streaming?

st.write_stream returns the complete text after it consumes a text stream. Waiting for that return value gives the app one completed string to store. If an exception interrupts the stream, the success branch does not run, so the message history does not claim that a complete assistant answer exists.

Why not parse SSE lines manually?

The CometAPI route uses SSE for incremental output, but the SDK and Streamlit already provide the two abstractions this example needs. Manual line parsing would add framing and chunk-extraction code without improving this small app. It may be appropriate only when a different HTTP client or a custom transport is a deliberate requirement.

Can this use the Responses API instead?

Streamlit’s st.write_stream documentation says it natively parses both OpenAI Chat Completions and Responses streams. This tutorial intentionally stays with Chat Completions because that is the CometAPI route, request shape, and Python pattern documented by the CometAPI quickstart. A migration should verify the separate endpoint contract rather than changing only the method name.

Is the example production ready?

It is a working integration foundation, not a complete public service. Before broader use, add access control, bounded conversation history, explicit timeouts, model-specific output limits, deployment monitoring, and an approved persistence policy. Test each addition against both a completed stream and an interrupted stream.

What should never appear in the logs?

Do not log keys, other sensitive configuration values, request headers, prompt text, response text, full message history, or raw streamed chunks. The example also omits the configured model to keep its logging policy simple. Use the short event reference to correlate the browser error with sanitized operational fields.

Reader next step

Run the happy-path check first, then force the non-production model error and verify that the app fails without recording a false assistant turn. Once both paths behave correctly, pin the package versions you tested, add a client timeout, and set a conversation-history limit appropriate to your selected model.

When you are ready to configure the provider and choose a current model, Start with CometAPI .