Capture Streaming Transcripts for Tutorial QA

Last reviewed: 2026-07-06

Direct answer

To capture streaming transcript evidence for tutorial QA, save the request fixture, each server-sent event chunk, the terminal data: [DONE] marker, and the final pass/fail note as separate parts of the same run record. The CometAPI chat completions reference documents streaming through stream: true on POST /v1/chat/completions, where partial output is delivered as Server-Sent Events and the stream ends with a done marker. That is enough to design a structural test without turning a model’s generated text into the pass condition.

Use this workflow before updating a tutorial that claims streamed chat output:

1. Setup assumptions: a sandbox CometAPI key is available through an environment variable, the tutorial fixture uses a placeholder prompt, and the model value is selected from the current models documentation at test time.
2. Happy-path request plan: send a minimal chat-completions request with stream: true, a documented message array, and placeholder content that does not include private user data.
3. Error-path check: repeat the request with a deliberately invalid placeholder credential and confirm the harness records the failed authorization outcome without printing any token value.
4. Minimum assertions: at least one SSE chunk is received, the stream ends with data: [DONE], the transcript file preserves event order, and the QA note links the fixture to the documentation pages checked that day.
5. Pass/fail logging fields: run_id, docs_checked, fixture_name, stream_started, done_marker_seen, chunk_count, error_case_result, maintainer_notes, and verdict.
6. What not to assert: do not assert a specific model inventory, price, quota, latency, uptime, exact generated wording, or account billing behavior from this smoke test.

For adjacent checks, pair this article with How to Smoke-test the CometAPI Chat Completion Contract and Validate CometAPI Chat Completions for Production . Those guides help you decide whether the transcript capture belongs in a small tutorial fixture, a broader release check, or a production validation package.

Who this is for

This guide is for tutorial maintainers who need a repeatable way to capture streaming output while keeping examples aligned with the current CometAPI documentation. It is useful when a tutorial says that an example supports streamed output and the maintainer needs evidence beyond a copied code block or a successful final answer.

The reader is assumed to be checking a tutorial sample, not benchmarking provider quality. The goal is to answer a narrow question: did the tutorial exercise the documented streaming shape, and can another maintainer inspect the transcript later without relying on memory? That is different from proving that a model is available in every account, that an account has a particular quota, or that a response will contain specific words.

This workflow also helps when a tutorial has both chat-completions and Responses examples nearby. The CometAPI docs describe the chat completions endpoint and the Responses endpoint separately, so a maintainer should confirm which endpoint the tutorial actually uses before saving a transcript. If the example uses chat messages, chat chunks, and the chat-completions path, keep the transcript attached to that endpoint. If the example uses a Responses fixture, compare it with the Responses reference instead of mixing the two shapes.

Key takeaways

• Capture the streaming transcript as ordered events, not as a single final assistant answer.
• Keep credentials out of fixtures and logs; use <API_KEY_PLACEHOLDER> in examples and documentation notes.
• Verify endpoint path, authentication, request fields, streaming behavior, and model selection against the linked documentation on the day the tutorial is checked.
• Treat prices, limits, usage cost, uptime, and account-specific availability as separate checks unless the current public source directly supports the claim.
• Record enough metadata for a later maintainer to reproduce the fixture without copying secrets, private prompts, or full model output.

A sanitized log record can look like this:

{
  "run_id": "stream-smoke-YYYYMMDD-001",
  "docs_checked": [
    "https://apidoc.cometapi.com/api/text/chat",
    "https://apidoc.cometapi.com/overview/models"
  ],
  "fixture_name": "tutorial-stream-placeholder",
  "credential_source": "COMETAPI_KEY environment variable",
  "credential_value": "<API_KEY_PLACEHOLDER>",
  "stream_started": true,
  "done_marker_seen": true,
  "chunk_count": "placeholder-integer",
  "error_case_result": "invalid credential rejected",
  "maintainer_notes": "placeholder notes only",
  "verdict": "pass-or-fail"
}

The transcript file should keep raw event order, but the public tutorial does not need to reproduce every event. A good tutorial can summarize the structure: request sent, chunks received, done marker observed, and unsupported claims excluded. If a code sample parses each SSE line, save the parser input and the parser output separately so the review can distinguish between an API behavior issue and a local parsing issue.

When choosing a model for the fixture, use the current model documentation or dashboard evidence available to the account running the check. Do not hard-code a model identifier from an old sample just because the shape of the request still looks valid. The model field is part of the request contract, but model availability can change by provider, account, or product tier.

Sources checked

• CometAPI chat completions reference - accessed 2026-07-06; purpose: verify chat completion contract areas.
• CometAPI documentation - accessed 2026-07-06; purpose: verify current CometAPI documentation navigation.
• CometAPI help center - accessed 2026-07-06; purpose: verify support and escalation documentation areas.
• CometAPI responses reference - accessed 2026-07-06; purpose: verify responses endpoint contract areas.
• CometAPI models overview - accessed 2026-07-06; purpose: verify model catalog discovery guidance.

Contract details to verify

Failure modes

• Evidence gap: the maintainer cannot inspect the failing log, source page, pull request, or local command output. The safe action is to stop and record the missing evidence instead of guessing.
• Scope drift: the repair changes files or examples that are not connected to the observed streaming failure. Keep the work tied to the failing fixture and leave unrelated cleanup for a separate update.
• Environment mismatch: the local check uses different package versions, credentials, feature flags, or runtime settings than the environment described by the tutorial. Record the mismatch before treating the result as proof.
• Unreviewed fallback: the sample changes models, endpoints, permissions, or retry behavior only to make a run pass. Treat access and provider failures as separate operational findings, not as proof that the tutorial text is correct.
• Weak handoff: the final note says the issue is fixed but omits the fixture name, observed result, changed files, and remaining uncertainty. That makes the next maintainer repeat the investigation.
• Over-broad assertions: the tutorial claims pricing, quota, latency, billing, or uptime based on a single streaming transcript. Keep the transcript focused on request and event structure.

Reader next step

Open the tutorial example you plan to update and identify the exact request shape before running anything. If it uses chat completions, check the current chat reference, choose a model value from current model documentation or account-visible options, and run one small streaming fixture with stream: true. Save the ordered SSE transcript, the done-marker observation, and the pass/fail log fields listed above. Then compare your notes with Review CometAPI Request Bodies Before Tutorial Tests before changing public code samples.

If the example fails, do not rewrite the tutorial around a guessed fix. First separate the failure into one of three buckets: source mismatch, local parser issue, or account/runtime issue. A source mismatch means the tutorial uses fields or endpoint shapes that no longer match the documentation. A parser issue means the API returned a streamed response but the local code read the events incorrectly. An account/runtime issue means credentials, model access, or environment setup blocked the run. Each bucket leads to a different edit.

Use Chat Schema Notes for Tutorial Maintainers as the next comparison point. Keep Verify CometAPI Base URLs Before Publishing Tutorial Examples nearby for setup and permission checks.

FAQ

Why capture every streaming chunk?

Chunk-level capture shows whether the tutorial exercised streaming behavior instead of only saving the final assistant text. It also makes it easier to spot parser mistakes in examples that read SSE events.

Should the transcript include the actual API key?

No. Store credentials outside the transcript and use <API_KEY_PLACEHOLDER> in examples, fixtures, and notes.

Can this smoke test prove pricing, quotas, or uptime?

No. This workflow checks a tutorial fixture against documented request and streaming behavior. Pricing, quotas, account limits, and uptime require separate current evidence.

Can the generated answer text be used as the pass condition?

No. The safer pass condition is structural: the stream starts, ordered chunks are captured, the done marker appears, and the fixture uses fields verified in the current documentation.

What if the tutorial could use either Chat Completions or Responses?

Check both endpoint references, then keep the fixture aligned with one endpoint. Do not combine request fields, response fields, or streaming assumptions from different endpoint families unless the current documentation explicitly supports that combination.