Last reviewed: 2026-08-29

Direct answer

A CometAPI LlamaIndex document Q&A application needs three distinct pieces: a loader for local files, an embedding model that turns file chunks into searchable vectors, and a CometAPI-backed language model that writes an answer from the retrieved context. The LlamaIndex CometAPI integration reference documents the dedicated provider, while the current provider implementation shows that the CometAPI class uses LlamaIndex’s OpenAI-compatible abstraction.

In this tutorial, Markdown and text files stay in a local data directory. A local Hugging Face embedding integration builds the vector index. When a user asks a question, LlamaIndex retrieves relevant chunks and sends that context to the CometAPI language model for answer synthesis. Therefore, local documents do not mean a fully offline workflow: selected document text can leave the machine during the model call.

Install the components

Create an isolated Python environment and install the LlamaIndex core package, the dedicated CometAPI LLM package, and the local embedding integration:

python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install llama-index llama-index-llms-cometapi llama-index-embeddings-huggingface
mkdir -p data
export COMETAPI_API_KEY='[REDACTED]'

Replace [REDACTED] only in your local shell. Do not paste the real value into the application, a repository, command output, or an issue report. For a fuller repository boundary, read Keep CometAPI Keys Out of Tutorial Repositories .

Before a team shares this example, record the resolved dependency set after a successful run:

python -m pip freeze > requirements.lock.txt

This matters because the refetched guides show that LlamaIndex configuration patterns can differ across releases. Capturing the working environment makes a later import or constructor change easier to diagnose.

Add a small known-answer corpus

Create data/handbook.md with fictional, easy-to-check content:

# Returns policy

An unopened item may be returned within 30 calendar days of delivery.
A return requires the order number and the original packaging.
Opened clearance items are not returnable.

# Support hours

The support desk is staffed from 09:00 to 17:00 UTC, Monday through Friday.

Use synthetic text for the first run. Do not begin with customer records, legal files, confidential source code, or other sensitive material. A tiny corpus makes retrieval mistakes visible and gives you an exact expected answer.

Build the application

Save the following as app.py:

import json
import os
import time
from pathlib import Path

from llama_index.core import Settings, SimpleDirectoryReader, VectorStoreIndex
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.llms.cometapi import CometAPI

DATA_DIR = Path('data')
ALLOWED_SUFFIXES = {'.md', '.txt'}


def emit(event: str, **fields: object) -> None:
    record = {'event': event, **fields}
    print(json.dumps(record, sort_keys=True), flush=True)


def main() -> int:
    started = time.perf_counter()
    stage = 'configuration'
    model = os.getenv('COMETAPI_MODEL', 'gpt-4o-mini')
    embedding_model = os.getenv(
        'LOCAL_EMBEDDING_MODEL',
        'BAAI/bge-small-en-v1.5',
    )

    if not os.getenv('COMETAPI_API_KEY'):
        emit(
            'run_failed',
            status='error',
            stage=stage,
            error_type='MissingEnvironmentVariable',
        )
        return 2

    if not DATA_DIR.is_dir():
        emit(
            'run_failed',
            status='error',
            stage='loading',
            error_type='MissingDataDirectory',
        )
        return 2

    files = sorted(
        path
        for path in DATA_DIR.iterdir()
        if path.is_file() and path.suffix.lower() in ALLOWED_SUFFIXES
    )
    if not files:
        emit(
            'run_failed',
            status='error',
            stage='loading',
            error_type='NoSupportedDocuments',
        )
        return 2

    emit('run_started', status='ok', stage=stage, model=model)

    try:
        llm = CometAPI(
            model=model,
            max_tokens=512,
            context_window=4096,
            max_retries=3,
        )
        Settings.llm = llm
        Settings.embed_model = HuggingFaceEmbedding(
            model_name=embedding_model,
        )

        stage = 'loading'
        documents = SimpleDirectoryReader(
            input_files=[str(path) for path in files]
        ).load_data()
        emit(
            'documents_loaded',
            status='ok',
            stage=stage,
            document_count=len(documents),
        )

        stage = 'indexing'
        index = VectorStoreIndex.from_documents(
            documents,
            show_progress=False,
        )
        engine = index.as_query_engine(similarity_top_k=3)
        emit(
            'index_ready',
            status='ok',
            stage=stage,
            document_count=len(documents),
        )

        question = input('Question: ').strip()
        if not question:
            emit(
                'run_failed',
                status='error',
                stage='query',
                error_type='EmptyQuestion',
            )
            return 2

        stage = 'query'
        prompt = (
            'Answer from the indexed documents. '
            'If the documents do not contain the answer, say so. '
            f'Question: {question}'
        )
        response = engine.query(prompt)
        print('\nAnswer:\n' + str(response))
        emit(
            'query_completed',
            status='ok',
            stage=stage,
            model=model,
            duration_ms=round((time.perf_counter() - started) * 1000),
        )
        return 0
    except Exception as exc:
        emit(
            'run_failed',
            status='error',
            stage=stage,
            model=model,
            error_type=type(exc).__name__,
            duration_ms=round((time.perf_counter() - started) * 1000),
        )
        return 1


if __name__ == '__main__':
    raise SystemExit(main())

The explicit embedding configuration is important. The dedicated CometAPI integration shown in the refetched provider source is an LLM adapter; it does not, by itself, choose the embedding model used to build the vector index. Separating the two also makes it clear which operation is local and which operation calls CometAPI.

Run the happy path

Start the program and ask a question whose answer appears verbatim in the sample file:

python app.py

What is the return window for an unopened item?

A successful run emits run_started, documents_loaded, index_ready, and query_completed. The answer should say that the window is 30 calendar days after delivery. Exact phrasing can vary, so validate the facts rather than comparing the output character for character.

Then run a negative grounding check:

What telephone number should I call?

The corpus contains no telephone number. A well-grounded response should acknowledge that the documents do not supply one. If the model invents a number, treat the run as a retrieval or prompting failure even though the request technically succeeded.

Run the error path

Confirm that a missing local credential fails before document loading or a network request:

unset COMETAPI_API_KEY
python app.py

The program should exit with code 2 and emit a sanitized record like this:

{"error_type": "MissingEnvironmentVariable", "event": "run_failed", "stage": "configuration", "status": "error"}

Restore the value locally before another happy-path run:

export COMETAPI_API_KEY='[REDACTED]'

Also test an empty data directory. That path should return NoSupportedDocuments without calling the model. These two checks distinguish configuration and ingestion failures from provider failures.

The log allowlist is deliberately small: event, status, stage, model, document_count, duration_ms, and error_type. Do not log the environment value, question text, retrieved chunks, generated answer, complete exception message, or upstream response body. Print the answer for the interactive reader, but keep it out of structured operational logs.

Who this is for

This guide is for Python developers who want a minimal retrieval-augmented application over local Markdown or text files while using CometAPI for final answer generation. It assumes comfort with a virtual environment and a terminal, but it does not require a separate vector database.

It is not yet a production architecture for regulated documents, multi-user access, durable index storage, document-level permissions, or audited citations. Those concerns should be designed before real data is introduced.

Key takeaways

  • LlamaIndex loads and indexes the files, while the dedicated CometAPI provider supplies the language model used for answer synthesis.
  • The LLM adapter and embedding model are separate contracts. This example selects a local embedding integration explicitly.
  • Retrieved source text can be included in the remote model request, even though the original files and vector index are local.
  • Validate one known answer, one missing answer, and one deliberate configuration error before expanding the corpus.
  • Log only an allowlisted set of operational metadata. Never log document passages, prompts, answers, credential values, or raw provider errors.
  • Confirm a current model ID before deployment with the CometAPI model catalog validation workflow .

Sources checked

  • The LlamaIndex CometAPI integration reference identifies CometAPI as an available LLM integration and describes its unified interface.
  • The maintained provider implementation confirms the import path, environment lookups, default model, provider base, output controls, and retry option.
  • The LlamaIndex starter tutorial establishes the core RAG sequence: load documents, construct an index, retrieve context, and synthesize an answer.
  • The refetched CometAPI LlamaIndex guide provides a CometAPI-specific document-ingestion and vector-index example and notes that framework interfaces can differ across releases.
  • The Python package project page is the claimed public distribution page for the dedicated integration package; its refetched body presented a client challenge, so package metadata was not used as factual evidence here.

Contract details to verify

Check these boundaries again whenever dependencies or model selection changes:

  1. Package and import path. The dedicated package is llama-index-llms-cometapi, and the current source imports CometAPI from llama_index.llms.cometapi. An import failure usually means the integration package is absent from the active environment or the resolved packages are incompatible.
  2. Credential lookup. The provider implementation reads COMETAPI_API_KEY when no constructor value is supplied. This tutorial relies on that environment lookup and never stores a value in Python source.
  3. Provider base. The maintained class already defines its default API base. Leave COMETAPI_API_BASE unset unless current provider documentation requires an override. A console page or general website address is not an inference endpoint.
  4. Model availability. The current source uses gpt-4o-mini as its default, but a model identifier is an operational dependency, not a permanent guarantee. Verify availability and capability before each release.
  5. Context and output limits. context_window and max_tokens are accepted provider settings. They must remain compatible with the selected model and the amount of retrieved context. Do not assume that a client-side number expands a model’s actual limit.
  6. Retries. The provider exposes max_retries. This sample uses three attempts, but retries should be bounded and observed; they do not repair invalid credentials, unknown models, or malformed requests.
  7. Embedding contract. Vector retrieval requires embeddings independently of the CometAPI LLM. Record the local embedding model, its package version, and whether its assets are already available in the runtime.
  8. Framework configuration. This example uses Settings, the current global configuration pattern. If a copied example uses ServiceContext, consult the current LlamaIndex migration guidance rather than mixing both patterns.

Failure modes

The integration package is installed in a different interpreter. ModuleNotFoundError can occur when pip and python refer to different environments. Run installation through python -m pip, confirm the virtual environment is active, and inspect the resolved package list.

The credential is missing or rejected. The local guard detects only absence. A present but invalid value fails during the provider call. Record the stage and exception class, then verify the local environment through the appropriate account workflow. Do not print the value or raw request metadata.

The model identifier is unavailable or incompatible. A stale or unsupported model name can produce a provider error. Recheck the catalog rather than silently switching models. A fallback can change cost, latency, context capacity, and answer behavior.

The data directory is empty or uses unsupported file types. This example accepts only .md and .txt. The explicit allowlist prevents the operator from believing a file was indexed when no supported document was loaded. Add other readers only after testing their parser dependencies and extraction quality.

The local embedding model cannot initialize. A fresh environment may not have the selected model assets available. Restricted egress, an empty local cache, limited memory, or incompatible dependencies can stop indexing before any CometAPI call. The stage field distinguishes this from a query failure.

Dependency interfaces have drifted. Imports, constructor parameters, and query-engine configuration can move between LlamaIndex releases. Rebuild from the recorded dependency set first. If you intentionally upgrade, follow the documentation drift checklist and rerun all positive and negative checks.

Retrieval returns the wrong chunks. The model can produce a fluent but irrelevant answer when chunking or similarity search misses the right passage. Use a tiny known-answer corpus first, inspect retrieval in a protected development environment, and then tune chunking and retrieval depth. Do not add retrieved text to routine logs.

The model answers beyond the evidence. The prompt asks the model to admit when the answer is absent, but that instruction is not a guarantee. Keep the missing-answer check in the acceptance set and add citations or an explicit abstention policy before relying on high-impact answers.

Requests time out, throttle, or fail upstream. Bounded retries may help transient failures, but repeated retries can increase latency and load. Add a client timeout and a clear failure response before placing the query engine behind a service. The client timeout tutorial covers that boundary.

Sensitive text crosses the service boundary. Retrieval sends selected context to the configured LLM. Classify the corpus, minimize retrieved content, and obtain the necessary approval before indexing confidential material. A local vector store alone does not make remote synthesis local.

Diagnostic output leaks content. Exception messages and upstream bodies may contain request details. Keep the structured log allowlist, route interactive answers separately, and grant log access as if it could expose application metadata.

FAQ

Does local document Q&A run completely offline?

No. Loading and embedding are configured locally in this example, but answer synthesis calls the CometAPI LLM. The retrieved context needed for that answer may be sent with the request.

Does the dedicated CometAPI provider also create the vector embeddings?

Not in this design. The refetched implementation is an LLM provider. The tutorial configures HuggingFaceEmbedding separately so vector construction does not depend on an implicit default embedding service.

Why does the code omit an API base setting?

The maintained provider class already supplies its default. Omitting an override reduces the chance of substituting a console or website address for the inference base. Verify the current source before overriding it.

Can I switch to another CometAPI model?

Yes. Set COMETAPI_MODEL to a currently supported identifier, then rerun the known-answer, missing-answer, and error-path checks. Do not treat model substitution as behaviorally neutral.

Can this read PDFs and office documents?

LlamaIndex documents a wider connector ecosystem, but this minimal application intentionally accepts Markdown and plain text. Additional formats may require parser packages and new validation cases. Confirm extracted text before trusting retrieval.

Can answers stream to the terminal?

The provider documentation and implementation support LLM streaming interfaces. Query-engine streaming configuration can depend on the installed LlamaIndex release, so add it only after the non-streaming path passes and verify the current interface.

Is the index durable?

No. This script builds an in-memory index each time it starts. Persistence, refresh behavior, deletion, and document versioning are separate production decisions.

Is a successful request proof that the answer is grounded?

No. Transport success proves only that the workflow completed. Grounding requires evidence-based checks against the corpus, including an answerable question and a question the documents cannot answer.

Reader next step

Run the example against two short synthetic files and create a three-question acceptance set: one direct fact, one fact that requires combining two passages, and one fact absent from the corpus. Save the resolved dependency list and only the sanitized event records. If all three behaviors are acceptable, verify the current model contract, add a timeout, and decide how document permissions and index persistence should work before introducing real data.