Last reviewed: 2026-08-08

Direct answer

Put LiteLLM between your application and CometAPI. Your application talks to one local OpenAI-compatible endpoint and sends a stable alias such as comet-text. LiteLLM resolves that alias from model_list, selects the upstream model from litellm_params, and forwards the request using the CometAPI base endpoint and an environment-backed credential. The CometAPI Quickstart documents the OpenAI SDK and base-URL pattern that makes this arrangement possible. The LiteLLM AI Gateway guide describes the proxy as a unified OpenAI-format interface with spend tracking and budgets per virtual key or user.

This design separates three contracts. The client-to-proxy contract is the endpoint and alias your application knows. The alias-to-deployment contract maps that public name to a provider and model. The proxy-to-CometAPI contract contains the upstream endpoint and credential. Keeping those contracts separate means a model change can stay in configuration instead of forcing every client to ship a new model string.

Create a small configuration file first:

model_list:
  - model_name: comet-text
    litellm_params:
      model: openai/COMET_MODEL_ID
      api_base: os.environ/BASE
      api_key: os.environ/KEY
      max_tokens: 512

general_settings:
  # Optional: require a proxy key for every client call.
  master_key: os.environ/MKEY

The LiteLLM config.yaml reference explains the fields used here. model_name is the name that an external client sends. litellm_params.model is the upstream model string. The openai/ prefix tells LiteLLM to use its OpenAI-compatible provider path, while api_base and api_key point the deployment at CometAPI. The os.environ form keeps values out of the file. Set BASE, KEY, and MKEY through the process environment or a secret manager; do not commit their values.

Start the proxy with the documented command:

litellm --config ./config.yaml

For a first check, use the same Python client shape that CometAPI documents, but point it at the proxy origin. The local origin is supplied by your environment so the example does not embed a deployment URL or a credential.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ['CLIENT_KEY'],
    base_url=os.environ['LITELLM_PROXY_URL'],
)
response = client.chat.completions.create(
    model='comet-text',
    messages=[{'role': 'user', 'content': 'Reply with one word: ready'}],
)
print(response.choices[0].message.content)

A happy path has four observable checkpoints: the proxy starts without a configuration error, the request uses the alias comet-text, LiteLLM resolves that alias to the configured CometAPI deployment, and the response contains the expected chat completion fields. If MKEY or a virtual-key policy is enabled, CLIENT_KEY must be injected by the deployment environment. The code intentionally uses a redacted runtime value rather than showing one in a tutorial.

Who this is for

This pattern is for teams that have more than one CometAPI client, want to rename models without editing every caller, or need one place to apply access and spend controls. It is also useful when a small service already speaks the OpenAI client format and should move behind a local gateway with minimal application changes.

It is less useful for a one-off script that will never share a model or credential boundary. A direct CometAPI client is simpler in that case. LiteLLM earns its place when the boundary itself is valuable: aliases give clients a stable vocabulary, and the proxy gives operators a single configuration and policy surface.

Key takeaways

  • Treat comet-text as a public interface owned by your application, not as the upstream model identifier.
  • Keep the upstream endpoint and credential in environment-backed LiteLLM parameters.
  • Preserve the openai/ provider marker when the upstream speaks the OpenAI-compatible contract.
  • Make proxy authentication and virtual-key policy explicit. A proxy key is separate from the upstream CometAPI credential.
  • Test the alias through the proxy before changing application code or adding another deployment.
  • Log routing metadata and outcomes, never prompts, credentials, or complete provider responses. For repository hygiene, see keep CometAPI keys out of tutorial repositories .

Sources checked

  • CometAPI Quickstart verifies the OpenAI-compatible base-URL pattern, SDK compatibility, and the need to keep the API key in environment variables.
  • LiteLLM AI Gateway (LLM Proxy) verifies the unified OpenAI-format gateway model and its spend, budget, and virtual-key scope.
  • LiteLLM Config.yaml Overview verifies model_list, model_name aliases, litellm_params, api_base, api_key, and the config-driven startup command.
  • OpenAI Python library verifies the client pattern used for OpenAI-compatible Python requests and the supported chat-completion interface.

These sources were refetched for this article. The configuration intentionally uses placeholders and environment references so the tutorial can be copied without exposing a credential.

Contract details to verify

Before connecting a real client, verify each of these values against the current documentation and your deployment:

  1. Base endpoint. Set BASE to the exact CometAPI v1 base endpoint shown in the quickstart. Do not append a second version segment or mix a dashboard origin with an API origin.
  2. Upstream model ID. Replace COMET_MODEL_ID with a model identifier that is currently listed for the operation you plan to call. The alias can remain comet-text even when the upstream model changes.
  3. Provider marker. Keep openai/ in litellm_params.model for an OpenAI-compatible upstream. Removing it can send the request through the wrong provider adapter.
  4. Alias spelling. The client model value must exactly equal model_name. A friendly alias is useful only when it is treated as a stable contract.
  5. Authentication mode. If MKEY or virtual-key controls are enabled, confirm how the proxy expects the client credential and inject it at runtime. Never put it in a URL query, a committed config file, or a log line.
  6. Endpoint family and fields. This example uses Chat Completions. Check the selected CometAPI model and the current LiteLLM endpoint documentation before switching to another operation or adding provider-specific fields.
  7. Request headers. Confirm the headers added by the client and proxy agree with the deployed policy; the companion guide to review CometAPI request headers is a useful checklist.

A small contract table kept next to config.yaml can record alias, upstream model ID, endpoint family, owner, and last verification date. Do not store the credential itself in that table.

Failure modes

The proxy starts, but the client reports an unknown model. The request probably used the upstream model ID instead of model_name, or the alias differs by punctuation or case. Send comet-text exactly, then inspect the loaded model list without printing secret fields.

LiteLLM selects the wrong provider. The common configuration mistake is dropping the openai/ marker from the upstream model string. Restore it, restart the proxy, and repeat the one-word test. Keep the alias unchanged so callers do not need a simultaneous code edit.

The proxy cannot reach CometAPI. Check BASE for the documented v1 endpoint and check that the process actually received the environment values. A configuration file can parse successfully while an empty environment reference still produces a connection or authentication failure. Log only the endpoint hostname, not the credential or full request.

Authentication fails at the proxy boundary. When master-key or virtual-key enforcement is active, a missing or invalid client credential is different from an upstream CometAPI failure. Confirm the policy, inject CLIENT_KEY through the runtime secret mechanism, and retry with the same alias. Do not paste the rejected value into an issue or transcript.

The alias resolves, but the request is rejected for capability or field reasons. A model can be present in the catalog without supporting every operation or parameter. Reduce the request to the documented chat fields, then verify the model’s supported operation before adding optional settings.

A virtual-key request is denied by policy. LiteLLM can apply budgets and model access at the proxy layer. Identify the key or user by an internal non-secret identifier, check the configured budget and model permission, and avoid changing the CometAPI upstream credential as a first response.

A request times out or is rate-limited. Record the layer that produced the response, the status, latency, alias, and a retry count. Retry only according to your service’s idempotency and backoff policy. Do not turn a transient transport error into an alias or credential change without evidence.

Use this sanitized event shape for logs:

{
  "request_id": "req_[REDACTED]",
  "model_alias": "comet-text",
  "upstream_model": "COMET_MODEL_ID",
  "endpoint_family": "chat_completions",
  "status": 200,
  "latency_ms": 0,
  "input_tokens": 0,
  "output_tokens": 0,
  "error_type": null
}

The zeros are placeholders, not expected production values. Keep prompts, completion text, authorization values, and raw upstream bodies out of this event. That gives an operator enough routing evidence to distinguish alias, proxy, and upstream failures without turning logs into a second credential store.

FAQ

Why use LiteLLM when CometAPI already accepts the OpenAI SDK? The direct SDK path is ideal for a single caller. LiteLLM adds a shared boundary: clients use aliases, while operators manage provider settings, access, budgets, and migration decisions in one place.

Do I have to expose the CometAPI credential to every application? No. Put the upstream value in the proxy environment and give applications only the proxy credential required by your policy. The application should know the alias and proxy origin, not the upstream secret.

Can I change the upstream model without changing the client? Yes, when the alias is kept stable. Change litellm_params.model, verify the new model contract, restart or reload according to your deployment process, and rerun the same alias test.

Why does the example use Chat Completions? It is a compact compatibility check and is shown in both the CometAPI quickstart and the OpenAI Python library. Verify the operation and fields for your chosen model before adapting the request.

What should I do with a failed test response? Preserve the alias, status, latency, and sanitized error type. Then classify the failure as client contract, proxy configuration, proxy policy, or upstream transport. That classification is more useful than copying a full request into a ticket.

Reader next step

Create a disposable config.yaml with the alias and environment references above, select one currently documented CometAPI model, and start LiteLLM with that file. Run the one-word test through the proxy, confirm the alias appears in your sanitized event, and deliberately test one rejected request so the team knows which layer owns the error. Once the happy and error paths are clear, wire your application to the proxy base URL and keep the alias as its only model dependency. Recheck the source links whenever you change the upstream model or proxy policy.