Nothing lied to you. "OpenAI-compatible" was never a boolean. It's a surface area, and every implementation covers a different subset of it.

I maintain a gateway, so I read a lot of compatibility bug reports. Disclosure: I work on daoxe, a multi-model gateway that speaks the OpenAI protocol among others. The checklist below is deliberately written so you can run it against us and against anyone else, and the script at the end doesn't know or care which endpoint you point it at. If it makes us look bad on a check, that's the correct output.

The nine surfaces

1. Streaming deltas. The reference behaviour is specific: the first chat.completion.chunk carries choices[0].delta.role = "assistant" and usually empty content, middle chunks carry delta.content fragments, the last content-bearing chunk carries finish_reason, and the stream terminates with a literal data: [DONE] line. Implementations diverge on all four. Some never send the role chunk, which breaks clients that use it to open a message. Some put finish_reason on a trailing chunk with an empty delta. Some omit [DONE] entirely and just close the connection, which is fine for a client that reads to EOF and fatal for one that blocks waiting for the sentinel.

2. Tool / function calling. Two traps here. First, function.arguments is a JSON-encoded string, not an object — endpoints that "helpfully" return a parsed object break every client that calls json.loads on it. Second, in streaming mode, tool calls arrive as fragments that you reassemble by the index field, with id and function.name typically only present on the first fragment. An endpoint that re-sends id on every fragment, or that omits index when there's only one call, will work with your naive accumulator and fail the moment a model emits two parallel calls. Also check that finish_reason is tool_calls and not stop — agent loops branch on that value.

3. response_format. Three tiers, and they're commonly conflated: no support, {"type": "json_object"} (valid JSON, any shape), and {"type": "json_schema", "json_schema": {..., "strict": true}} (constrained decoding against your schema). The dangerous middle case is an endpoint that accepts the parameter and ignores it. You get prose with a code fence around it, your parser fails one request in fifty, and it looks like a model quality problem.

4. logprobs / top_logprobs. Usually the first thing a proxy layer drops, because almost nobody notices. If you do classification by comparing token probabilities, or you use logprobs for confidence gating, this is load-bearing and you should test it explicitly.

5. temperature and sampling params. Reasoning-style models reject temperature outright on some upstreams, accept-and-ignore it on others, and honour it on a third set. All three are defensible; not knowing which one you're on is not. The same applies to top_p, presence_penalty, and max_tokens vs max_completion_tokens.

6. stop sequences. Does the endpoint honour an array of stop strings? Is the stop sequence included in or excluded from the returned content (the reference excludes it)? Does finish_reason come back as "stop"? A surprising number of shims implement stop by post-truncating the full completion, which means you pay for tokens you never see — and the usage numbers will show it.

7. Usage accounting. Non-streaming responses should carry usage.prompt_tokens, completion_tokens, total_tokens. Streaming responses only include usage if you pass stream_options: {"include_usage": true}, and then it arrives in a final chunk with an empty choices array — a shape that crashes clients which assume choices[0] always exists. If you do cost attribution per request, also check whether cached-prompt and reasoning-token breakdowns survive.

8. Error-body shape. Every retry layer you've ever written parses this, and nobody tests it. The reference is {"error": {"message": ..., "type": ..., "param": ..., "code": ...}} with an HTTP status that matches the semantics. Real-world variations: a 200 with an error object in the body (fatal for retry logic — you'll cheerfully return an error string to your user), an HTML error page from an intermediate proxy, or a 500 where a 400 belonged, which turns a permanent client error into an infinite retry storm.

9. /v1/models fidelity. Does it exist, does it return {"object": "list", "data": [{"id": ...}]}, and — the part that matters — does it list the IDs your key can actually call? A catalogue endpoint that returns everything the vendor sells, rather than everything your key is entitled to, is worse than no catalogue at all, because you'll build CI checks on top of it.

The script

Stdlib only, Python 3.8+, no install. It runs eleven checks and prints a verdict table.

#!/usr/bin/env python3
"""compat_probe.py — how OpenAI-compatible is this endpoint, really?

    export LLM_BASE_URL=https://api.example.com/v1
    export LLM_API_KEY=sk-...
    python3 compat_probe.py --model <exact-model-id>
"""
import argparse, json, os, sys, urllib.error, urllib.request

BASE = os.environ.get("LLM_BASE_URL", "").rstrip("/")
KEY = os.environ.get("LLM_API_KEY", "")
OUT = []


def _open(path, payload=None, method="GET", timeout=90):
    data = json.dumps(payload).encode() if payload is not None else None
    req = urllib.request.Request(BASE + path, data=data, method=method)
    req.add_header("Authorization", "Bearer " + KEY)
    if data is not None:
        req.add_header("Content-Type", "application/json")
    try:
        return urllib.request.urlopen(req, timeout=timeout)
    except urllib.error.HTTPError as exc:   # still a readable file object
        return exc


def call(path, payload=None, method="GET"):
    resp = _open(path, payload, method)
    raw = resp.read().decode("utf-8", "replace")
    try:
        return resp.getcode(), json.loads(raw)
    except ValueError:
        return resp.getcode(), raw


def stream(payload):
    payload = dict(payload, stream=True)
    resp = _open("/chat/completions", payload, "POST")
    chunks, saw_done = [], False
    for line in resp:
        line = line.decode("utf-8", "replace").strip()
        if not line.startswith("data:"):
            continue
        body = line[5:].strip()
        if body == "[DONE]":
            saw_done = True
            break
        try:
            chunks.append(json.loads(body))
        except ValueError:
            pass
    return chunks, saw_done


def record(name, ok, detail):
    OUT.append((name, "PASS" if ok is True else "FAIL" if ok is False else "PARTIAL", detail))


def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--model", required=True)
    model = ap.parse_args().model
    ask = {"model": model, "messages": [{"role": "user", "content": "Say hi."}],
           "max_tokens": 24}

    # 1 — catalogue
    code, body = call("/models")
    ids = [m.get("id") for m in body.get("data", [])] if isinstance(body, dict) else []
    record("models_endpoint", code == 200 and bool(ids),
           f"HTTP {code}, {len(ids)} ids, target listed: {model in ids}")

    # 2 — basic completion + model echo + usage
    code, body = call("/chat/completions", ask, "POST")
    msg = (body.get("choices") or [{}])[0].get("message", {}) if isinstance(body, dict) else {}
    usage = body.get("usage") if isinstance(body, dict) else None
    record("basic_chat", bool(msg.get("content")), f"HTTP {code}")
    record("model_echo", isinstance(body, dict) and body.get("model") == model,
           f"asked {model!r}, got {body.get('model')!r}" if isinstance(body, dict) else "n/a")
    record("usage_fields", bool(usage and usage.get("total_tokens") is not None), str(usage))

    # 3 — streaming shape
    chunks, done = stream(ask)
    role = any((c.get("choices") or [{}])[0].get("delta", {}).get("role") for c in chunks)
    fin = any((c.get("choices") or [{}])[0].get("finish_reason") for c in chunks)
    record("stream_shape", bool(chunks) and role and fin and done,
           f"{len(chunks)} chunks, role_chunk={role}, finish_reason={fin}, [DONE]={done}")

    # 4 — usage on stream
    chunks, _ = stream(dict(ask, stream_options={"include_usage": True}))
    record("stream_usage", any(c.get("usage") for c in chunks),
           "final usage chunk present" if any(c.get("usage") for c in chunks) else "absent")

    # 5 — tool calling
    tool = {"type": "function", "function": {"name": "get_weather", "parameters": {
        "type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}}
    code, body = call("/chat/completions", dict(
        ask, messages=[{"role": "user", "content": "Weather in Osaka? Use the tool."}],
        tools=[tool], tool_choice="auto"), "POST")
    choice = (body.get("choices") or [{}])[0] if isinstance(body, dict) else {}
    calls = choice.get("message", {}).get("tool_calls") or []
    args_is_str = bool(calls) and isinstance(calls[0].get("function", {}).get("arguments"), str)
    record("tool_calls", bool(calls) and args_is_str and choice.get("finish_reason") == "tool_calls",
           f"n={len(calls)} arguments_is_string={args_is_str} finish={choice.get('finish_reason')}")

    # 6 — json mode
    code, body = call("/chat/completions", dict(
        ask, messages=[{"role": "user", "content": "Return {\"ok\": true} as JSON."}],
        response_format={"type": "json_object"}), "POST")
    text = ((body.get("choices") or [{}])[0].get("message", {}).get("content")
            if isinstance(body, dict) else "") or ""
    try:
        json.loads(text); parsed = True
    except ValueError:
        parsed = False
    record("json_object", code == 200 and parsed, f"HTTP {code}, parses={parsed}")

    # 7 — strict schema
    schema = {"type": "json_schema", "json_schema": {"name": "r", "strict": True, "schema": {
        "type": "object", "properties": {"ok": {"type": "boolean"}},
        "required": ["ok"], "additionalProperties": False}}}
    code, _ = call("/chat/completions", dict(ask, response_format=schema), "POST")
    record("json_schema_strict", code == 200, f"HTTP {code}")

    # 8 — logprobs
    code, body = call("/chat/completions", dict(ask, logprobs=True, top_logprobs=3), "POST")
    lp = ((body.get("choices") or [{}])[0].get("logprobs") if isinstance(body, dict) else None)
    record("logprobs", bool(lp and lp.get("content")), f"HTTP {code}")

    # 9 — temperature: accepted / rejected / ignored is three different worlds
    code, _ = call("/chat/completions", dict(ask, temperature=0.5), "POST")
    record("temperature", None if code == 400 else code == 200,
           "rejected with 400 (reasoning model?)" if code == 400 else f"HTTP {code}")

    # 10 — stop sequences
    code, body = call("/chat/completions", dict(
        ask, messages=[{"role": "user", "content": "Count: one two three four five"}],
        stop=["three"], max_tokens=48), "POST")
    text = ((body.get("choices") or [{}])[0].get("message", {}).get("content")
            if isinstance(body, dict) else "") or ""
    record("stop_sequences", "three" not in text, f"stop string leaked into content: {'three' in text}")

    # 11 — error shape on a model that cannot exist
    code, body = call("/chat/completions", dict(ask, model="definitely-not-a-model-xyz"), "POST")
    shaped = isinstance(body, dict) and isinstance(body.get("error"), dict)
    record("error_shape", 400 <= code < 500 and shaped, f"HTTP {code}, error object: {shaped}")

    width = max(len(n) for n, _, _ in OUT)
    for name, verdict, detail in OUT:
        print(f"{name.ljust(width)}  {verdict:<7} {detail}")
    print("\nPARTIAL is not a failure — it is a behaviour you now have to design around.")
    return 0 if all(v != "FAIL" for _, v, _ in OUT) else 1


if __name__ == "__main__":
    sys.exit(main())

Two notes on running it. It costs a handful of tiny completions, so run it against a cheap model first to confirm your base URL and key are right. And run it against the exact model ID you plan to ship, because conformance varies per model on the same endpoint — reasoning models in particular reject parameters that their non-reasoning siblings accept.

Reading the output

FAIL on basic_chat or error_shape is a real problem. FAIL on logprobs is only a problem if you use logprobs. The point of the table isn't a score; it's that you now have a written record of which surfaces you're allowed to depend on, which you can re-run after any provider change and diff.

Three habits that follow from it:

• Check it into CI. One nightly run, output committed as an artifact. Compatibility regressions are silent by nature — nobody sends a changelog entry saying "we stopped forwarding logprobs".

• Assert the model echo in production, not just in the probe. If you asked for one ID and the response's model field says another, you want that in your logs the day it starts happening, not the week you notice quality dropped.

• Treat PARTIAL as a design input. If stream_usage is absent, your cost attribution needs a non-streaming path or a local tokenizer estimate. That's a two-hour decision now or a month-end reconciliation mystery later.

What this can't tell you

The probe tests the protocol, not the model behind it. An endpoint can pass all eleven checks and still be routing you to a smaller model, a quantized build, or a truncated context window — the wire format would be perfect either way. That's a different investigation with different tools, and the honest framing there is that behavioural probes give you signals, not proof: nobody outside the provider can see which weights served your request.

But protocol conformance is the part you can settle in an afternoon, with a script that fits on one screen, and it's the part that will wake you up at 3am if you skip it.

Руководство на русском: поднять собственный чат-клиент за пять команд и подключить к нему несколько LLM через один endpoint. Дисклеймер: мы ведём DaoXE, к которому ведёт часть конфигурации, — но сам клиент вы держите у себя, и это как раз цель статьи.

Почему «свой клиент» ≠ «полностью локальные модели»

Честно разделим два разных желания:

• Локальные модели (Ollama, GGUF) — данные и вычисления не покидают вашу машину. Ценой: качество ниже фронтирных моделей, нужны GPU, а «приватность» достигается только тем, что запрос никуда не идёт.

• Свой клиент + внешний API — интерфейс, история чатов и ключи у вас; генерация идёт к провайдеру модели. Приватность — это контроль над тем, куда и что уходит, а не «ничего никуда не уходит».

Эта статья про второй случай: поднять LobeChat в Docker и направить его на единый OpenAI-совместимый endpoint. Смысл этого пути — не держать десять разных клиентов и одиннадцать аккаунтов, сохранив при этом контроль над собственной частью стека.

Шаг 1. Поднять LobeChat в Docker

# docker-compose.yml
services:
  lobechat:
    image: lobehub/lobe-chat
    ports: ["3210:3210"]
    restart: unless-stopped
    environment:
      - NEXT_AUTH_SSO_PROVIDERS=

docker compose up -d

Откройте http://localhost:3210. Клиент локальный; ключи хранятся в вашем браузере/базе, а не у третьих сторон.

Шаг 2. Подключить несколько моделей через один ключ

В настройках провайдера LobeChat:

• OpenAI: Base URL https://api.daoxe.com/v1, ваш ключ → модели из GET /v1/models.

• Anthropic: отдельно можно прописать нативный /v1/messages для Claude.

Один ключ, один баланс, много моделей — без отдельной подписки под каждую.

Шаг 3. Что видно в логах (и зачем это нужно)

Главное преимущество своего клиента — наблюдаемость. Включите логирование запросов и смотрите:

• model в ответе — та ли модель реально ответила (см. нашу статью про верификацию подмены);

• usage — сколько токенов ушло; по нему сверяете биллинг endpoint'а. Расхождение больше, чем объясняет тариф, — это вопрос к поставщику;

• куда уходит запрос — URL вы задаёте сами, и видите его же в логах.

# быстрый smoke-тест через ваш клиент-эндпоинт
curl -s https://api.daoxe.com/v1/chat/completions \
  -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"model":"ТОЧНЫЙ_ID","max_tokens":16,"messages":[{"role":"user","content":"ping"}]}' \
  | jq '{model, usage}'

Шаг 4. Модельный выбор: как не ошибиться

• Нужен фронтир (сложный код, длинные контексты) → топовые Claude/GPT через endpoint.

• Дешёвые массовые задачи (классификация, черновики) → открытые модели тем же ключом.

• Чувствительные данные → держите отдельный локальный Ollama для того, что не должно уходить наружу, и не путайте его с облачным путём.

Гибрид «чувствительное — локально, остальное — облако через один ключ» — самый частый рабочий паттерн.

Цены и оплата

Способы оплаты и актуальные цены по каждой модели — на daoxe.com/pricing; тарификация по моделям — в USD.

Дальше

Гид по десяткам клиентов: seven7763.github.io/daoxe-guide/ru. Вопросы — Telegram @daoxe_ai.

Мы не обещаем «100% стабильности» и не утверждаем, что умеем доказывать происхождение токена. Мы утверждаем меньше: всё перечисленное выше можно проверить у себя в логах.

Полное руководство на русском: как свести Claude, GPT, Gemini и DeepSeek к одному API-ключу и настроить три популярных клиента за пять минут. Дисклеймер: мы ведём сервис DaoXE, о котором идёт речь, — все шаги ниже проверяемы и работают с любым OpenAI-совместимым endpoint'ом.

Зачем это нужно

Типичный набор разработчика в 2026-м: подписка на ChatGPT, доступ к Claude, Gemini в отдельном аккаунте, плюс пара открытых моделей для дешёвых задач. Итог — несколько аккаунтов, несколько биллингов, и каждый раз при смене модели нужно менять клиент или ключ.

Единый OpenAI-совместимый шлюз решает это одной точкой входа: один base_url, один ключ, один баланс — а моделей много. Разберём на трёх популярных клиентах.

Три протокола, которые должны быть у шлюза

1. OpenAI Chat Completions (/v1/chat/completions) — говорит почти весь существующий инструментарий.

2. OpenAI Responses (/v1/responses) — новый формат OpenAI, его начинают ждать клиенты.

3. Anthropic Messages (/v1/messages) — нативный протокол Claude. Без него Claude Code подключается через OpenAI-обёртку, и часть фич (tool use, cache_control, thinking) ведёт себя иначе.

Проверить наличие третьего просто: прямой запрос к /v1/messages с заголовком x-api-key должен отвечать, а не возвращать 404.

Шаг 1. Ключ и список моделей

export DAOXE_API_KEY="ваш_ключ"
curl -s https://api.daoxe.com/v1/models \
  -H "Authorization: Bearer ${DAOXE_API_KEY}" | jq -r '.data[].id' | sort

Список аккаунт-скоупд: видите именно то, что доступно вам. Берите точные ID — «примерное название» не работает ни у кого.

Шаг 2. Минимальный запрос

curl -s https://api.daoxe.com/v1/chat/completions \
  -H "Authorization: Bearer ${DAOXE_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"model":"ТОЧНЫЙ_ID_МОДЕЛИ","max_tokens":16,
       "messages":[{"role":"user","content":"ping"}]}'

Ответ содержит поле usage — по нему сверяете биллинг (об этом ниже).

Шаг 3. Cursor

Settings → Models → включить OpenAI API Key:

• Override OpenAI Base URL: https://api.daoxe.com/v1

• API Key: ваш ключ

• Добавить точные ID моделей в список.

Шаг 4. Cline / Roo Code

Провайдер OpenAI Compatible, Base URL https://api.daoxe.com/v1, ключ + точный ID модели.

Шаг 5. Claude Code — нативно

export ANTHROPIC_BASE_URL="https://api.daoxe.com"   # корень хоста, клиент сам добавит /v1/messages
export ANTHROPIC_AUTH_TOKEN="ваш_ключ"

После этого Claude Code работает штатно: tool use, стриминг, длинные сессии — без прослойки-роутера.

Частые грабли

• Неверный ID модели — самая частая причина 404. Сверяйтесь с GET /v1/models, а не с чужими статьями: списки меняются.

• Перепутаны endpoint'ы: /v1/chat/completions и /v1/messages — разные протоколы, ключ один.

• Таймауты на длинных задачах — для них есть отдельная прямая линия jp.daoxe.com; рекомендуемый основной endpoint — api.daoxe.com (см. также daoxe.com/v1).

• Биллинг: сверяйте списания с полем usage в ответах. Расхождение больше, чем объясняет тариф, — вопрос к поставщику, и это правильный вопрос.

Оплата и цены

Способы оплаты и актуальные цены по каждой модели — на daoxe.com/pricing. Каждая модель тарифицируется по своей цене в USD; пополнение баланса — несколькими способами, включая USDT.

Дальше

Подробный гид по подключению (включая LobeChat, Cherry Studio, OpenCat и ещё десятки клиентов): seven7763.github.io/daoxe-guide/ru. Вопросы — в Telegram @daoxe_ai.

Не верьте на слово: три шага выше — регистрация, GET /v1/models, один минимальный запрос — проверяемы самостоятельно.

Follow-up to a Bluesky thread on verifying LLM output. Disclosure up front: I run an OpenAI-compatible gateway (DaoXE), so "did the model actually do what it claims" is a question I have to answer for customers weekly. The method below is provider-agnostic — run it against any endpoint, including ours.

The failure a smart reader pointed out

A common guard against silent model degradation is a restatement probe: after a model produces an answer, ask it to restate the key numbers/claims, and diff the restatement against the original output. If they drift, the leg failed.

The objection, which is correct: if the model misread the source in the first place, the output and the restatement will agree — on the wrong value. Restatement checks internal consistency. It cannot detect a consistent confabulation, because both passes reconstruct from the same prose the model already misunderstood.

So restatement catches drift (the model contradicts itself) but not misreading (the model is confidently, consistently wrong). Those are different bugs and they need different guards.

The stronger check: verify against the source, not the echo

Instead of asking the model to restate its own conclusion, require it to cite the exact span in the source that supports each number/claim, then validate the citation mechanically:

1. Ask for structured output: each extracted fact carries a source_quote field copied verbatim from the input.

2. Programmatically check that source_quote is a substring of the input (normalized for whitespace).

3. Optionally check the extracted value appears inside the cited span.

If the model misread, step 2 or 3 fails — because the check is anchored to the source text, not to the model's own second opinion. A hallucinated number can't produce a real substring of the input that contains it.

def verify_citations(facts, source):
    norm = " ".join(source.split())
    bad = []
    for f in facts:
        q = " ".join(f["source_quote"].split())
        if q not in norm:
            bad.append((f["claim"], "quote not in source"))
        elif f.get("value") and f["value"] not in q:
            bad.append((f["claim"], "value not in cited span"))
    return bad  # empty == every claim is anchored to real source text

This is still not proof the model understood the span — it proves the model isn't inventing the number out of thin air. That's a weaker, honest claim, and it's the kind worth making: a signal you can reproduce, not a guarantee you have to trust.

Why this matters for gateway selection specifically

When you route one workload across several models (or several upstreams behind one key), the failure mode that bites isn't the crash — it's the leg that answers fluently and wrongly. A citation check like the above is cheap to run per-call and turns "is this gateway's cheap model actually the model I asked for" into a measurable question with a reproducible answer, instead of a vibe.

Run it against your current provider. If the resolved model id in the response and the citation pass-rate both look wrong, you have a specific thing to ask them about — not a vague suspicion.

If you've built a better version of this (semantic entailment against the span, not just substring), I'd genuinely like to see it — the thread that prompted this is on Bluesky @daoxe.bsky.social.

• The bug that isn't a bug: your guessed model ID worked, so you shipped it.

• Three failure modes of a wrong ID, ranked by how much they'll cost you.

• The ten-second test for whether an endpoint guesses on your behalf.

• Dated vs floating IDs: two different risks, one policy.

• GET /v1/models is the only source of truth, and it's per-key.

• models.lock.json and a CI check that fails when a required ID vanishes.

• The alias layer: roles, not model names, everywhere in your code.

• A lint that stops model IDs leaking back into the codebase.

• Retirement playbook: six steps from "it's gone" to "merged".

• Channel variants on gateways: read the label, never pattern-match.

• Behavioural drift under a pinned ID — signals, not proof.

Draft

You typed claude-sonnet-4-5 from memory, it returned a completion, and you shipped. Six weeks later the outputs are subtly different, or a deploy fails with a 404 on a Tuesday morning, and the git blame points at a string literal in a file nobody has opened since.

Model IDs are a runtime dependency on someone else's release schedule, and almost nobody treats them like one. We pin requests==2.32.3 and then interpolate a model name from a config file we last checked in March.

Disclosure: I work on daoxe, a gateway that fronts hundreds of models from roughly 25 vendors — which means I get to watch upstream renames arrive from several directions at once. Everything below works against any endpoint that implements GET /v1/models.

Three ways a wrong ID fails

Mode A — a clean error. 404 or 400 with "code": "model_not_found". This is the good outcome. You find out in development, in the first second, and you fix a typo.

Mode B — alias resolution. You ask for gpt-4o, the provider resolves it to a dated build, and the response's model field says which one. Fine and normal — as long as you read the echo. If you never look, you've silently accepted whatever the alias points at this month.

Mode C — nearest-match routing. Some endpoints and proxies try to be helpful: they don't recognise your ID, so they route to the closest thing they do recognise. You asked for a flagship and got something else, with a 200 and no indication anywhere in the response that a substitution happened. This is the mode that costs real money and real trust, and it's invisible in exactly the way that matters.

The ten-second test for Mode C

Send an ID that cannot possibly exist, formed by mangling a real one:

curl -s https://api.daoxe.com/v1/chat/completions \
  -H "Authorization: Bearer $LLM_API_KEY" -H "Content-Type: application/json" \
  -d '{"model":"claude-sonnet-4-5-TYPO-NOT-A-MODEL",
       "max_tokens":8,"messages":[{"role":"user","content":"hi"}]}' \
  | python3 -c 'import json,sys; d=json.load(sys.stdin); \
print("ERROR:", d["error"]["message"][:80]) if "error" in d else print("SERVED BY:", d.get("model"))'

If you get an error, every model ID you send is authoritative. If you get a completion, every model ID you send is advisory — the endpoint reserves the right to pick for you, and no amount of pinning on your side will help until you know the rule it uses. Run this before you build anything else in this article.

Dated versus floating

Two ID styles, two different risks:

The policy that follows: exploration floats, production is dated, and every eval artifact records the ID that actually served it — read from the response's model field, not from what you asked for. That last part is what lets you answer "was this eval run on the same build as that one" three months later.

GET /v1/models is the source of truth, and it's per-key

Not the docs, not the pricing page, not a blog post. The catalogue endpoint returns the IDs your key is entitled to call, which is frequently a different set from what the vendor sells:

curl -s https://api.daoxe.com/v1/models \
  -H "Authorization: Bearer $LLM_API_KEY" \
  | python3 -c 'import json,sys; [print(m["id"]) for m in json.load(sys.stdin)["data"]]' \
  | sort

One caveat worth knowing up front: most implementations return IDs and little else. Context windows, tool support, vision support and modality are usually not in there, so you still need a small capability sidecar of your own. Don't infer capabilities from the name.

Lock the catalogue

#!/usr/bin/env python3
"""check_models.py — fail CI when a model ID this repo depends on disappears.

    python3 check_models.py --write      # re-pin (review the diff!)
    python3 check_models.py              # verify
"""
import argparse, json, os, sys, urllib.request

BASE = os.environ["LLM_BASE_URL"].rstrip("/")
KEY = os.environ["LLM_API_KEY"]


def catalogue():
    req = urllib.request.Request(BASE + "/models")
    req.add_header("Authorization", "Bearer " + KEY)
    with urllib.request.urlopen(req, timeout=30) as resp:
        return sorted({m["id"] for m in json.load(resp).get("data", []) if m.get("id")})


def required_ids(path="models.json"):
    """The IDs this repo actually uses — derived from the alias map, never hand-listed."""
    with open(path) as fh:
        return sorted(set(json.load(fh)["roles"].values()))


def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--lock", default="models.lock.json")
    ap.add_argument("--write", action="store_true")
    args = ap.parse_args()

    live = catalogue()
    if args.write or not os.path.exists(args.lock):
        with open(args.lock, "w") as fh:
            json.dump({"catalogue": live}, fh, indent=2)
            fh.write("\n")
        print(f"pinned {len(live)} ids to {args.lock}")
        return 0

    with open(args.lock) as fh:
        pinned = set(json.load(fh)["catalogue"])
    live_set, needed = set(live), set(required_ids())

    for mid in sorted(live_set - pinned):
        print(f"note   new in catalogue: {mid}")
    for mid in sorted(pinned - live_set - needed):
        print(f"warn   left the catalogue (unused here): {mid}")
    gone = sorted(needed - live_set)
    for mid in gone:
        print(f"ERROR  required by models.json but not in the catalogue: {mid}")
    return 1 if gone else 0


if __name__ == "__main__":
    sys.exit(main())

Run it nightly and on every pull request:

name: model-catalogue
on:
  schedule: [{ cron: "17 6 * * *" }]
  pull_request:
  workflow_dispatch:

jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.12" }
      - run: python3 check_models.py
        env:
          LLM_BASE_URL: https://api.daoxe.com/v1
          LLM_API_KEY: ${{ secrets.LLM_API_KEY }}

The two failure signals mean different things. A nightly failure means an upstream retired or renamed something you depend on, and you found out before your users did. A pull-request failure means someone added a role pointing at an ID that isn't callable — a typo caught at review time instead of at 2am.

The alias layer

Nothing in your application code should contain a model name. Code refers to roles; one file maps roles to exact IDs.

{
  "pinned_at": "2026-08-12",
  "roles": {
    "fast":   "claude-sonnet-4-5",
    "deep":   "claude-sonnet-4-5",
    "bulk":   "claude-sonnet-4-5",
    "vision": "claude-sonnet-4-5",
    "embed":  "text-embedding-3-large"
  },
  "notes": {
    "deep": "dated build; re-pin only with an eval run linked in the PR"
  }
}

import functools, json, pathlib

@functools.lru_cache(maxsize=1)
def _roles():
    return json.loads(pathlib.Path("models.json").read_text())["roles"]

def model_for(role):
    try:
        return _roles()[role]
    except KeyError:
        raise KeyError(f"no model pinned for role {role!r} — add it to models.json") from None

Three things fall out of this for free. A rename becomes a one-line diff in a reviewable file instead of a search-and-replace across forty call sites. check_models.py can derive its required set from the same file, so the lock and the code can't disagree. And git log models.json becomes a readable history of what your product was running and when — which is the artifact you'll want the first time someone asks why output quality changed in July.

Then stop the IDs from leaking back in. One test, one line:

# tests/no_hardcoded_model_ids.sh — fails if a model-looking literal appears in src/
! rg -n --glob '!models.json' \
    -e '"(gpt|claude|gemini|deepseek|qwen|kimi|grok|llama)-[a-z0-9._-]+"' src/

When an upstream retires a model mid-project

1. Find out from CI, not from users. That's what the nightly is for.

2. Read the label before you pick a successor. Vendors publish migration notes; gateways publish catalogue metadata. Don't infer the replacement from the name — -mini, -fast, -pro and numeric bumps are marketing, not a capability contract.

3. Freeze a golden set. Twenty to fifty real requests from your own traffic with their recorded outputs. If you don't have one yet, this is the moment it becomes worth an afternoon; you will need it again.

4. Run the candidate against the golden set and diff the things that actually break in production: output format compliance, tool-call rate, refusal rate, response length, and latency. Not vibes.

5. One PR, one line. Change models.json, link the eval run in the description, let CI re-verify the lock.

6. Keep the corpse. Leave the old ID in the git history of models.lock.json and in your eval artifacts. When you're asked to reproduce a result from two quarters ago, that record is the only thing that will let you say what actually served it.

Variants and channels on a gateway

If you route through a gateway, two IDs that look like the same model may reach different upstream channels with different reliability characteristics. Some catalogues label this explicitly — the one I work on tags channels as full-power, official relay, reverse-engineered ("quality not guaranteed"), or promotional, and that labelling is the whole point: you can choose a cheaper or less stable path deliberately, or refuse to. What you should never do is pattern-match a suffix and assume you know what it means. Read the catalogue metadata, pick the exact string, pin it.

Drift under a pinned ID

Pinning protects you from renames. It does not protect you from an upstream changing what sits behind a name it didn't change. The only handle you have from the outside is behavioural: record how a pinned ID answers a fixed battery of prompts at temperature 0, re-run it on a schedule, and diff.

pipx install git+https://github.com/seven7763/llm-honesty-probe

Be precise about what that gives you: signals, not proof. No probe run from outside can prove which weights served a request, and a change in output can have several innocent explanations. What it does produce is a timestamped record that something changed on a date, which is a much stronger position to argue from than "it feels worse lately."

Pin the ID, alias the role, lock the catalogue, watch the behaviour. It's an afternoon of work, and it converts your loudest class of production surprise into a CI failure.

Disclosure: I run DaoXE, an OpenAI-compatible LLM gateway. Everything below applies to us exactly as it applies to any other provider — that's the point.

Why "is this the model I paid for?" became the main question

The market of intermediaries between users and LLMs grows faster than any guarantee attached to it. In reviews of API gateways you'll find very different opinions — and a lot of arguments about how to check transparency. That's not an argument against intermediaries as a class; it's an argument for verifiability.

Below is a set of checks you can run against any API provider — including us — before you commit budget. We deliberately refuse the "trust us" answer: there's nothing to trust here, and there's something to check.

Step 0. What a client can actually verify

Honest about the limits of the method:

• Verifiable from the client side: which models your account can reach, response format, latency, actual behavior on tasks where a strong and a weak model answer visibly differently, and whether billing matches the traffic you sent.

• Not verifiable from the client side: what infrastructure sits behind the endpoint, or what terms the intermediary has with upstream providers. Anyone claiming otherwise is selling you trust, not a mechanism.

So the goal isn't "catch them or forgive them" — it's making the service's behavior predictable and measurable for you.

Step 1. The model list of your own account

A standard OpenAI-compatible endpoint exposes the models available to your account:

export API_KEY="your_key"
curl -s https://api.daoxe.com/v1/models \
  -H "Authorization: Bearer ${API_KEY}" | jq -r '.data[].id' | sort

What to look for:

• the model you're paying for is actually in the list (not just a cheaper cousin);

• exact IDs (claude-sonnet-4-5, not gpt-4-style vagueness) — a mismatched ID is the first red flag;

• the list changes without notice — that's a question worth asking.

Step 2. Behavioral tests: tasks where models disagree

The crudest and most honest method. Pick 5–10 tasks where models you know answer differently and predictably:

• rare arithmetic with an exact answer (strong models compute, weak ones approximate);

• a prompt with a deliberate trap in the wording (a strong model notices it, a weak one ignores it);

• code with a known edge case (a strong model names it);

• "what can't you do?" — every model has its own refusal pattern.

Run the same set through the endpoint and compare against the same model's answers in its official interface (wherever you have access). Matching patterns is an indirect sign the claimed model is answering. Divergence is a question to ask.

This is a signal, not proof. We're deliberate about that wording: proving token provenance from the client side is impossible, and anyone promising such proof is also selling you trust.

Step 3. Billing versus traffic

Keep a simple ledger: tokens sent/received (the usage field in every response) versus what was deducted from your balance. A discrepancy larger than the tariff difference explains isn't a trust question anymore — it's an accounting question for the provider.

import requests, json
r = requests.post("https://api.daoxe.com/v1/chat/completions",
    headers={"Authorization": f"Bearer {API_KEY}"},
    json={"model": "claude-sonnet-4-5", "messages": [{"role":"user","content":"ping"}]})
print(json.dumps(r.json().get("usage"), indent=2))

Step 4. Protocol details that give away a shim

• Native Anthropic /v1/messages: if a provider only speaks OpenAI format, Claude Code connects through a translation layer, and parts of the feature set (tool use, cache_control, thinking) behave differently. Check: a direct request to /v1/messages with an x-api-key header — does it work natively or not?

• Streaming truncation: long streamed responses are where cheap channels commonly cut generation short.

• Latency as an indirect signal: a flat 3–5 seconds to first token on a "top" model can mean something else is behind the endpoint. Signal, not verdict — latency depends on geography, load, and context length.

Step 5. What to ask a provider before paying

1. Show me how I can verify I'm getting the model I asked for. (The answer should be a method, not "we're honest.")

2. When a channel degrades, do you tell me or silently switch?

3. How is billing computed — per token or by an "approximate rate"? Where's the deduction history?

4. Is there a public changelog of models and groups?

5. What happens to my balance if the service shuts down? (An honest answer to this one is rare and therefore valuable.)

If the answers to 1–2 are elaborate, that is the answer.

What we do about it

We run a gateway at https://api.daoxe.com/v1 (OpenAI-compatible) with native Anthropic /v1/messages, one balance, per-model USD pricing listed at daoxe.com/pricing. We don't promise "100% stability" and we don't claim we can prove token provenance. We claim less and let you check more:

• your account's model list is open (GET /v1/models);

• deduction history and usage in every response — for billing reconciliation;

• Claude Code works through the native protocol, not an OpenAI shim;

• an open-source CLI checks latency and connectivity without printing or storing your key (github.com/seven7763/DaoXE-AI).

Every check in this article applies to us the same as to anyone else. That's the offer: don't take our word for it — run the three steps.

If you found a factual error, say so — we'll fix it and note the correction at the end.

Рабочие шаблоны для повседневных задач: перевод, документы, поддержка, код, маркетинг. Копируйте, подставляйте текст в {фигурные скобки} — и отправляйте любой модели, с которой работаете.

Как пользоваться: подставьте свой текст вместо {...}, при необходимости сократите или расширьте роль в начале. Шаблон — это каркас, а не догма: чем конкретнее детали (аудитория, тон, ограничения), тем полезнее результат.

1. Перевод EN ↔ RU (4 шаблона)

1.1 Перевод с сохранением стиля

Переведи следующий текст на {язык}. Сохрани тон, регистр и структуру оригинала. Не интерпретируй и не добавляй от себя. Если идиома непереводима, дай вариант closest по смыслу и поясни выбор в скобке. Текст: {…}

1.2 Техническая документация

Переведи фрагмент технической документации на русский. Держи термины единообразно: {список терминов → перевод}. Код, имена переменных и пути не переводи и не меняй. Если в оригинале двусмысленность, отметь её комментарием // NB:. Фрагмент: {…}

1.3 Деловое письмо (RU → EN)

Переведи это письмо на английский так, чтобы оно звучало как нативная деловая переписка: вежливо, без излишней формальности, с активным залогом. Ключевые факты и цифры не менять. Письмо: {…}

1.4 Субтитры/озвучка (EN → RU)

Сделай перевод субтитров на русский: уложись в {N} символов на строку, сохрани смысл реплик, адаптируй шутки и культурные отсылки, не оставляй «кальку» с английского. Формат — тот же, что во входе: {…}

2. Документы и саммари (4 шаблона)

2.1 Выжимка длинного документа

Сожми документ до {объём}. Структура: 1) главная мысль в одном предложении; 2) 3–7 ключевых тезисов; 3) цифры/факты, которые важны для решения; 4) что в документе не раскрыто. Не выдумывай то, чего в тексте нет. Документ: {…}

2.2 Протокол встречи

Из заметок ниже сделай протокол встречи: решения (в виде списка), задачи с ответственными и сроками (если упомянуты), открытые вопросы. Стиль — сухой деловой. Заметки: {…}

2.3 Договор / ТЗ — объяснение простым языком

Объясни разделы этого документа простыми словами: что каждое условие означает на практике, где риски для стороны {мы — заказчик/исполнитель}, какие пункты стоит обсудить до подписания. Юридические формулировки цитируй точно. Документ: {…}

2.4 Ответ на претензию/отзыв

Напиши ответ на отзыв: {текст отзыва}. Тон: спокойный, по делу, без оправданий и канцелярита. Структура: признание проблемы (если она реальна) → что сделали/сделаем → конкретный следующий шаг. Не обещай того, что не подтверждено.

3. Поддержка и коммуникации (4 шаблона)

3.1 Ответ в техподдержке

Пользователь пишет: {обращение}. Продукт: {краткое описание}. Ответь на русском: 1) по делу — решение или следующий шаг; 2) без извинений-заготовок; 3) если информации не хватает — один уточняющий вопрос. Тон — как у коллеги, а не бота.

3.2 База знаний по частым вопросам

Из переписок ниже извлеки частые проблемы и собери FAQ: вопрос → короткий ответ (2–4 предложения) → когда эскалировать человеку. Переписки: {…}

3.3 Черновик письма клиенту

Напиши письмо клиенту: {ситуация}. Цель письма: {что должен сделать клиент после прочтения}. Не больше 150 слов, тема письма — отдельной строкой, призыв к действию один.

3.4 Коммуникация инцидента

Сформулируй статус инцидента для клиентов: что произошло (факты), кого затронуло, текущее состояние, что делаем дальше, когда следующий апдейт. Без оценочных слов «критический», «невероятный». Данные: {…}

4. Код и ревью (4 шаблона)

4.1 Ревью кода

Сделай код-ревью: {код}. Порядок: 1) баги и риски (с номерами строк); 2) безопасность; 3) читаемость/структура; 4) производительность — только там, где это важно для данного пути выполнения. По каждому пункту: проблема → почему → как исправить. Не переписывай стиль без необходимости.

4.2 Объяснение чужого кода

Объясни этот код: что делает каждый блок, какие внешние зависимости и скрытые предусловия, где могут быть побочные эффекты. Код: {…}

4.3 Тесты к функции

Напиши тесты для функции: {код}. Стек: {pytest / jest / …}. Покрой граничные случаи (пустые значения, границы диапазонов, ошибки). Не тестируй поведение сторонних библиотек.

4.4 Миграция/рефакторинг

Предложи план миграции {со} на {на}: шаги по порядку, что сломается, как проверить каждый шаг, вариант отката. Код: {…}

5. Маркетинг и контент (4 шаблона)

5.1 Описание продукта

Напиши описание {продукта} для {аудитория}. Формат: 1 заголовок (до 60 знаков) + 3 буллита о выгодах + 1 предложение с фактом, который можно проверить. Не используй «лучший», «уникальный», «инновационный» и прочие слова-пустышки.

5.2 Пост в соцсети по теме

Сделай пост на тему {тема} для {площадка}. Требование: начинай с конкретного (цифра/ситуация), без вступлений про «в современном мире», финальный призыв один. Тон: {…}. Ограничение по длине: {N} знаков.

5.3 Анализ конкурентного предложения

Вот тексты двух сайтов: {наш} и {конкурент}. Сравни: кому что важно, какие аргументы у нас сильнее, где конкурент честнее/убедительнее, какие формулировки нам не подходят и почему. Без «мы молодцы», только разбор.

5.4 Письмо-рассылка (без спама)

Напиши письмо для {аудитория} по поводу {повод}. Не больше 120 слов. Заголовок — по делу, без капса и «!!!». Один призыв к действию. Дай также альтернативный заголовок для A/B-теста.

Что дальше

Эти шаблоны работают с любой языковой моделью. Если хотите один ключ к нескольким моделям (Claude, GPT, Gemini, DeepSeek и др.) через единый OpenAI-совместимый endpoint https://daoxe.com/v1:

• русскоязычный гид по подключению: https://seven7763.github.io/daoxe-guide/ru/?utm\_source=hashnode&utm\_medium=organic&utm\_campaign=ru\_test\_0904&utm\_content=hn\_ru

• актуальные цены моделей: https://daoxe.com/pricing?utm\_source=hashnode&utm\_medium=organic&utm\_campaign=ru\_test\_0904&utm\_content=hn\_ru

• оплата и актуальные цены по каждой модели — на daoxe.com/pricing

• поддержка: Telegram @daoxe_ai

Короткий опрос (2 минуты, ответьте в комментариях или в @daoxe_ai)

Опрос анонимный: не указывайте в ответах имя, email, телефон и платёжные реквизиты.

1. Какими AI-моделями/сервисами вы сейчас пользуетесь чаще всего?

2. Что для вас важнее всего при выборе: цена, стабильность API, качество ответов на русском, оплата (USDT/карта), что-то ещё?

3. Вам нужнее: единый API-ключ к нескольким моделям — или готовый чат «для общения, не для кода»?

4. Какой формат оплаты вам удобнее для API-сервисов: крипто-кошельки (USDT/USDC) или карты?

5. Что было бы полезнее дальше: ещё шаблоны промптов, туториалы по подключению, сравнения моделей — или кейсы «как мы решили задачу X»?

Спасибо! Ответы читаем и используем, чтобы делать сервис полезнее.

Posted first on DEV.to: https://dev.to/seven7763/one-api-key-three-terminals-driving-many-models-from-the-shell-with-llm-mods-and-aichat-c0b

The fastest way to use an LLM is often not a chat window — it's a pipe. git diff | ... "write a commit message", ... "explain this error" < stderr.log, cat main.py | ... "find the bug". Three of the best command-line tools for this — llm, mods, and aichat — all accept a custom OpenAI-compatible endpoint, which means one key can back all three and give you Claude, GPT, Gemini and more from the shell.

This is the terminal setup, end to end. It's genuinely tool-agnostic: every config below works against any OpenAI-compatible endpoint. I use daoxe as the example endpoint (disclosure below); swap the base URL for your own.

Disclosure: I work on daoxe, an OpenAI-compatible gateway (it also speaks Anthropic Messages natively). Everything here is standard OpenAI-compatible config — point these tools at whatever endpoint you trust.

One convention across all three: the key lives in an environment variable, DAOXE_API_KEY, never on the command line (so it doesn't land in your shell history). And model ids are account-scoped — list yours with curl https://daoxe.com/v1/models -H "Authorization: Bearer $DAOXE_API_KEY".

1. llm (Simon Willison) — the extensible one

llm is the most extensible of the three: a pluggy-based plugin system, a local SQLite log of everything, templates, embeddings. There are two ways to add an OpenAI-compatible gateway.

Option A — zero-YAML via a plugin

There's a small plugin, llm-daoxe, that registers the gateway's models and adds a command to pull your account's real model list:

llm install llm-daoxe           # (coming soon on PyPI; until then: clone + `pip install -e .`)
llm keys set daoxe              # store the key (or export DAOXE_API_KEY=...)
llm daoxe models --refresh     # pull your account's real models from GET /v1/models, and cache them
llm -m daoxe/claude-sonnet-4-6 "In one line, what's an OpenAI-compatible gateway?"
git diff | llm -m daoxe/gpt-5.5 -s "Write a concise commit message."

Model ids are namespaced as daoxe/<model> so they don't collide with other plugins' ids. The key is read only from llm's key store or DAOXE_API_KEY, and it's never printed.

Option B — no plugin, native config

Don't want a plugin? llm natively reads custom OpenAI-compatible models from extra-openai-models.yaml in its config dir:

- model_id: daoxe/claude-sonnet-4-6
  model_name: claude-sonnet-4-6
  api_base: https://daoxe.com/v1
  api_key_name: daoxe          # matches `llm keys set daoxe`
  can_stream: true
- model_id: daoxe/gpt-5.5
  model_name: gpt-5.5
  api_base: https://daoxe.com/v1
  api_key_name: daoxe

The plugin's advantage is just convenience: zero YAML plus llm daoxe models --refresh to auto-discover your account's models. Both paths use llm's built-in OpenAI transport, so streaming and tools behave the same as with official models.

2. mods (charmbracelet) — the pipe-native one

mods is built for Unix pipes and looks great doing it. Open its config with mods --settings and add a provider under apis::

default-api: daoxe            # optional: make it the default so you can skip -a
apis:
  daoxe:
    base-url: https://daoxe.com/v1
    api-key-env: DAOXE_API_KEY
    models:                  # use real ids from GET /v1/models; max-input-chars is a client-side value
      claude-sonnet-4-6:
        aliases: ["daoxe-sonnet"]
        max-input-chars: 600000
      gpt-5.5:
        aliases: ["daoxe-gpt"]
        max-input-chars: 600000

export DAOXE_API_KEY=your-daoxe-key
mods -a daoxe -m claude-sonnet-4-6 "explain this diff" < patch.diff
mods --model daoxe-sonnet "summarize" < README.md      # via alias

Gotcha: mods requires api-key-env to point at an environment variable that already exists, or it complains about a missing key. max-input-chars is a client-side truncation setting — tune it to your model, it's not a claim about the gateway.

3. aichat (sigoden) — the all-in-one one

aichat is a single binary that's a CLI, a REPL, and a local server. It uses an openai-compatible client type. Edit ~/.config/aichat/config.yaml:

clients:
  - type: openai-compatible
    name: daoxe
    api_base: https://daoxe.com/v1
    # omit api_key and aichat auto-reads DAOXE_API_KEY (client name → {NAME}_API_KEY)
    models:                     # real ids from GET /v1/models; tune capabilities per model
      - name: claude-sonnet-4-6
        max_input_tokens: 200000
        supports_function_calling: true
      - name: gpt-5.5
        max_input_tokens: 272000
        supports_function_calling: true

export DAOXE_API_KEY=your-daoxe-key
aichat -m daoxe:claude-sonnet-4-6 "explain this code" < main.py
aichat --list-models | grep daoxe

Nice touch: because the client is named daoxe, aichat automatically looks for DAOXE_API_KEY — you can omit the api_key field entirely. Model selection is client:model, e.g. daoxe:claude-sonnet-4-6.

Which one should you use?

They're not mutually exclusive — I keep all three configured against the same key. The same mental model applies to other CLIs too: anything that takes an OpenAI-compatible base_url + key + model id works the same way (point the base URL at the /v1 root, use a real id from /v1/models, keep the key in an env var). Verify each tool's current config fields before you rely on them — these tools move fast.

The reason one key matters here

CLI users tend to script things and forget them: a cron job that summarizes logs, a git hook that drafts commit messages, a shell function that explains errors. When each of those needs a different vendor key, it's a mess of secrets and bills. One OpenAI-compatible key across llm, mods, and aichat — and across every script that calls them — means one secret to rotate, one bill to read, and model choice as a string.

daoxe is the endpoint I point them at: https://daoxe.com/v1, one key across many models, plus native Anthropic Messages if a tool wants Claude's protocol. It's not available in mainland China. As always, none of the config above is daoxe-specific — and because it's a standard endpoint, you can benchmark and verify it rather than trust it.

TL;DR

• llm, mods, and aichat all take a custom OpenAI-compatible endpoint — one key backs all three.
• llm: a plugin (llm-daoxe, coming soon) or extra-openai-models.yaml.
• mods: an apis: block; api-key-env must point at an existing env var.
• aichat: an openai-compatible client; name it daoxe and it auto-reads DAOXE_API_KEY.
• Keep the key in an env var, use exact ids from /v1/models, and pipe away.

Posted first on DEV.to: https://dev.to/seven7763/bring-your-own-key-to-github-copilot-chat-cli-app-and-sdk-with-the-honest-caveat-mod

Most people don't know Copilot can do this yet: as of 2026, GitHub Copilot has GA'd Bring Your Own Key (BYOK). You can point Copilot Chat (in VS Code), the Copilot CLI, the Copilot app, and the Copilot SDK at any OpenAI-compatible endpoint, Anthropic, or Azure — including local runtimes and third-party gateways.

That's a big deal if you already pay for Copilot but want to run its chat and agent features on a model or a gateway of your choice. This guide covers all four surfaces, with the exact steps and one honest caveat that trips people up.

Disclosure: I work on daoxe, an OpenAI-compatible gateway that also speaks the Anthropic Messages protocol. It happens to line up neatly with Copilot's three wire protocols, so I use it as the example — but everything here works with any compatible endpoint. Substitute your own base URL.

The caveat, first (so nobody's misled)

BYOK affects Copilot Chat and agent sessions. It does not change inline completions. Those gray "Tab to accept" suggestions still run on Copilot's own infrastructure and are unaffected by your base URL. If your goal is to change what powers chat/agent, BYOK is for you. If you wanted to swap the model behind inline completions, BYOK won't do it. That's the honest scope.

Why a gateway fits Copilot's three protocols

Copilot's BYOK supports three provider "types", and a good multi-protocol gateway hits all three:

• openai → the gateway's /v1/chat/completions (default).
• openai with wireApi: "responses" → the gateway's /v1/responses.
• anthropic → the gateway's /v1/messages (Claude's native protocol).

So one key can serve GPT-shaped, Responses-shaped, and Claude-native requests, depending on which model you pick. Now the four ways to wire it.

1. Copilot CLI (easiest — just env vars)

export COPILOT_PROVIDER_BASE_URL=https://daoxe.com/v1
export COPILOT_PROVIDER_API_KEY=$DAOXE_API_KEY     # your gateway key, from an env var
export COPILOT_MODEL=gpt-5.5                        # any exact id from GET /v1/models
# optional: COPILOT_PROVIDER_TYPE=openai (default) | anthropic | azure
copilot

For Claude's native protocol, set COPILOT_PROVIDER_TYPE=anthropic and use the gateway's Anthropic entry point. Confirm the exact path with curl first — many gateways take the host root (https://daoxe.com) and let the client append /v1/messages; some want /v1. Don't assume; check your account.

2. Copilot Chat (VS Code)

1. Command Palette → Chat: Manage Language Models.
2. Choose OpenAI Compatible / Custom Endpoint.
3. Base URL: https://daoxe.com/v1 (it must serve /chat/completions). VS Code probes GET /models to fill the model dropdown — if your gateway supports it, models auto-populate; otherwise type the Model ID by hand.
4. Paste your API Key, pick a Model ID (e.g. claude-sonnet-4-6) → Add Model.

The model now shows up in the Copilot Chat model picker alongside the built-ins.

3. Copilot App (desktop)

Settings → Model Providers → Add provider. Fill in the endpoint https://daoxe.com/v1 and your API key (or choose the anthropic type for Claude-native). Your models then appear in the model picker next to Copilot's hosted ones; the key is stored in your OS keychain.

4. Copilot SDK (for integrators)

provider: {
  type: "openai",                    // or "anthropic"
  baseUrl: "https://daoxe.com/v1",   // full path; for anthropic, use your account's path
  apiKey: process.env.DAOXE_API_KEY,
  wireApi: "completions",            // use "responses" when you need Responses-API behavior
}

Getting the model ids right

Copilot only calls what you tell it to, so use the exact ids your endpoint exposes:

curl https://daoxe.com/v1/models -H "Authorization: Bearer $DAOXE_API_KEY"

Guessed names fail silently or fall back. A gateway with an account-scoped /v1/models is ideal here, because the list you see is exactly the list you can call.

Troubleshooting

• Model dropdown empty in VS Code? Your endpoint may not expose GET /models, or the key lacks access. Type the id manually and confirm the key with the curl above.
• 401 / auth errors? Check whether the provider expects Authorization: Bearer (OpenAI type) vs x-api-key (some Anthropic setups). Match the provider type to the protocol.
• Anthropic type 404s? It's almost always the base URL shape — root vs /v1. Verify against your gateway's docs.
• Chat works but inline completions "didn't change"? Expected — see the caveat. Inline stays on Copilot's models.

What to look for in a BYOK target

Since you're choosing the backend now, pick one that is:

• Multi-protocol. OpenAI Chat Completions and Responses and Anthropic Messages, so all of Copilot's provider types work and Claude runs on its native protocol.
• One key, many models, with an account-scoped /v1/models.
• Verifiable. Fine with you benchmarking it and checking it isn't silently downgrading you (there are open-source tools for exactly this).
• Reachable where official billing isn't. If your card keeps getting declined at an official checkout, a gateway that accepts other payment paths is the difference between shipping and not.

daoxe checks those boxes — OpenAI-compatible base URL (https://daoxe.com/v1), native Anthropic Messages, one key across many models — and it's not available in mainland China. But BYOK is the point: you're not locked in, so point Copilot at whatever endpoint you trust and can verify.

TL;DR

• Copilot BYOK works in Chat, CLI, App, and SDK — set a base URL + key.
• A multi-protocol gateway covers all three Copilot provider types (openai / responses / anthropic).
• Use exact model ids from /v1/models.
• Inline completions don't change — BYOK is for chat/agent only.

Posted first on DEV.to: https://dev.to/seven7763/one-key-for-crewai-autogen-llamaindex-and-8-more-the-baseurl-trick-for-python-agents-5a3g

Here's a fact that quietly makes multi-model agent development a lot less painful: in 2026, virtually every Python agent framework natively supports pointing its underlying LLM at a custom OpenAI-compatible base_url. No new package, no fork, no framework change. You set one URL and one key, and the framework talks to whatever backend you choose.

That means a single OpenAI-compatible gateway — one key that fronts Claude, GPT, Gemini, DeepSeek, Kimi and more, with fallback and one bill — is a natural backend for all of these frameworks at once. The orchestration stays with the framework; the models come from one endpoint.

This post is the map: the exact one-liner for each framework, plus the one honest gotcha each has, because those gotchas are what actually cost you an afternoon.

Disclosure: I work on daoxe, one such gateway. But nothing below is daoxe-specific — every snippet works against any OpenAI-compatible endpoint. Swap the base URL for your own; the frameworks don't care who's behind it.

Why this works at all

Almost every framework builds on the OpenAI Chat Completions shape:

POST {base_url}/chat/completions
Authorization: Bearer <key>

The only two things that route this call to OpenAI are the base URL and the key. Change the base URL to a compatible gateway and the same orchestration code now runs against a different backend. Frameworks expose this under slightly different names — base_url, api_base, api_base_url — but it's the same idea.

Two universal rules before the snippets:

• Model ids are account-scoped. Don't copy an id from a blog post. List yours with curl {base_url}/models -H "Authorization: Bearer $KEY" and use an exact id.
• Keys go in env vars, never hardcoded. Every snippet below reads from the environment.

The 11 frameworks, grouped by mechanism

A) Frameworks with a direct base_url parameter

LangChain (+ LangGraph) — the biggest ecosystem, so start here:

from langchain_openai import ChatOpenAI
llm = ChatOpenAI(model="MODEL_ID", base_url="https://daoxe.com/v1", api_key="...")  # key via env
print(llm.invoke("ping").content)

# LangGraph: feed that llm straight into a prebuilt agent
from langgraph.prebuilt import create_react_agent
agent = create_react_agent(llm, tools=[...])

Gotcha: ChatOpenAI targets the official OpenAI spec; non-standard fields some proxies add aren't preserved. Fine for standard responses.

AutoGen (Microsoft AgentChat):

from autogen_agentchat.agents import AssistantAgent
from autogen_core.models import ModelInfo
from autogen_ext.models.openai import OpenAIChatCompletionClient

client = OpenAIChatCompletionClient(
    model="MODEL_ID", base_url="https://daoxe.com/v1", api_key="...",
    model_info=ModelInfo(vision=False, function_calling=True, json_output=True,
                         family="unknown", structured_output=True),
)
agent = AssistantAgent("assistant", model_client=client)

Gotcha: a non-OpenAI model id needs both base_url and model_info (capability flags). Omit model_info and it errors. Set the flags to match the model behind your id.

smolagents (Hugging Face):

from smolagents import CodeAgent, OpenAIModel
model = OpenAIModel(model_id="MODEL_ID", api_base="https://daoxe.com/v1", api_key="...")
print(CodeAgent(tools=[], model=model).run("ping"))

Gotcha: pre-1.9 the class was OpenAIServerModel (same params). Uses api_base, not base_url.

Haystack (deepset):

from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.utils import Secret

gen = OpenAIChatGenerator(model="MODEL_ID", api_base_url="https://daoxe.com/v1",
                          api_key=Secret.from_env_var("DAOXE_API_KEY"))
print(gen.run([ChatMessage.from_user("ping")])["replies"][0].text)

Gotcha: the param is api_base_url (note the extra word), and the key goes through Secret.from_env_var(...).

Agno (formerly phidata):

from agno.agent import Agent
from agno.models.openai.like import OpenAILike
Agent(model=OpenAILike(id="MODEL_ID", base_url="https://daoxe.com/v1", api_key="...")).print_response("ping")

Gotcha: use OpenAILike (from agno.models.openai.like), not the plain OpenAIChat, for a third-party endpoint.

B) Frameworks with an explicit provider object

PydanticAI — type-safe agents:

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider

model = OpenAIChatModel("MODEL_ID",
    provider=OpenAIProvider(base_url="https://daoxe.com/v1", api_key="..."))
agent = Agent(model)
print(agent.run_sync("ping").output)

Gotcha: build a generic OpenAIProvider and pass it to OpenAIChatModel — this is the same pattern the docs use for any OpenAI-compatible provider.

LlamaIndex:

from llama_index.llms.openai_like import OpenAILike  # pip install llama-index-llms-openai-like

llm = OpenAILike(model="MODEL_ID", api_base="https://daoxe.com/v1", api_key="...",
                 is_chat_model=True, is_function_calling_model=True, context_window=128000)
print(llm.complete("ping"))

Gotcha: set is_chat_model=True or it calls the /completions endpoint instead of /chat/completions. Also set a real context_window.

C) Frameworks that route through LiteLLM (need the openai/ prefix)

CrewAI:

from crewai import LLM, Agent, Crew, Task
llm = LLM(model="openai/MODEL_ID", base_url="https://daoxe.com/v1", api_key="...")
agent = Agent(role="Greeter", goal="Greet briefly.", backstory="Concise.", llm=llm)
task = Task(description="ping", expected_output="A short greeting.", agent=agent)
print(Crew(agents=[agent], tasks=[task]).kickoff())

Gotcha: keep the openai/ prefix on the model id. Without it, CrewAI (via LiteLLM) may match a native provider client for a familiar model name and ignore base_url — you'll get a misleading "API key not valid". Use the API root (.../v1), not .../chat/completions.

DSPy:

import dspy
lm = dspy.LM("openai/MODEL_ID", api_base="https://daoxe.com/v1", api_key="...")
dspy.configure(lm=lm)
print(dspy.Predict("question -> answer")(question="ping").answer)

Gotcha: same openai/ prefix, and keep /v1 in api_base with no trailing slash — a trailing slash can double the path and 404.

D) Wrap the OpenAI client directly

Instructor — structured output validated into Pydantic models:

import instructor
from openai import OpenAI
from pydantic import BaseModel

class Out(BaseModel):
    text: str

client = instructor.from_openai(OpenAI(base_url="https://daoxe.com/v1", api_key="..."))
print(client.chat.completions.create(model="MODEL_ID", response_model=Out,
        messages=[{"role": "user", "content": "ping"}]))

Gotcha: the default (tool-calling) mode works for tool-capable models; if a model lacks tool calling, pass mode=instructor.Mode.JSON.

The gotchas, in one table

Why one endpoint beats juggling keys

Multi-agent orchestration is where model sprawl hurts most: a planner on a strong model, workers on a cheap one, a critic on a third — that's three vendors, three keys, three bills, three failure modes. Pointing all of them at one OpenAI-compatible endpoint means:

Three agent roles, three models, one endpoint and one bill — model choice is just a string.

• One key, one bill across every framework and every agent role.
• Model choice is a string, not a new integration. Swap MODEL_ID and the same agent runs on Claude, GPT, or Gemini.
• You're not locked to a framework. The same key runs the same task across all 11.

And because it's a standard endpoint, you can verify it rather than trust it — run a probe against it and against the official API and compare (I wrote a separate piece on catching silent model swaps).

daoxe is the endpoint I use for this: OpenAI-compatible at https://daoxe.com/v1, plus native Anthropic Messages for Claude-native tools, one key across many models. It's not available in mainland China. But the whole point of this article is that none of the code above is daoxe-specific — point it at any compatible endpoint you trust.

TL;DR

• Every framework here takes a custom OpenAI-compatible endpoint natively — it's a one-liner.
• Watch the param name (base_url vs api_base vs api_base_url) and the per-framework gotcha.
• CrewAI/DSPy need the openai/ prefix; AutoGen needs model_info; LlamaIndex needs is_chat_model=True.
• One key, one bill, model choice as a string — and you can verify the backend instead of trusting it.

This week, OpenAI slashed Luna (GPT-5.6) pricing by 80% — from $7 to $1.40 per million tokens. Anthropic shipped Opus 5 at half the price of its predecessor. The LLM API market is in a genuine price war, and for once, developers are winning.

I should be celebrating. I'm not. Here's why.

## The hidden math of cheaper models

When official API prices drop, the economics of model-swapping change in a way nobody talks about.

A bad relay has a simple business model: bill you for Model X, serve you Model Y, pocket the difference. When Model X costs \(15/M tokens and Model Y costs \)0.15/M, the margin is $14.85. That's already attractive.

When Model X drops to $1.40? The absolute margin shrinks — and that's what most people see. But the *relative* margin — the ratio of what you're charged vs what's actually served — can actually grow if budget models get cheaper still. An

The point isn't the exact math. The point is: **price competition between legitimate providers doesn't eliminate the incentive to swap models. It often masks it.** When everyone's prices are dropping, a relay that's 30% cheaper than official looks like aggressive discounting, not fraud.

## "What model are you?" doesn't work

If you're still asking an API endpoint to identify itself, stop. That's a system prompt — trivially spoofed. Every relay operator knows to make the model say "I am Claude, created by Anthropic."

What works: **behavioral fingerprinting at temp=0.**

Give the model a fixed set of tasks that different models handle differently — reasoning problems, long-context recall, strict JSON formatting, refusal boundaries. Run them at temperature 0 to remove sampling noise. Diff the results against the official API.

This isn't theoretical. Here's the actual tool:

```bash

git clone https://github.com/seven7763/llm-honesty-probe

cd llm-honesty-probe

python3 obe --self-test --card

```

That runs a battery of deterministic probes and produces a verdict card: PASS, SUSPICIOUS, or DEGRADED — with the specific probes that triggered each finding. No API key leaves your machine. No judgment calls required. The self-test runs without any API key and takes about 30 seconds.

To test your actual endpoint:

```bash

export LLM_ENDPOINT="https://your-endpoint/v1"

export LLM_API_KEY="sk-your-key"

python3 -m llm_honesty_probe --compare

```

## Why this matters more now, not less

The price war is genuinely good news. Lower costs mean more experiments, more products, more people who can afford to build with LLMs. I want all of that.

But cheaper official APIs also mean more users signing up for the first time — users who don't know what "Claude" is supposed to sound like, what GPT-5.6's reasoning depth should feel like, or what latency to expect. These are exactly the users a model-swapping relay wants: no baseline for comparison.

Thet. It's **verification that costs less than being wrong.**

## Test your endpoint right now

If you're using any third-party API endpoint — mine included — run the probe. The self-test takes 30 seconds. The compare against your endpoint takes about 2 minutes.

If everything passes, you've got evidence — not proof, but signals — that you're getting what you pay for. If something flags, you know exactly which probe to investigate.

Either way, you know more than you did before. That's the whole point.

---

*Disclosure: I work on [daoxe](https://daoxe.com/?utm\_source=hashnode&utm\_medium=organic&utm\_campaign=en\_launch), an OpenAI-compatible LLM gateway. I also maintain `llm-honesty-probe`, the open-source tool mentioned in this post. The probe treats every endpoint identically — including daoxe. Point it at anyone. If it flags something on my own service, I want to know before you do.*

*The price war is real. The trust problem is real. They're related, and fixing one doesn'y fix the other.*

Tags: #ai #llm #opensource #testingt automaticall defense isn't trus-m llm_honesty_prd they have.