diff --git a/examples/providers/byteplus/.gitignore b/examples/providers/byteplus/.gitignore new file mode 100644 index 000000000..77ac75498 --- /dev/null +++ b/examples/providers/byteplus/.gitignore @@ -0,0 +1,3 @@ +.venv/ +__pycache__/ +*.pyc diff --git a/examples/providers/byteplus/README.md b/examples/providers/byteplus/README.md new file mode 100644 index 000000000..3b74edebb --- /dev/null +++ b/examples/providers/byteplus/README.md @@ -0,0 +1,80 @@ +# BytePlus ModelArk tracing example + +Runnable companion to the docs page `src/pages/docs/integrations/providers/byteplus.mdx` +(`/docs/integrations/providers/byteplus`). It traces BytePlus ModelArk chat completions made with the +stock `openai` Python client and the existing `traceAI-openai` instrumentor. There is no BytePlus +package and no `traceai-byteplus`. + +Status: alpha. **No BytePlus account was used and no request was ever sent to a BytePlus host.** See +"What is real and what is a stand-in" below. + +## Contents + +| Path | What it is | +| --- | --- | +| `app.py` | The recipe (about 10 lines of integration) plus scenarios: `chat`, `stream`, `tools`, `thinking`, `error` | +| `run_e2e.sh` | Creates a venv, runs the tests, runs `app.py` against a collector, optionally runs the privacy comparison | +| `standin/byteplus_modelark_stub.py` | Local STAND-IN shaped like the ModelArk Chat API (`/api/v3/chat/completions`). Not BytePlus | +| `standin/connect_probe.py` | Refusing HTTP proxy that records which host the unmodified client dials. Opens no tunnel | +| `tests/` | pytest suite (see below) | +| `requirements.txt` | Pinned versions the example was tested with | + +## Run it + +Python 3.10 to 3.13 (traceAI-openai 0.1.10 does not support 3.14). + +```bash +export FI_API_KEY=... FI_SECRET_KEY=... +export FI_BASE_URL=http://localhost:4318 # your Future AGI collector (run_e2e.sh defaults to this) +./run_e2e.sh # tests, then app.py against the stand-in. No BytePlus needed +PRIVACY=1 ./run_e2e.sh # also: content capture off vs default, with a unique marker +``` + +Against the real BytePlus API (needs your own account, activated ModelArk and a key; this can cost money, +and the script never creates an account): + +```bash +export ARK_API_KEY=... # BytePlus ModelArk key, goes to the OpenAI client only +export ARK_MODEL=... # Model ID or endpoint ID from your ModelArk console +export ARK_REGION=ap # ap (Johor, default) or eu (Dublin): must match the region that issued the key +./run_e2e.sh +``` + +Any OpenAI-compatible server can stand in for the target: set `ARK_BASE_URL`, `ARK_API_KEY` and `ARK_MODEL`. +`ARK_BASE_URL` containing `/api/coding` is refused: that is the Coding Plan, not this recipe. + +Run a single scenario with `APP_ARGS="--scenario stream"`. `python app.py --help` lists the flags. + +## What the tests cover + +| File | Covers | +| --- | --- | +| `tests/test_recipe.py` | Region URLs and default; Coding Plan URLs refused; trace shape (AGENT, LLM, TOOL, one session); model and usage; streaming usage; `thinking` in `extra_body`; error span; provider label `openai`; keys go only where they belong; uncompressed OTLP with `project_name`; an unreachable collector never breaks the request | +| `tests/test_regions.py` | With the documented AP and EU URLs, the unmodified client dials exactly `ark.ap-southeast.bytepluses.com:443` / `ark.eu-west.bytepluses.com:443`, never the other region; the span records neither host | +| `tests/test_privacy.py` | A unique marker in every prompt, answer, tool argument and tool result: present by default, absent from every exported byte with `FI_HIDE_*`; what the hide flags do not remove | +| `tests/test_page.py` | The Python snippet on the docs page runs as written (with the base URL swapped for the stand-in) and produces the span the page describes | +| `tests/test_stub.py` | The stand-in only claims fields BytePlus documents | +| `tests/test_ark_sdk.py` | Optional (`pip install arkruntime`): BytePlus's own `Ark` client is not traced by traceAI-openai | + +```bash +.venv/bin/python -m pytest -q +``` + +## What is real and what is a stand-in + +| Piece | Real | Stand-in | +| --- | --- | --- | +| `openai` SDK, `traceAI-openai`, `fi-instrumentation-otel`, OTLP export | yes | | +| Future AGI collector | yours | the tests use a recording collector | +| The ModelArk API | only if you set `ARK_API_KEY` without `ARK_BASE_URL` | `standin/byteplus_modelark_stub.py`, shaped from BytePlus's Chat API reference. Nothing in it was captured from a real BytePlus response. The `{"error": {...}}` envelope is an assumption | +| BytePlus hostnames | the client dials them in `tests/test_regions.py` | a refusing proxy answers 403; no tunnel opens | + +What a live run is still needed for: the exact `model` value ModelArk returns for an endpoint id, whether +reasoning tokens are inside `usage.completion_tokens`, the real error body for a cross-region key, and +whether the real SSE stream matches the stand-in's. + +## Versions + +Tested 2026-10-15 on Python 3.13: traceAI-openai 0.1.10, fi-instrumentation-otel 1.1.0, openai 3.31.0 (latest) +and 1.109.1, opentelemetry-sdk 1.45.1. The suite also passes with traceAI PR 249 (open) overlaid; the two +assertions that differ are written to accept both. diff --git a/examples/providers/byteplus/app.py b/examples/providers/byteplus/app.py new file mode 100755 index 000000000..0731c5ed9 --- /dev/null +++ b/examples/providers/byteplus/app.py @@ -0,0 +1,209 @@ +#!/usr/bin/env python3 +"""Trace BytePlus ModelArk chat completions with traceAI-openai. + +This is the recipe from docs/integrations/providers/byteplus, plus a few small +scenarios so it can be run end to end: + + chat one non-streaming chat completion + stream one streaming call (asks for the usage chunk BytePlus documents) + tools a tool round trip: model asks for a tool, the app runs it, the model answers + thinking a call that passes BytePlus's `thinking` field through extra_body + error a call with a wrong API key (the failure case) + +The integration is only these lines (everything else is demo scaffolding): + + trace_provider = register(project_type=ProjectType.OBSERVE, project_name=...) + OpenAIInstrumentor().instrument(tracer_provider=trace_provider) + client = OpenAI(api_key=, base_url="https://ark.ap-southeast.bytepluses.com/api/v3") + +Environment + ARK_API_KEY BytePlus ModelArk API key. Goes to the OpenAI client only. + ARK_MODEL Model ID (or endpoint ID) from your ModelArk console. Not frozen here. + ARK_REGION ap (default, Johor) or eu (Dublin). Must match the region that issued the key. + ARK_BASE_URL Override of the regional URL. Use it only to point at a local + stand-in; a real setup leaves it unset. A URL that contains + /api/coding is refused (that is the Coding Plan, not this recipe). + FI_API_KEY, FI_SECRET_KEY, FI_BASE_URL Future AGI. Go to the tracer only. + FI_PROJECT_NAME Observe project (default: byteplus). + FI_HIDE_INPUTS, FI_HIDE_OUTPUTS, FI_HIDE_LLM_INVOCATION_PARAMETERS + Set to true to keep prompts, completions and request + parameters out of the spans (traceAI TraceConfig). +""" +import argparse +import json +import os +import sys +import uuid + +import openai +from fi_instrumentation import FITracer, TraceConfig, register, using_session +from fi_instrumentation.fi_types import FiSpanKindValues, ProjectType +from openai import OpenAI +from traceai_openai import OpenAIInstrumentor + +# The two ModelArk inference regions BytePlus documents (docs: Region availability). +# They are not interchangeable: a key and an inference endpoint belong to one region. +REGION_URLS = { + "ap": "https://ark.ap-southeast.bytepluses.com/api/v3", # Johor, Asia Pacific, ap-southeast-1 + "eu": "https://ark.eu-west.bytepluses.com/api/v3", # Dublin, Europe, eu-west-1 +} +DEFAULT_REGION = "ap" + +TOOLS = [ + { + "type": "function", + "function": { + "name": "lookup_record", + "description": "Look up a record by a free-text query.", + "parameters": { + "type": "object", + "properties": {"query": {"type": "string"}}, + "required": ["query"], + }, + }, + } +] + + +def lookup_record(query): + return f"record for {query}" + + +def resolve_base_url(env=None): + """The regional URL, or the stand-in override. Returns (base_url, error_message).""" + env = os.environ if env is None else env + override = env.get("ARK_BASE_URL") + region = (env.get("ARK_REGION") or DEFAULT_REGION).lower() + if region not in REGION_URLS: + return None, f"ARK_REGION must be one of {sorted(REGION_URLS)}, got {region!r}" + base_url = override or REGION_URLS[region] + if "/api/coding" in base_url: + return None, ( + "refusing a Coding Plan URL (contains /api/coding). That is a different BytePlus product and is " + "not covered by this recipe; use the ModelArk inference URL for your region." + ) + return base_url, None + + +def _messages(prompt, system): + return ([{"role": "system", "content": system}] if system else []) + [{"role": "user", "content": prompt}] + + +def run_chat(client, model, prompt, system): + reply = client.chat.completions.create(model=model, messages=_messages(prompt, system)) + return reply.choices[0].message.content + + +def run_stream(client, model, prompt, system, include_usage=True): + extra = {"stream_options": {"include_usage": True}} if include_usage else {} # documented in the ModelArk Chat API + stream = client.chat.completions.create(model=model, messages=_messages(prompt, system), stream=True, **extra) + return "".join(part.choices[0].delta.content or "" for part in stream if part.choices) + + +def run_thinking(client, model, prompt, system, mode="disabled"): + # `thinking` is a ModelArk field the OpenAI SDK does not know; extra_body carries it. + # BytePlus documents "enabled" and "disabled"; which values a model accepts varies by model. + reply = client.chat.completions.create( + model=model, + messages=_messages(prompt, system), + extra_body={"thinking": {"type": mode}}, + ) + return reply.choices[0].message.content + + +def run_tools(client, model, prompt, system, tracer): + messages = _messages(prompt, system) + first = client.chat.completions.create( + model=model, + messages=messages, + tools=TOOLS, + tool_choice={"type": "function", "function": {"name": "lookup_record"}}, + ) + message = first.choices[0].message + if not message.tool_calls: + return message.content + call = message.tool_calls[0] + # Your own spans carry only what you put on them. This one records no arguments. + with tracer.start_as_current_span("tool lookup_record", fi_span_kind=FiSpanKindValues.TOOL): + result = lookup_record(**json.loads(call.function.arguments)) + messages += [ + {"role": "assistant", "content": message.content or "", "tool_calls": [c.model_dump() for c in message.tool_calls]}, + {"role": "tool", "tool_call_id": call.id, "content": result}, + ] + final = client.chat.completions.create(model=model, messages=messages, tools=TOOLS) + return final.choices[0].message.content + + +def run_error(base_url, model, prompt): + """A deliberately wrong API key: ModelArk answers 401 AuthenticationError and the span is ERROR.""" + bad = OpenAI(api_key="not-a-real-ark-key", base_url=base_url) + try: + bad.chat.completions.create(model=model, messages=[{"role": "user", "content": prompt}]) + except openai.APIStatusError as exc: + return f"HTTP {exc.status_code}" + return "no error (unexpected)" + + +def main(argv=None): + ap = argparse.ArgumentParser(description=__doc__.split("\n\n")[0]) + ap.add_argument("--scenario", choices=["all", "chat", "stream", "tools", "thinking", "error"], default="all") + ap.add_argument("--prompt", default="Say hello to BytePlus ModelArk in five words.") + ap.add_argument("--system", default="") + ap.add_argument("--thinking", choices=["disabled", "enabled"], default="disabled", help="value for the thinking scenario") + ap.add_argument("--no-stream-usage", action="store_true", help="stream without stream_options.include_usage (the span then has no token counts)") + ap.add_argument("--session-id", default=None) + a = ap.parse_args(argv) + + api_key = os.environ.get("ARK_API_KEY") + model = os.environ.get("ARK_MODEL") + if not api_key or not model: + print("set ARK_API_KEY and ARK_MODEL (a Model ID or endpoint ID from your ModelArk console)", file=sys.stderr) + return 2 + base_url, problem = resolve_base_url() + if problem: + print(problem, file=sys.stderr) + return 2 + + # 1. Future AGI tracing. The Future AGI key pair is read from FI_API_KEY / FI_SECRET_KEY. + trace_provider = register( + project_type=ProjectType.OBSERVE, + project_name=os.environ.get("FI_PROJECT_NAME", "byteplus"), + ) + OpenAIInstrumentor().instrument(tracer_provider=trace_provider) + + # 2. The OpenAI client, pointed at the BytePlus region that issued the key. + client = OpenAI(api_key=api_key, base_url=base_url) + + tracer = FITracer(trace_provider.get_tracer("byteplus-example"), config=TraceConfig()) + session_id = a.session_id or f"byteplus-{uuid.uuid4()}" + wanted = ["chat", "stream", "tools", "thinking", "error"] if a.scenario == "all" else [a.scenario] + # Optional app-level parent span and session, so the calls are grouped in Observe. + with using_session(session_id), tracer.start_as_current_span( + "byteplus-demo", fi_span_kind=FiSpanKindValues.AGENT + ) as root: + trace_id = str(uuid.UUID(int=root.get_span_context().trace_id)) + for name in wanted: + try: + if name == "chat": + out = run_chat(client, model, a.prompt, a.system) + elif name == "stream": + out = run_stream(client, model, a.prompt, a.system, include_usage=not a.no_stream_usage) + elif name == "tools": + out = run_tools(client, model, a.prompt, a.system, tracer) + elif name == "thinking": + out = run_thinking(client, model, a.prompt, a.system, a.thinking) + else: + out = run_error(base_url, model, a.prompt) + except openai.OpenAIError as exc: # keep the demo going; the span already records the failure + out = f"{type(exc).__name__}" + print(f"RESULT {name}: {out}") + trace_provider.force_flush() + trace_provider.shutdown() + print(f"base_url={base_url}") + print(f"trace_id={trace_id}") + print(f"session_id={session_id}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/examples/providers/byteplus/pytest.ini b/examples/providers/byteplus/pytest.ini new file mode 100644 index 000000000..0e02926da --- /dev/null +++ b/examples/providers/byteplus/pytest.ini @@ -0,0 +1,3 @@ +[pytest] +testpaths = tests +addopts = -p no:cacheprovider diff --git a/examples/providers/byteplus/requirements.txt b/examples/providers/byteplus/requirements.txt new file mode 100644 index 000000000..984dc7eed --- /dev/null +++ b/examples/providers/byteplus/requirements.txt @@ -0,0 +1,6 @@ +# Tested 2026-10-15 on Python 3.13 (traceAI-openai 0.1.10 requires Python < 3.14). +traceAI-openai==0.1.10 +fi-instrumentation-otel==1.1.0 +openai==3.31.0 +# tests +pytest>=8 diff --git a/examples/providers/byteplus/run_e2e.sh b/examples/providers/byteplus/run_e2e.sh new file mode 100755 index 000000000..84c27147b --- /dev/null +++ b/examples/providers/byteplus/run_e2e.sh @@ -0,0 +1,88 @@ +#!/usr/bin/env bash +# End-to-end run of the BytePlus ModelArk recipe against a Future AGI collector. +# +# FI_API_KEY=... FI_SECRET_KEY=... FI_BASE_URL=http://localhost:4318 ./run_e2e.sh +# +# What it does +# 1. creates ./.venv (Python 3.12 or 3.13: traceAI-openai needs < 3.14) and installs requirements.txt +# (set VENV_DIR to reuse an existing environment) +# 2. runs the tests (they use the local stand-in, a recording collector and a refusing CONNECT +# probe; no Future AGI and no BytePlus needed) +# 3. runs app.py (chat, stream, tools, thinking, error) with tracing on to FI_BASE_URL +# 4. PRIVACY=1: runs app.py twice more with a unique marker in the prompts, once with content +# capture off (FI_HIDE_*) and once with the default, into -private and -optin, +# and prints the markers so you can search your own Future AGI store for them +# +# Which API it calls +# default the local stand-in in standin/ (NOT BytePlus) +# ARK_API_KEY + ARK_MODEL the real BytePlus ModelArk API for ARK_REGION (ap by default). +# Needs a BytePlus account with ModelArk activated; this script +# never creates one, and a real run can cost money. +# ARK_BASE_URL=... any OpenAI-compatible base URL (for example a local proxy running a +# real model); combine with ARK_API_KEY and ARK_MODEL. +# +# Other knobs: FI_PROJECT_NAME (default byteplus-e2e), SKIP_TESTS=1, APP_ARGS="--scenario chat". +set -euo pipefail + +HERE="$(cd "$(dirname "$0")" && pwd)" +cd "$HERE" +: "${FI_API_KEY:?set FI_API_KEY}" "${FI_SECRET_KEY:?set FI_SECRET_KEY}" +export FI_BASE_URL="${FI_BASE_URL:-http://localhost:4318}" +export FI_PROJECT_NAME="${FI_PROJECT_NAME:-byteplus-e2e}" + +VENV_DIR="${VENV_DIR:-$HERE/.venv}" +if [ ! -x "$VENV_DIR/bin/python" ]; then + PY="${PYTHON:-$(command -v python3.13 || command -v python3.12 || true)}" + [ -n "$PY" ] || { echo "need python3.12 or python3.13 on PATH (or set PYTHON)"; exit 2; } + "$PY" -m venv "$VENV_DIR" + "$VENV_DIR/bin/python" -m pip install --quiet --upgrade pip + "$VENV_DIR/bin/python" -m pip install --quiet -r requirements.txt +fi +PYBIN="$VENV_DIR/bin/python" +"$PYBIN" - <<'PY' +import importlib.metadata as m, sys +names = ["traceai-openai", "fi-instrumentation-otel", "openai", "opentelemetry-sdk"] +print("versions: python", sys.version.split()[0], *(f"{n}=={m.version(n)}" for n in names)) +PY + +if [ "${SKIP_TESTS:-0}" != "1" ]; then + "$PYBIN" -m pytest -q +fi + +STUB_PID="" +cleanup() { [ -n "$STUB_PID" ] && kill "$STUB_PID" 2>/dev/null || true; } +trap cleanup EXIT + +if [ -n "${ARK_API_KEY:-}" ]; then + : "${ARK_MODEL:?set ARK_MODEL (a Model ID or endpoint ID from your ModelArk console)}" + if [ -n "${ARK_BASE_URL:-}" ]; then + echo "target: custom base URL $ARK_BASE_URL (not necessarily BytePlus)" + else + echo "target: REAL BytePlus ModelArk, region ${ARK_REGION:-ap}" + fi +else + PORT="$("$PYBIN" -c 'import socket; s=socket.socket(); s.bind(("127.0.0.1",0)); print(s.getsockname()[1])')" + "$PYBIN" standin/byteplus_modelark_stub.py --port "$PORT" --api-key e2e-stand-in-key >/dev/null 2>&1 & + STUB_PID=$! + for _ in $(seq 1 30); do curl -fs "http://127.0.0.1:$PORT/healthz" >/dev/null 2>&1 && break; sleep 0.2; done + export ARK_BASE_URL="http://127.0.0.1:$PORT/api/v3" ARK_API_KEY=e2e-stand-in-key ARK_MODEL="${ARK_MODEL:-example-model-id}" + echo "target: STAND-IN (not BytePlus) at $ARK_BASE_URL" +fi + +echo "== app.py -> API, traces -> $FI_BASE_URL project=$FI_PROJECT_NAME" +"$PYBIN" app.py ${APP_ARGS:-} | grep -E '^(RESULT|base_url|trace_id|session_id)' + +if [ "${PRIVACY:-0}" = "1" ]; then + for mode in private optin; do + MARK="PRIV-$(date +%s)-$RANDOM$RANDOM" + if [ "$mode" = "private" ]; then + export FI_HIDE_INPUTS=true FI_HIDE_OUTPUTS=true FI_HIDE_LLM_INVOCATION_PARAMETERS=true + else + unset FI_HIDE_INPUTS FI_HIDE_OUTPUTS FI_HIDE_LLM_INVOCATION_PARAMETERS + fi + echo "== privacy run ($mode), marker $MARK, project ${FI_PROJECT_NAME}-$mode" + FI_PROJECT_NAME="${FI_PROJECT_NAME}-$mode" "$PYBIN" app.py --prompt "find $MARK" --system "system note $MARK" ${APP_ARGS:-} \ + | grep -E '^(RESULT|trace_id)' + echo "MARKER_${mode}=$MARK" + done +fi diff --git a/examples/providers/byteplus/standin/byteplus_modelark_stub.py b/examples/providers/byteplus/standin/byteplus_modelark_stub.py new file mode 100755 index 000000000..1a728d752 --- /dev/null +++ b/examples/providers/byteplus/standin/byteplus_modelark_stub.py @@ -0,0 +1,276 @@ +#!/usr/bin/env python3 +"""Local STAND-IN for the BytePlus ModelArk Chat API. + +This is NOT BytePlus. It is a small HTTP server shaped like the Chat API that +BytePlus documents at +https://ai.byteplus.com/ark/region:ap-southeast-1/docs/chat-api +(`POST /api/v3/chat/completions`, `Authorization: Bearer `, the +documented response fields, SSE chunks that end with `data: [DONE]`, and the +error codes table at .../docs/error-codes). It exists so the recipe in +../app.py can be exercised without a BytePlus account, an API key or any +spend. Nothing here was captured from a real BytePlus response. Fields the +documentation lists are produced; fields it does not list are not. + +Behaviour (deterministic, no model is called): + +* The reply echoes the last user or tool message, so a unique marker in a + prompt shows up in the response, in tool-call arguments and in tool results. +* A request that carries "tools" and no "tool" message yet returns one tool + call whose arguments are {"query": }. +* `thinking.type` other than "disabled" adds `reasoning_content` (and + `completion_tokens_details.reasoning_tokens`) to the response, as the Chat + API documents for models that support deep reasoning. "disabled" omits it. +* A wrong bearer token returns 401 / AuthenticationError, an unknown model + (name starting with "does-not-exist") returns 404 / + InvalidEndpointOrModel.NotFound. Both use the code, type and message from + BytePlus's error-codes table. The `{"error": {code, message, param, type}}` + envelope is NOT shown on the pages that were read; it is an assumption + (OpenAI-style) and the recipe does not depend on it. +* "stream": true returns SSE. A usage chunk (empty `choices`, full `usage`) is + added before `[DONE]` only when `stream_options.include_usage` is true, as + the Chat API documents. Without it every chunk carries `"usage": null`. + +The server records what it received (header names, the Authorization value, +the request body) and serves it at GET /__requests so tests can assert that +Future AGI keys never reach BytePlus. +""" +import argparse +import json +import os +import threading +import time +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer + +CHAT_PATH = "/api/v3/chat/completions" +WORD = " " + + +def _count_tokens(text): + return len(str(text).split()) + + +def _message_text(message): + content = message.get("content") + if isinstance(content, list): + return " ".join(p.get("text", "") for p in content if isinstance(p, dict)) + return content or "" + + +def request_id(counter=[0]): + counter[0] += 1 + # BytePlus request ids in the docs look like "0217426318107460cfa43dc3f3683b1de1c09624ff49085a456ac" + return f"02{int(time.time() * 1000):013d}{counter[0]:04d}stub" + + +class Stub: + def __init__(self, api_key, response_model=None): + self.api_key = api_key + self.response_model = response_model + self.requests = [] + self.lock = threading.Lock() + + def record(self, path, headers, body): + lowered = {k.lower(): v for k, v in headers.items()} + with self.lock: + self.requests.append( + { + "path": path, + "header_names": sorted(lowered), + "authorization": lowered.get("authorization"), + "headers": lowered, + "body": body, + } + ) + + +def make_handler(stub): + class Handler(BaseHTTPRequestHandler): + protocol_version = "HTTP/1.1" + + def log_message(self, *args): # keep test output quiet + pass + + def _send_json(self, status, payload): + data = json.dumps(payload).encode() + self.send_response(status) + self.send_header("Content-Type", "application/json") + self.send_header("Content-Length", str(len(data))) + self.end_headers() + self.wfile.write(data) + + def _error(self, status, code, etype, message): + rid = request_id() + self._send_json( + status, + {"error": {"code": code, "message": f"{message} Request id: {rid}", "param": "", "type": etype}}, + ) + + def do_GET(self): + if self.path == "/__requests": + with stub.lock: + return self._send_json(200, stub.requests) + if self.path == "/healthz": + return self._send_json(200, {"ok": True}) + self._error(404, "NotFound", "NotFound", "The requested resource does not exist.") + + def do_POST(self): + length = int(self.headers.get("Content-Length") or 0) + raw = self.rfile.read(length) if length else b"" + try: + body = json.loads(raw or b"{}") + except ValueError: + body = {"_unparsed": raw.decode("utf-8", "replace")} + stub.record(self.path, dict(self.headers.items()), body) + if self.headers.get("Authorization") != f"Bearer {stub.api_key}": + return self._error( + 401, "AuthenticationError", "Unauthorized", "The API key or AK/SK in the request is missing or invalid." + ) + if self.path.rstrip("/") != CHAT_PATH: + return self._error(404, "NotFound", "NotFound", "The requested resource does not exist.") + model = body.get("model") + if not model: + return self._error( + 400, + "MissingParameter", + "BadRequest", + "The request failed because it is missing one or multiple required parameters.", + ) + if str(model).startswith("does-not-exist"): + return self._error( + 404, + "InvalidEndpointOrModel.NotFound", + "NotFound", + f"The model or endpoint {model} does not exist or you do not have access to it.", + ) + return self._complete(body, model) + + def _complete(self, body, model): + messages = body.get("messages") or [] + last = messages[-1] if messages else {} + last_text = _message_text(last) + prompt_tokens = sum(_count_tokens(_message_text(m)) for m in messages) or 1 + response_model = stub.response_model or model + call_id = request_id() + created = int(time.time()) + thinking = (body.get("thinking") or {}).get("type") + reasons = thinking is not None and thinking != "disabled" + reasoning = "Reasoning about the request before answering." if reasons else None + reasoning_tokens = _count_tokens(reasoning) if reasoning else 0 + wants_tool = bool(body.get("tools")) and last.get("role") != "tool" + if wants_tool: + message = { + "role": "assistant", + "content": "", + "tool_calls": [ + { + "id": "call_stub_1", + "type": "function", + "function": { + "name": body["tools"][0]["function"]["name"], + "arguments": json.dumps({"query": last_text}), + }, + } + ], + } + finish, completion_tokens = "tool_calls", 6 + else: + message = {"role": "assistant", "content": f"echo: {last_text}"} + finish, completion_tokens = "stop", _count_tokens(message["content"]) + if reasoning: + message["reasoning_content"] = reasoning + completion_tokens += reasoning_tokens + usage = { + "completion_tokens": completion_tokens, + "prompt_tokens": prompt_tokens, + "total_tokens": prompt_tokens + completion_tokens, + "prompt_tokens_details": {"cached_tokens": 0}, + "completion_tokens_details": {"reasoning_tokens": reasoning_tokens}, + } + if not body.get("stream"): + return self._send_json( + 200, + { + "choices": [{"finish_reason": finish, "index": 0, "logprobs": None, "message": message}], + "created": created, + "id": call_id, + "model": response_model, + "service_tier": "default", + "object": "chat.completion", + "usage": usage, + }, + ) + self.send_response(200) + self.send_header("Content-Type", "text/event-stream") + self.send_header("Cache-Control", "no-cache") + self.send_header("Transfer-Encoding", "chunked") + self.end_headers() + + def chunk(payload): + data = ("data: " + payload + "\n\n").encode() + self.wfile.write(f"{len(data):x}\r\n".encode() + data + b"\r\n") + self.wfile.flush() + + def event(delta, finish_reason=None, usage_value=None, choices=True): + choice = {"delta": delta, "index": 0} + if finish_reason: + choice["finish_reason"] = finish_reason + payload = { + "choices": [choice] if choices else [], + "created": created, + "id": call_id, + "model": response_model, + "service_tier": "default", + "object": "chat.completion.chunk", + "usage": usage_value, + } + chunk(json.dumps(payload)) + + if reasoning: + event({"content": "", "reasoning_content": reasoning, "role": "assistant"}) + if wants_tool: + tc = message["tool_calls"][0] + event({"role": "assistant", "content": "", "tool_calls": [{"index": 0, "id": tc["id"], "type": "function", "function": {"name": tc["function"]["name"], "arguments": ""}}]}) + event({"tool_calls": [{"index": 0, "function": {"arguments": tc["function"]["arguments"]}}]}) + else: + words = message["content"].split(WORD) + event({"content": "", "role": "assistant"}) + for i, w in enumerate(words): + event({"content": w if i == 0 else WORD + w, "role": "assistant"}) + event({"content": "", "role": "assistant"}, finish_reason=finish) + if (body.get("stream_options") or {}).get("include_usage"): + event(None, usage_value=usage, choices=False) + chunk("[DONE]") + self.wfile.write(b"0\r\n\r\n") + self.wfile.flush() + self.close_connection = True + + return Handler + + +def serve(host="127.0.0.1", port=0, api_key="stub-ark-key", response_model=None): + stub = Stub(api_key, response_model) + server = ThreadingHTTPServer((host, port), make_handler(stub)) + server.stub = stub + return server + + +def main(): + ap = argparse.ArgumentParser(description=__doc__.split("\n\n")[0]) + ap.add_argument("--host", default="127.0.0.1") + ap.add_argument("--port", type=int, default=8789) + ap.add_argument("--api-key", default=os.environ.get("STUB_API_KEY", "stub-ark-key")) + ap.add_argument("--response-model", default=os.environ.get("STUB_RESPONSE_MODEL")) + a = ap.parse_args() + server = serve(a.host, a.port, a.api_key, a.response_model) + print( + f"STAND-IN for BytePlus ModelArk listening on http://{a.host}:{server.server_address[1]}/api/v3 (not BytePlus)", + flush=True, + ) + try: + server.serve_forever() + except KeyboardInterrupt: + pass + + +if __name__ == "__main__": + main() diff --git a/examples/providers/byteplus/standin/connect_probe.py b/examples/providers/byteplus/standin/connect_probe.py new file mode 100644 index 000000000..0351896d2 --- /dev/null +++ b/examples/providers/byteplus/standin/connect_probe.py @@ -0,0 +1,54 @@ +#!/usr/bin/env python3 +"""A refusing HTTP proxy that records where a client tries to connect. + +Point a process at it with HTTPS_PROXY=http://127.0.0.1:. An HTTPS client +that is configured with the real BytePlus base URL asks the proxy for +`CONNECT ark.ap-southeast.bytepluses.com:443`. This server writes that target +down and answers 403, so no tunnel is opened and nothing leaves the machine. + +It proves one thing: which host and port the unmodified recipe client dials. +It does not speak to BytePlus and it does not forward anything. +""" +import socketserver +import threading + + +class _Handler(socketserver.StreamRequestHandler): + def handle(self): + line = self.rfile.readline(65536).decode("latin-1").strip() + while True: # drain the header block + header = self.rfile.readline(65536) + if header in (b"\r\n", b"\n", b""): + break + method, _, rest = line.partition(" ") + target = rest.rsplit(" ", 1)[0] + self.server.seen.append({"method": method, "target": target}) + body = b"refused by connect_probe (no tunnel is opened)\n" + self.wfile.write( + b"HTTP/1.1 403 Forbidden\r\nContent-Type: text/plain\r\nConnection: close\r\n" + + f"Content-Length: {len(body)}\r\n\r\n".encode() + + body + ) + + +class ConnectProbe(socketserver.ThreadingTCPServer): + allow_reuse_address = True + daemon_threads = True + + def __init__(self, host="127.0.0.1", port=0): + super().__init__((host, port), _Handler) + self.seen = [] + + @property + def url(self): + return f"http://127.0.0.1:{self.server_address[1]}" + + def start(self): + threading.Thread(target=self.serve_forever, daemon=True).start() + return self + + +if __name__ == "__main__": + probe = ConnectProbe(port=0).start() + print(probe.url, flush=True) + threading.Event().wait() diff --git a/examples/providers/byteplus/tests/conftest.py b/examples/providers/byteplus/tests/conftest.py new file mode 100644 index 000000000..84242c110 --- /dev/null +++ b/examples/providers/byteplus/tests/conftest.py @@ -0,0 +1,176 @@ +"""Shared fixtures: the ModelArk-shaped stand-in, a recording OTLP collector, a refusing +CONNECT probe, and a runner that launches ../app.py exactly as a customer would (a +separate process).""" +import gzip +import json +import os +import subprocess +import sys +import threading +import urllib.request +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer +from pathlib import Path + +import pytest +from google.protobuf.json_format import MessageToDict +from opentelemetry.proto.collector.trace.v1 import trace_service_pb2 + +HERE = Path(__file__).resolve().parent +ROOT = HERE.parent +sys.path.insert(0, str(ROOT / "standin")) + +import byteplus_modelark_stub # noqa: E402 +import connect_probe # noqa: E402 + +ARK_KEY = "test-ark-key-0001" +FI_API_KEY = "test-fi-api-key-0002" +FI_SECRET_KEY = "test-fi-secret-key-0003" +MODEL = "example-model-id" +AP_URL = "https://ark.ap-southeast.bytepluses.com/api/v3" +EU_URL = "https://ark.eu-west.bytepluses.com/api/v3" + + +class Collector: + """Records every OTLP/HTTP trace export it receives, like fi-collector would see it.""" + + def __init__(self): + self.exports = [] + self.lock = threading.Lock() + collector = self + + class Handler(BaseHTTPRequestHandler): + protocol_version = "HTTP/1.1" + + def log_message(self, *args): + pass + + def do_POST(self): + body = self.rfile.read(int(self.headers.get("Content-Length") or 0)) + with collector.lock: + collector.exports.append( + {"path": self.path, "headers": {k.lower(): v for k, v in self.headers.items()}, "body": body} + ) + out = trace_service_pb2.ExportTraceServiceResponse().SerializeToString() + self.send_response(200) + self.send_header("Content-Type", "application/x-protobuf") + self.send_header("Content-Length", str(len(out))) + self.end_headers() + self.wfile.write(out) + + self.server = ThreadingHTTPServer(("127.0.0.1", 0), Handler) + self.url = f"http://127.0.0.1:{self.server.server_address[1]}" + threading.Thread(target=self.server.serve_forever, daemon=True).start() + + def close(self): + self.server.shutdown() + + # decoded views ----------------------------------------------------------- + def requests(self): + out = [] + for e in self.exports: + body = e["body"] + if e["headers"].get("content-encoding") == "gzip": + body = gzip.decompress(body) + msg = trace_service_pb2.ExportTraceServiceRequest() + msg.ParseFromString(body) + out.append(MessageToDict(msg, preserving_proto_field_name=True)) + return out + + def spans(self): + found = [] + for req in self.requests(): + for rs in req.get("resource_spans", []): + resource = {a["key"]: _any(a["value"]) for a in rs.get("resource", {}).get("attributes", [])} + for ss in rs.get("scope_spans", []): + for sp in ss.get("spans", []): + sp = dict(sp) + sp["_resource"] = resource + sp["_attrs"] = {a["key"]: _any(a["value"]) for a in sp.get("attributes", [])} + found.append(sp) + return found + + def raw_texts(self): + """Every byte the collector received, as text, for a marker search.""" + return [e["body"].decode("utf-8", "replace") for e in self.exports] + [ + json.dumps(r) for r in self.requests() + ] + + +def _any(value): + for k in ("string_value", "int_value", "double_value", "bool_value"): + if k in value: + return value[k] + return json.dumps(value) + + +def kind(span): + return span["_attrs"].get("gen_ai.span.kind") + + +def llm_spans(collector): + return sorted((s for s in collector.spans() if kind(s) == "LLM"), key=lambda s: int(s["start_time_unix_nano"])) + + +@pytest.fixture() +def stub(): + server = byteplus_modelark_stub.serve(api_key=ARK_KEY) + threading.Thread(target=server.serve_forever, daemon=True).start() + server.root = f"http://127.0.0.1:{server.server_address[1]}" + server.url = server.root + "/api/v3" + yield server + server.shutdown() + + +@pytest.fixture() +def collector(): + c = Collector() + yield c + c.close() + + +@pytest.fixture() +def probe(): + p = connect_probe.ConnectProbe().start() + yield p + p.shutdown() + + +def clean_env(): + return {k: v for k, v in os.environ.items() if not k.startswith(("FI_", "ARK_", "OPENAI", "OTEL_", "HTTP_PROXY", "HTTPS_PROXY", "ALL_PROXY", "NO_PROXY", "http_proxy", "https_proxy", "all_proxy", "no_proxy"))} + + +@pytest.fixture() +def run_app(stub, collector): + def _run(*args, env=None, base_url=None, fi_base_url=None, use_stub=True): + e = clean_env() + e.update( + { + "ARK_API_KEY": ARK_KEY, + "ARK_MODEL": MODEL, + "FI_API_KEY": FI_API_KEY, + "FI_SECRET_KEY": FI_SECRET_KEY, + "FI_BASE_URL": fi_base_url or collector.url, + "FI_PROJECT_NAME": "byteplus-test", + "OTEL_EXPORTER_OTLP_TRACES_TIMEOUT": "3", + "PYTHONDONTWRITEBYTECODE": "1", + "NO_PROXY": "127.0.0.1,localhost", + } + ) + if use_stub: + e["ARK_BASE_URL"] = base_url or stub.url + e.update(env or {}) + return subprocess.run( + [sys.executable, str(ROOT / "app.py"), *args], + cwd=ROOT, + env=e, + capture_output=True, + text=True, + timeout=120, + ) + + return _run + + +def ark_requests(stub): + with urllib.request.urlopen(stub.root + "/__requests") as r: + return json.load(r) diff --git a/examples/providers/byteplus/tests/test_ark_sdk.py b/examples/providers/byteplus/tests/test_ark_sdk.py new file mode 100644 index 000000000..df1c8d6f9 --- /dev/null +++ b/examples/providers/byteplus/tests/test_ark_sdk.py @@ -0,0 +1,36 @@ +"""Optional: BytePlus's own SDK (`pip install arkruntime`) is not covered by traceAI-openai. + +The docs page says so. Skipped when arkruntime is not installed.""" +import pytest + +arkruntime = pytest.importorskip("arkruntime") + + +def test_ark_client_is_not_an_openai_client_and_produces_no_spans(stub): + import openai + from opentelemetry.sdk.trace import TracerProvider + from opentelemetry.sdk.trace.export import SimpleSpanProcessor + from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter + from traceai_openai import OpenAIInstrumentor + + from conftest import ARK_KEY + + exporter = InMemorySpanExporter() + provider = TracerProvider() + provider.add_span_processor(SimpleSpanProcessor(exporter)) + instrumentor = OpenAIInstrumentor() + instrumentor.instrument(tracer_provider=provider) + try: + ark = arkruntime.Ark(api_key=ARK_KEY, base_url=stub.url) + assert not isinstance(ark, openai.OpenAI) + reply = ark.chat.completions.create(model="m", messages=[{"role": "user", "content": "hi"}]) + assert reply.choices[0].message.content == "echo: hi" + assert exporter.get_finished_spans() == () + + # the same instrumentor does trace the stock client against the same server + openai.OpenAI(api_key=ARK_KEY, base_url=stub.url).chat.completions.create( + model="m", messages=[{"role": "user", "content": "hi"}] + ) + assert len(exporter.get_finished_spans()) == 1 + finally: + instrumentor.uninstrument() diff --git a/examples/providers/byteplus/tests/test_page.py b/examples/providers/byteplus/tests/test_page.py new file mode 100644 index 000000000..d37da2320 --- /dev/null +++ b/examples/providers/byteplus/tests/test_page.py @@ -0,0 +1,68 @@ +"""The Python snippet on the docs page, run as written. + +The test reads the page, takes the code block that builds the client, swaps only the +BytePlus base URL for the local stand-in, and runs it in a separate process like a +customer would. A second run keeps the URL as written and sends it through the refusing +CONNECT probe to show which host the snippet dials.""" +import re +import subprocess +import sys + +from conftest import AP_URL, ARK_KEY, FI_API_KEY, FI_SECRET_KEY, ROOT, clean_env, llm_spans + +PAGE = ROOT.parents[2] / "src" / "pages" / "docs" / "integrations" / "providers" / "byteplus.mdx" + + +def snippet(): + text = PAGE.read_text() + blocks = re.findall(r"```python\n(.*?)```", text, re.S) + chosen = [b for b in blocks if AP_URL in b and "chat.completions.create" in b] + assert len(chosen) == 1, "the page must have exactly one runnable Python recipe" + return chosen[0] + + +def run(code, env_extra, tmp_path): + env = clean_env() + env.update({"ARK_API_KEY": ARK_KEY, "FI_API_KEY": FI_API_KEY, "FI_SECRET_KEY": FI_SECRET_KEY, + "OTEL_EXPORTER_OTLP_TRACES_TIMEOUT": "3", "PYTHONDONTWRITEBYTECODE": "1", "NO_PROXY": "127.0.0.1,localhost"}) + env.update(env_extra) + script = tmp_path / "page_snippet.py" + script.write_text(code + "\nimport time\ntrace_provider.force_flush()\ntrace_provider.shutdown()\n") + return subprocess.run([sys.executable, "-I", str(script)], cwd=tmp_path, env=env, capture_output=True, text=True, timeout=120) + + +def test_the_page_snippet_runs_and_is_traced_on_the_named_project(stub, collector, tmp_path): + code = snippet().replace(AP_URL, stub.url) + result = run(code, {"FI_BASE_URL": collector.url}, tmp_path) + assert result.returncode == 0, result.stderr[-1500:] + assert "echo: What is the capital of France?" in result.stdout + (span,) = llm_spans(collector) + attrs = span["_attrs"] + assert attrs["gen_ai.request.model"] == "your-model-or-endpoint-id" + assert attrs["gen_ai.provider.name"] == "openai" + assert collector.spans()[0]["_resource"]["project_name"] == "my-chatbot" + assert collector.spans()[0]["_resource"]["project_type"] == "observe" + + +def test_the_page_snippet_as_written_dials_the_asia_pacific_host(collector, probe, tmp_path): + result = run(snippet(), {"FI_BASE_URL": collector.url, "HTTPS_PROXY": probe.url}, tmp_path) + assert result.returncode != 0 # the probe refuses the tunnel, the snippet's request fails + assert {p["target"] for p in probe.seen} == {"ark.ap-southeast.bytepluses.com:443"} + (span,) = llm_spans(collector) + assert span["status"]["code"] == "STATUS_CODE_ERROR" + + +def test_the_page_names_both_region_urls_and_the_coding_plan_warning(): + text = PAGE.read_text() + assert "https://ark.eu-west.bytepluses.com/api/v3" in text + assert "/api/coding" in text and "Coding Plan is not this page" in text + assert chr(0x2014) not in text, "the docs style guide forbids em dashes" + assert "ARK_API_KEY=\"YOUR_BYTEPLUS_ARK_API_KEY\"" in text # placeholders only, no key in the page + + +def test_qualification_and_release_scope_are_explicit(): + text = PAGE.read_text() + assert "ModelArk vendor qualification is pending" in text + assert "None is a BytePlus service receipt" in text + assert "merged to `dev` on 2026-10-28" in text + assert not re.search(r"not tested|untested|unverified", text, re.I) diff --git a/examples/providers/byteplus/tests/test_privacy.py b/examples/providers/byteplus/tests/test_privacy.py new file mode 100644 index 000000000..5c45e7611 --- /dev/null +++ b/examples/providers/byteplus/tests/test_privacy.py @@ -0,0 +1,86 @@ +"""Content capture: what the span export contains by default, and with it switched off. + +The marker is unique per run and is placed in the system prompt, the user prompt, and +(through the stand-in's echo) the model answer, the tool-call arguments and the tool +result. The search covers every byte the collector received and every decoded span +attribute, event and resource attribute.""" +import json +import uuid + +from conftest import MODEL + +HIDE_ALL = {"FI_HIDE_INPUTS": "true", "FI_HIDE_OUTPUTS": "true", "FI_HIDE_LLM_INVOCATION_PARAMETERS": "true"} + + +def marker(): + return f"MARKER-{uuid.uuid4().hex[:12]}" + + +def run_with_marker(run_app, env=None): + m = marker() + result = run_app("--prompt", f"find {m}", "--system", f"system note {m}", env=env) + assert result.returncode == 0, result.stderr[-1500:] + # the application itself still gets the full answer, tool call and tool result + assert f"RESULT chat: echo: find {m}" in result.stdout + assert f"RESULT tools: echo: record for find {m}" in result.stdout + return m + + +def all_exported_text(collector): + chunks = list(collector.raw_texts()) + for span in collector.spans(): + chunks.append(json.dumps(span["_resource"])) + chunks.append(json.dumps(span["_attrs"])) + chunks.append(json.dumps(span.get("events", []))) + chunks.append(json.dumps(span.get("status", {}))) + return "\n".join(chunks) + + +def test_prompts_and_answers_are_captured_by_default(run_app, collector): + m = run_with_marker(run_app) + text = all_exported_text(collector) + assert text.count(m) >= 8, "default capture should include prompts, answers, tool arguments and results" + + +def test_hiding_inputs_and_outputs_keeps_the_marker_out_of_every_exported_byte(run_app, collector): + m = run_with_marker(run_app, env=HIDE_ALL) + assert collector.exports + assert m not in all_exported_text(collector) + # the spans are still useful: model and usage survive on the non-streaming calls + chat = sorted( + (s for s in collector.spans() if s["_attrs"].get("gen_ai.span.kind") == "LLM"), + key=lambda s: int(s["start_time_unix_nano"]), + )[0] + assert chat["_attrs"]["gen_ai.request.model"] == MODEL + assert int(chat["_attrs"]["gen_ai.usage.input_tokens"]) > 0 + assert chat["_attrs"]["input.value"] == "__REDACTED__" + + +def test_invocation_parameters_follow_their_own_flag(run_app, collector): + run_with_marker(run_app, env={"FI_HIDE_INPUTS": "true", "FI_HIDE_OUTPUTS": "true"}) + assert any("gen_ai.request.parameters" in s["_attrs"] for s in collector.spans()) + collector.exports.clear() + run_with_marker(run_app, env=HIDE_ALL) + assert not any("gen_ai.request.parameters" in s["_attrs"] for s in collector.spans()) + + +def test_documented_residue_of_the_hide_flags(run_app, collector): + """What FI_HIDE_* does NOT remove on traceai-openai 0.1.10. The docs page lists each of + these. If one of these assertions starts failing the instrumentor got better: update the + page's privacy section.""" + run_with_marker(run_app, env=HIDE_ALL) + spans = collector.spans() + # developer-authored tool definitions (name, description, JSON schema) + assert any("Look up a record" in s["_attrs"].get("gen_ai.tool.definitions", "") for s in spans) + # session id + assert all(s["_attrs"].get("session.id") for s in spans) + # error text: status message and exception event (type, message, stack trace with local file paths) + failed = [s for s in spans if s.get("status", {}).get("code") == "STATUS_CODE_ERROR"] + assert failed and "401" in failed[0]["status"]["message"] + assert "AuthenticationError" in failed[0]["status"]["message"] + exc = [e for e in failed[0]["events"] if e["name"] == "exception"][0] + attrs = {a["key"]: next(iter(a["value"].values())) for a in exc["attributes"]} + assert {"exception.type", "exception.message", "exception.stacktrace"} <= set(attrs) + assert "site-packages" in attrs["exception.stacktrace"] # a local file path + # BytePlus's request id is part of the error text the span keeps + assert "Request id:" in attrs["exception.message"] diff --git a/examples/providers/byteplus/tests/test_recipe.py b/examples/providers/byteplus/tests/test_recipe.py new file mode 100644 index 000000000..75be87333 --- /dev/null +++ b/examples/providers/byteplus/tests/test_recipe.py @@ -0,0 +1,196 @@ +"""The recipe, run as a customer would run it, against the stand-in and a recording +collector. Every assertion is on bytes that really left the process.""" +import importlib.util +import json +import sys + +import pytest +from conftest import AP_URL, ARK_KEY, EU_URL, FI_API_KEY, FI_SECRET_KEY, MODEL, ROOT, ark_requests, kind, llm_spans + +HAS_PROVIDER_TABLE = importlib.util.find_spec("traceai_openai._providers") is not None + + +def load_app(): + sys.path.insert(0, str(ROOT)) + spec = importlib.util.spec_from_file_location("recipe_app", ROOT / "app.py") + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +def test_documented_region_urls_and_the_default(): + app = load_app() + assert app.REGION_URLS == {"ap": AP_URL, "eu": EU_URL} + assert app.DEFAULT_REGION == "ap" + assert app.resolve_base_url({}) == (AP_URL, None) + assert app.resolve_base_url({"ARK_REGION": "eu"}) == (EU_URL, None) + assert app.resolve_base_url({"ARK_REGION": "EU"}) == (EU_URL, None) + + +def test_there_is_no_third_host_and_no_cross_region_fallback(): + app = load_app() + assert set(app.REGION_URLS.values()) == {AP_URL, EU_URL} + url, problem = app.resolve_base_url({"ARK_REGION": "cn"}) + assert url is None and "ARK_REGION" in problem + + +@pytest.mark.parametrize( + "bad", + [ + "https://ark.ap-southeast.bytepluses.com/api/coding/v3", + "https://ark.ap-southeast.bytepluses.com/api/coding", + ], +) +def test_coding_plan_urls_are_refused_before_anything_is_traced(run_app, collector, stub, bad): + url, problem = load_app().resolve_base_url({"ARK_BASE_URL": bad}) + assert url is None and "Coding Plan" in problem + result = run_app(env={"ARK_BASE_URL": bad}) + assert result.returncode == 2 + assert "Coding Plan" in result.stderr + assert not collector.exports and not ark_requests(stub) + + +def test_every_scenario_runs_and_the_trace_has_the_expected_shape(run_app, collector): + result = run_app("--prompt", "hello byteplus") + assert result.returncode == 0, result.stderr[-1500:] + for name in ("chat", "stream", "tools", "thinking"): + assert f"RESULT {name}: echo:" in result.stdout + assert "RESULT error: HTTP 401" in result.stdout + + spans = collector.spans() + assert len({s["trace_id"] for s in spans}) == 1 + by_kind = {} + for s in spans: + by_kind.setdefault(kind(s), []).append(s) + assert len(by_kind["AGENT"]) == 1 and len(by_kind["TOOL"]) == 1 and len(by_kind["LLM"]) == 6 + root = by_kind["AGENT"][0] + assert not root.get("parent_span_id") + for child in by_kind["LLM"] + by_kind["TOOL"]: + assert child["parent_span_id"] == root["span_id"] + assert len({s["_attrs"].get("session.id") for s in spans}) == 1 + + +def test_chat_and_tool_calls_carry_model_and_usage(run_app, collector): + assert run_app().returncode == 0 + llm = llm_spans(collector) + chat, tool_first, tool_final = llm[0], llm[2], llm[3] + for span in (chat, tool_first, tool_final): + attrs = span["_attrs"] + assert attrs["gen_ai.request.model"] == MODEL + assert int(attrs["gen_ai.usage.input_tokens"]) > 0 + assert int(attrs["gen_ai.usage.output_tokens"]) > 0 + assert int(attrs["gen_ai.usage.total_tokens"]) == int(attrs["gen_ai.usage.input_tokens"]) + int( + attrs["gen_ai.usage.output_tokens"] + ) + assert span["status"]["code"] == "STATUS_CODE_OK" + assert "lookup_record" in tool_first["_attrs"]["gen_ai.tool.definitions"] + + +def test_streaming_with_the_documented_usage_option_has_tokens(run_app, collector, stub): + assert run_app("--scenario", "stream").returncode == 0 + (stream,) = llm_spans(collector) + attrs = stream["_attrs"] + assert int(attrs["gen_ai.usage.total_tokens"]) > 0 + assert [e["name"] for e in stream.get("events", [])] == ["First Token Stream Event"] + # the option really went to the (stand-in) API + assert ark_requests(stub)[0]["body"]["stream_options"] == {"include_usage": True} + # traceai-openai 0.1.10 leaves the model off streamed spans; traceAI PR 249 records it. + assert attrs.get("gen_ai.request.model") in (None, MODEL) + assert (attrs.get("gen_ai.request.model") == MODEL) == HAS_PROVIDER_TABLE + + +def test_streaming_without_the_usage_option_has_no_token_counts(run_app, collector, stub): + """The docs page tells readers to pass stream_options={"include_usage": True}. Without it + ModelArk sends no usage chunk, and the span has no token counts.""" + assert run_app("--scenario", "stream", "--no-stream-usage").returncode == 0 + assert "stream_options" not in ark_requests(stub)[0]["body"] + (stream,) = llm_spans(collector) + attrs = stream["_attrs"] + assert not any(k.startswith("gen_ai.usage.") for k in attrs) + assert stream["status"]["code"] == "STATUS_CODE_OK" + + +@pytest.mark.parametrize("mode", ["disabled", "enabled"]) +def test_thinking_in_extra_body_reaches_the_api_and_still_traces(run_app, collector, stub, mode): + """G3: the ModelArk `thinking` field rides in extra_body. The call is traced, and the field + is not a span attribute. With thinking enabled the reasoning text is not on the span either.""" + assert run_app("--scenario", "thinking", "--thinking", mode).returncode == 0 + assert ark_requests(stub)[0]["body"]["thinking"] == {"type": mode} + (span,) = llm_spans(collector) + attrs = span["_attrs"] + assert span["status"]["code"] == "STATUS_CODE_OK" + assert attrs["gen_ai.request.model"] == MODEL + assert "thinking" not in attrs.get("gen_ai.request.parameters", "") # extra_body is not captured + assert "Reasoning about the request" not in json.dumps(attrs) # nor is reasoning_content + if mode == "enabled": + # reasoning tokens are inside completion_tokens, which is what the span reports + assert int(attrs["gen_ai.usage.output_tokens"]) > len("echo: Say hello to BytePlus ModelArk in five words.".split()) + + +def test_failed_call_is_an_error_span(run_app, collector): + assert run_app("--scenario", "error").returncode == 0 + (failed,) = llm_spans(collector) + assert failed["status"]["code"] == "STATUS_CODE_ERROR" + assert "AuthenticationError" in failed["status"]["message"] and "401" in failed["status"]["message"] + assert failed["_attrs"].get("gen_ai.request.model") in (None, MODEL) + assert "exception" in [e["name"] for e in failed.get("events", [])] + + +def test_span_model_is_the_model_in_the_response_body(run_app, collector, stub): + """Observed with the stand-in only: when ModelArk answers with a different model id than the + one requested (a dated snapshot, say), the non-streaming span shows the response body's model.""" + stub.stub.response_model = "resolved-model-260101" + assert run_app("--scenario", "chat").returncode == 0 + (span,) = llm_spans(collector) + assert span["_attrs"]["gen_ai.request.model"] == "resolved-model-260101" + + +def test_provider_label_is_openai_for_the_byteplus_hosts(run_app, collector): + assert run_app("--scenario", "chat").returncode == 0 + assert {s["_attrs"]["gen_ai.provider.name"] for s in llm_spans(collector)} == {"openai"} + if HAS_PROVIDER_TABLE: + # traceAI PR 249 maps hosts to providers; neither BytePlus host is in its table. + from traceai_openai._providers import provider_from_host + + for host in ("ark.ap-southeast.bytepluses.com", "ark.eu-west.bytepluses.com"): + assert provider_from_host(host) == "openai" + + +def test_each_key_goes_only_where_it_belongs(run_app, collector, stub): + assert run_app().returncode == 0 + seen = ark_requests(stub) + assert seen and all(r["path"] == "/api/v3/chat/completions" for r in seen) + assert {r["authorization"] for r in seen} == {f"Bearer {ARK_KEY}", "Bearer not-a-real-ark-key"} + blob = json.dumps(seen) + assert FI_API_KEY not in blob and FI_SECRET_KEY not in blob + + assert collector.exports + for export in collector.exports: + assert export["headers"]["x-api-key"] == FI_API_KEY + assert export["headers"]["x-secret-key"] == FI_SECRET_KEY + assert "authorization" not in export["headers"] + assert not any(ARK_KEY in t for t in collector.raw_texts()) + + +def test_export_is_uncompressed_otlp_protobuf_with_project_resource(run_app, collector): + assert run_app().returncode == 0 + for export in collector.exports: + assert export["path"] == "/tracer/v1/traces" + assert export["headers"]["content-type"] == "application/x-protobuf" + assert "content-encoding" not in export["headers"] + resource = collector.spans()[0]["_resource"] + assert resource["project_name"] == "byteplus-test" + assert resource["project_type"] == "observe" + + +def test_an_unreachable_collector_never_fails_the_request(run_app): + result = run_app("--scenario", "chat", fi_base_url="http://127.0.0.1:9") + assert result.returncode == 0 + assert "RESULT chat: echo:" in result.stdout + + +def test_missing_configuration_exits_before_tracing(run_app, collector): + result = run_app(env={"ARK_MODEL": ""}) + assert result.returncode == 2 + assert "ARK_MODEL" in result.stderr + assert not collector.exports diff --git a/examples/providers/byteplus/tests/test_regions.py b/examples/providers/byteplus/tests/test_regions.py new file mode 100644 index 000000000..bbaeb9b21 --- /dev/null +++ b/examples/providers/byteplus/tests/test_regions.py @@ -0,0 +1,49 @@ +"""Where the unmodified recipe client dials when it uses the documented BytePlus URLs. + +No request leaves the machine: HTTPS_PROXY points at a probe that records the +`CONNECT host:port` the client asks for and answers 403. That is the strongest +statement possible about the BytePlus hosts without a BytePlus account: the +documented base URL makes the OpenAI client dial exactly that host, on 443.""" +import pytest +from conftest import kind, llm_spans + +AP_HOST = "ark.ap-southeast.bytepluses.com:443" +EU_HOST = "ark.eu-west.bytepluses.com:443" + + +@pytest.mark.parametrize("region, host", [("ap", AP_HOST), ("eu", EU_HOST)]) +def test_the_documented_url_dials_exactly_that_region(run_app, collector, probe, region, host): + result = run_app( + "--scenario", "chat", env={"ARK_REGION": region, "HTTPS_PROXY": probe.url}, use_stub=False + ) + assert result.returncode == 0, result.stderr[-1500:] + assert "RESULT chat: APIConnectionError" in result.stdout + assert probe.seen, "the client never tried to connect" + assert {p["target"] for p in probe.seen} == {host} + assert {p["method"] for p in probe.seen} == {"CONNECT"} + other = EU_HOST if host == AP_HOST else AP_HOST + assert other not in {p["target"] for p in probe.seen}, "the recipe must not fall back to the other region" + + +def test_the_default_region_is_asia_pacific(run_app, probe): + assert run_app("--scenario", "chat", env={"HTTPS_PROXY": probe.url}, use_stub=False).returncode == 0 + assert {p["target"] for p in probe.seen} == {AP_HOST} + + +@pytest.mark.parametrize("region", ["ap", "eu"]) +def test_either_region_url_is_traced_by_the_same_instrumentor(run_app, collector, probe, region): + assert run_app("--scenario", "chat", env={"ARK_REGION": region, "HTTPS_PROXY": probe.url}, use_stub=False).returncode == 0 + (span,) = llm_spans(collector) + assert span["status"]["code"] == "STATUS_CODE_ERROR" + assert "APIConnectionError" in span["status"]["message"] + assert span["_attrs"]["gen_ai.provider.name"] == "openai" + assert span["_attrs"]["gen_ai.span.kind"] == "LLM" + assert collector.spans()[0]["_resource"]["project_name"] == "byteplus-test" + assert {kind(s) for s in collector.spans()} == {"AGENT", "LLM"} + + +def test_the_span_does_not_record_the_base_url_or_region(run_app, collector, probe): + assert run_app("--scenario", "chat", env={"HTTPS_PROXY": probe.url}, use_stub=False).returncode == 0 + exported = "\n".join(collector.raw_texts()) + assert "bytepluses" not in exported + assert "ap-southeast" not in exported diff --git a/examples/providers/byteplus/tests/test_stub.py b/examples/providers/byteplus/tests/test_stub.py new file mode 100644 index 000000000..814220e82 --- /dev/null +++ b/examples/providers/byteplus/tests/test_stub.py @@ -0,0 +1,87 @@ +"""The stand-in only claims the parts of the ModelArk Chat API that BytePlus documents.""" +import json +import urllib.error +import urllib.request + +from conftest import ARK_KEY, MODEL + + +def _post(stub, payload, key=ARK_KEY, path="/chat/completions", root=None): + req = urllib.request.Request((root or stub.url) + path, data=json.dumps(payload).encode(), method="POST") + req.add_header("Content-Type", "application/json") + if key: + req.add_header("Authorization", f"Bearer {key}") + try: + with urllib.request.urlopen(req) as resp: + return resp.status, resp.read().decode() + except urllib.error.HTTPError as err: + return err.code, err.read().decode() + + +def test_chat_completion_has_the_documented_fields(stub): + status, body = _post(stub, {"model": MODEL, "messages": [{"role": "user", "content": "hi there"}]}) + data = json.loads(body) + assert status == 200 + assert data["object"] == "chat.completion" and data["model"] == MODEL and data["service_tier"] == "default" + assert data["choices"][0]["message"] == {"role": "assistant", "content": "echo: hi there"} + assert data["choices"][0]["finish_reason"] == "stop" and data["choices"][0]["logprobs"] is None + usage = data["usage"] + assert usage["total_tokens"] == usage["prompt_tokens"] + usage["completion_tokens"] + assert usage["prompt_tokens_details"] == {"cached_tokens": 0} + assert usage["completion_tokens_details"] == {"reasoning_tokens": 0} + + +def test_the_documented_path_is_the_only_path(stub): + assert _post(stub, {"model": MODEL, "messages": []}, path="/chat/completions")[0] == 200 + # the Coding Plan prefix is a different product and the stand-in does not serve it + status, _ = _post(stub, {"model": MODEL, "messages": []}, path="/api/coding/v3/chat/completions", root=stub.root) + assert status == 404 + + +def test_thinking_adds_reasoning_content_and_tokens_only_when_not_disabled(stub): + base = {"model": MODEL, "messages": [{"role": "user", "content": "q"}]} + off = json.loads(_post(stub, {**base, "thinking": {"type": "disabled"}})[1]) + on = json.loads(_post(stub, {**base, "thinking": {"type": "enabled"}})[1]) + assert "reasoning_content" not in off["choices"][0]["message"] + assert on["choices"][0]["message"]["reasoning_content"] + assert off["usage"]["completion_tokens_details"]["reasoning_tokens"] == 0 + assert on["usage"]["completion_tokens_details"]["reasoning_tokens"] > 0 + + +def test_errors_use_the_documented_code_type_and_message(stub): + status, body = _post(stub, {"messages": []}) + assert status == 400 + assert json.loads(body)["error"]["code"] == "MissingParameter" + for key in ("wrong", None): + status, body = _post(stub, {"model": MODEL, "messages": []}, key=key) + err = json.loads(body)["error"] + assert (status, err["code"], err["type"]) == (401, "AuthenticationError", "Unauthorized") + assert err["message"].startswith("The API key or AK/SK in the request is missing or invalid.") + assert "Request id:" in err["message"] + status, body = _post(stub, {"model": "does-not-exist-1", "messages": []}) + assert (status, json.loads(body)["error"]["code"]) == (404, "InvalidEndpointOrModel.NotFound") + + +def test_streaming_ends_with_done_and_usage_chunk_is_opt_in(stub): + base = {"model": MODEL, "stream": True, "messages": [{"role": "user", "content": "one two"}]} + _, plain = _post(stub, base) + assert plain.rstrip().endswith("data: [DONE]") + chunks = [json.loads(line[6:]) for line in plain.splitlines() if line.startswith("data: {")] + assert all(c["usage"] is None for c in chunks) # documented: usage is null unless asked for + assert chunks[-1]["choices"][0]["finish_reason"] == "stop" + _, with_usage = _post(stub, {**base, "stream_options": {"include_usage": True}}) + last = [json.loads(line[6:]) for line in with_usage.splitlines() if line.startswith("data: {")][-1] + assert last["choices"] == [] and last["usage"]["total_tokens"] > 0 + + +def test_tool_call_then_final_answer(stub): + tools = [{"type": "function", "function": {"name": "lookup_record", "parameters": {"type": "object"}}}] + _, first = _post(stub, {"model": MODEL, "tools": tools, "messages": [{"role": "user", "content": "find x"}]}) + call = json.loads(first)["choices"][0]["message"]["tool_calls"][0] + assert call["function"]["name"] == "lookup_record" + assert json.loads(call["function"]["arguments"]) == {"query": "find x"} + _, second = _post( + stub, + {"model": MODEL, "tools": tools, "messages": [{"role": "tool", "tool_call_id": call["id"], "content": "done"}]}, + ) + assert json.loads(second)["choices"][0]["finish_reason"] == "stop" diff --git a/src/lib/integrations.mjs b/src/lib/integrations.mjs index 040dfe275..a7a465b0a 100644 --- a/src/lib/integrations.mjs +++ b/src/lib/integrations.mjs @@ -39,6 +39,7 @@ export const integrations = [ { id: 'cursor', name: 'Cursor IDE', href: '/docs/integrations/coding-agents/cursor', category: 'coding', summary: 'Trace macOS IDE agent sessions through observation hooks.', method: 'Hook-to-OTLP', logo: 'generic-framework' }, { id: 'langsmith', name: 'LangSmith', href: '/docs/integrations/langsmith', category: 'frameworks', summary: 'Forward supported LangSmith SDK traces over OTLP.', method: 'Native OTLP', logo: 'generic-framework' }, { id: 'openai', name: 'OpenAI', href: '/docs/integrations/traceai/openai', category: 'providers', summary: 'Instrument OpenAI client calls in your application.', method: 'traceAI SDK', logo: 'openai' }, + { id: 'byteplus', name: 'BytePlus ModelArk', href: '/docs/integrations/providers/byteplus', category: 'providers', summary: 'Configure the Python OpenAI client for regional ModelArk endpoints.', method: 'OpenAI client', logo: 'generic-model' }, { id: 'anthropic', name: 'Anthropic', href: '/docs/integrations/traceai/anthropic', category: 'providers', summary: 'Capture Claude calls, token usage and latency.', method: 'traceAI SDK', logo: 'anthropic', aliases: ['Claude'] }, { id: 'bedrock', name: 'AWS Bedrock', href: '/docs/integrations/traceai/bedrock', category: 'providers', summary: 'Trace model calls through Amazon Bedrock.', method: 'traceAI SDK', logo: 'bedrock', aliases: ['Amazon'] }, { id: 'vertexai', name: 'Vertex AI', href: '/docs/integrations/traceai/vertexai', category: 'providers', summary: 'Instrument Gemini calls through Google Cloud.', method: 'traceAI SDK', logo: 'vertexai', aliases: ['Google', 'Gemini'] }, diff --git a/src/pages/docs/integrations/providers/byteplus.mdx b/src/pages/docs/integrations/providers/byteplus.mdx new file mode 100644 index 000000000..3a3c803c5 --- /dev/null +++ b/src/pages/docs/integrations/providers/byteplus.mdx @@ -0,0 +1,208 @@ +--- +title: "Trace BytePlus ModelArk chat completions" +description: "Trace BytePlus ModelArk calls made with the OpenAI Python client in Future AGI using traceAI-openai. Two regional base URLs, no new package. Vendor qualification pending." +--- + +## About + +BytePlus ModelArk serves an OpenAI-compatible chat API. You keep using `openai.OpenAI`, set `base_url` to the region that issued your key, and the existing [traceAI-openai](/docs/integrations/traceai/openai) instrumentor traces the call. There is no BytePlus package to install and nothing to configure on the BytePlus side. + +This page is not: + +- **A BytePlus export.** It traces calls your own code makes. It does not read BytePlus usage logs or billing, and it does not import history. +- **A new traceAI package.** There is no `traceai-byteplus`. The existing instrumentor handles the call, and [What is not captured](#what-is-not-captured) lists the gaps found. +- **Coding Plan setup.** Coding Plan is a different BytePlus product with a different base URL. See [Coding Plan is not this page](#coding-plan-is-not-this-page). +- **A recipe for BytePlus's own `arkruntime` SDK.** Its `Ark` client is not traced by traceAI-openai. See [What is not captured](#what-is-not-captured). + +The provider name on the span reads `openai`, not BytePlus. That is a display limit of the instrumentor, not something this page can change. + +
+ +| | | +| --- | --- | +| Tested with | traceAI-openai 0.1.10 (latest on PyPI, checked 2026-10-29), fi-instrumentation-otel 1.1.0, openai 3.31.0 and 1.109.1, Python 3.13. The traceAI-openai metadata allows Python 3.10 to 3.13. | +| BytePlus docs read | [Region availability](https://docs.byteplus.com/en/docs/modelark/2191806) (last updated 2026-09-10), [Common SDK usage examples](https://docs.byteplus.com/en/docs/ModelArk/1544136) (last updated 2026-09-28), [Compatible with OpenAI API](https://docs.byteplus.com/en/docs/modelark/1330626), [Base URL and authentication](https://docs.byteplus.com/docs/modelark/base-url-and-authentication), [Coding Plan Team quick start](https://docs.byteplus.com/en/docs/ModelArk/2277233) (last updated 2026-09-24) and the [Chat API reference](https://api.byteplus.com/api-explorer/?action=ChatCompletions&groupName=Chat%20API&serviceCode=ark&version=2024-01-01), all read 2026-10-15. Several of these pages render in the browser, so parts were read through search excerpts only. | +| Source | [examples/providers/byteplus](https://github.com/future-agi/docs/tree/main/examples/providers/byteplus) | + +
+ + +**ModelArk vendor qualification is pending; keep this recipe in draft.** The recorded evidence is the real `openai` SDK and traceAI-openai instrumentor against (1) a local ModelArk-shaped stand-in (`STANDIN`), (2) an OpenAI-compatible proxy running a real model, and (3) a refusing proxy showing which host the unmodified client dials. None is a BytePlus service receipt. A chat, streaming, tool-call and error round trip with an authorized ModelArk platform key is required before treating this as a qualified vendor integration. [Run it yourself](#run-it-yourself) shows the acceptance path. + + +## Regions + +BytePlus documents two ModelArk inference regions. Both are current. They are not interchangeable. + +
+ +| Region | Region ID | Base URL | +| --- | --- | --- | +| Johor, Asia Pacific | `ap-southeast-1` | `https://ark.ap-southeast.bytepluses.com/api/v3` | +| Dublin, Europe | `eu-west-1` | `https://ark.eu-west.bytepluses.com/api/v3` | + +
+ +BytePlus's region page says an inference endpoint created in a region must be invoked with that region's base URL, that it cannot be called through another region's URL, and that API keys and model activation status are isolated by region. Use the URL of the region that issued your key. This page does not retry against the other region, and it does not document any other host. + +The worked example below uses Asia Pacific. If your key is from Dublin, use the Europe URL from the table and change nothing else. + +## Install + +Install the OpenAI instrumentor and the OpenAI SDK. + +```bash +pip install "traceAI-openai==0.1.10" "fi-instrumentation-otel==1.1.0" "openai==3.31.0" +``` + +## Trace a chat completion + +Set the three keys. The BytePlus key goes to the OpenAI client. The Future AGI keys go to `register()`. Neither key belongs on a span. + +```bash +export ARK_API_KEY="YOUR_BYTEPLUS_ARK_API_KEY" +export FI_API_KEY="YOUR_API_KEY" +export FI_SECRET_KEY="YOUR_SECRET_KEY" +``` + +Then register the tracer, install the instrumentor, and build the client. Replace the model with a Model ID or endpoint ID from your ModelArk console. + +```python +import os + +from fi_instrumentation import register +from fi_instrumentation.fi_types import ProjectType +from openai import OpenAI +from traceai_openai import OpenAIInstrumentor + +trace_provider = register( + project_type=ProjectType.OBSERVE, + project_name="my-chatbot", +) +OpenAIInstrumentor().instrument(tracer_provider=trace_provider) + +client = OpenAI( + api_key=os.environ["ARK_API_KEY"], + base_url="https://ark.ap-southeast.bytepluses.com/api/v3", +) + +response = client.chat.completions.create( + model="your-model-or-endpoint-id", + messages=[{"role": "user", "content": "What is the capital of France?"}], +) +print(response.choices[0].message.content) +``` + +
+ +| Piece | Value | +| --- | --- | +| Client | `openai.OpenAI` from the `openai` package. Not `arkruntime.Ark`. | +| Base URL | `https://ark.ap-southeast.bytepluses.com/api/v3` for Asia Pacific, `https://ark.eu-west.bytepluses.com/api/v3` for Europe | +| Auth | `Authorization: Bearer `, which the OpenAI client sends for `api_key` | +| Model | The Model ID or endpoint ID from your console. BytePlus's own examples use dated ids such as `seed-2-0-lite-260228`; do not copy one without checking it is available to your key | +| Project | `project_name` in `register()`. The instrumentor does not set it | +| Self-hosted Future AGI | Also export `FI_BASE_URL` with your collector URL | + +
+ +Build the client after `register()` and `instrument()`, as above. BytePlus still receives your prompt: tracing sends a copy to Future AGI and does not change what the API sees. + +### Coding Plan is not this page + + +If your base URL contains `/api/coding`, this page does not apply. BytePlus's Coding Plan has its own base URLs for coding tools such as Claude Code and Cursor: `https://ark.ap-southeast.bytepluses.com/api/coding` for tools that speak the Anthropic protocol and `https://ark.ap-southeast.bytepluses.com/api/coding/v3` for tools that speak the OpenAI protocol. BytePlus's Coding Plan Team quick start says "Do not use `https://ark.ap-southeast.bytepluses.com/api/v3`; connecting will incur additional charges." It also says the Coding Plan Team API key is different from the ModelArk platform key and the two must not be mixed. This page is not a second worked example for the Coding Plan URLs, and the example app refuses them. Set up coding tools from their own integration pages. + + +The URLs and the billing note are from BytePlus's [Coding Plan Team quick start](https://docs.byteplus.com/en/docs/ModelArk/2277233) (last updated 2026-09-24, read 2026-10-15). BytePlus's personal Coding Plan pages ([Other tools](https://docs.byteplus.com/docs/ModelArk/coding-plan-personal-ai-other-tools), last updated 2026-09-28) render in the browser and their bodies were not re-extracted. Treat BytePlus's pages as the authority if they differ. + +## What you get + +Each call below produced the spans shown. "Stand-in" means the ModelArk-shaped local server; "real model" means the local OpenAI-compatible proxy running a real model. Neither is BytePlus. + +
+ +| Call | What lands in Observe | Notes | +| --- | --- | --- | +| `chat.completions.create` | One `LLM` span with the model, input and output messages, and input, output and total tokens | The model on a non-streaming span is the `model` field of the response body, not the id you sent. When the stand-in answered with a different model id, the span showed the response's id. The recorded response-id behavior is from the stand-in, not a ModelArk service response | +| `stream=True` | One `LLM` span with a first-token event | Token counts are present only when you pass `stream_options={"include_usage": True}`, which the ModelArk Chat API documents. Without it the span has no token counts | +| Tool calls | Two `LLM` spans, one per round trip. The first carries the tool definitions | Running the tool is your code. It becomes a span only if you create one | +| A failed call | An `LLM` span with status `ERROR`, the exception type and the HTTP status in the message, and an `exception` event | Observed with a wrong key against the stand-in and the proxy | +| `extra_body={"thinking": {"type": "enabled"}}` | The call is traced like any other | The field reaches BytePlus. It is not a span attribute | + +
+ +Spans group under any parent span or session you create. The example app wraps its calls in one `AGENT` span and a session id so the whole run is one trace. + +## What is not captured + +- **`thinking` and reasoning text.** The `thinking` field travels in `extra_body`, and the instrumentor does not record it. Reasoning text in `reasoning_content` is not on the span either. The span reports the response's `usage.completion_tokens` as output tokens; vendor reasoning-token accounting is outside the recorded evidence. The call is still traced. +- **The region or host.** The exported span carries no BytePlus hostname and no region. You cannot filter by region from the span. Put the region in your `project_name` or in your own span attribute if you need it. +- **BytePlus request ids and headers.** A `X-Client-Request-Id` header you send is a BytePlus correlation header. It does not become a Future AGI session id. +- **The model on streaming and failed calls, on 0.1.10.** traceAI-openai 0.1.10 leaves `gen_ai.request.model` off streaming spans and failed-call spans. [traceAI PR 249](https://github.com/future-agi/traceAI/pull/249) (merged to `dev` on 2026-10-28; absent from the published 0.1.10 package as of 2026-10-29) records it. The tests in the example pass on both. +- **Provider name.** `gen_ai.provider.name` is `openai` for both BytePlus hosts. PR 249 adds a host-to-provider table that does not contain either BytePlus host, so the value stays `openai` there too. +- **Cost.** The span carries no price. The local platform stored a cost of 0 for the stand-in model id and for the proxy's model id, and shows `$0.00`. The recorded evidence contains no real ModelArk pricing result. +- **`arkruntime.Ark`.** BytePlus's own SDK (arkruntime 0.13.0, checked against the stand-in) is not a subclass of `openai.OpenAI`, and traceAI-openai produced no spans for it while the stock `openai.OpenAI` client produced one. Use the `openai` package, as on this page. +- **Other ModelArk APIs.** The Responses API (`/api/v3/responses`), embeddings, multimodal embeddings, image, video and 3D generation and context caching are outside this recipe's supported API scope. The multimodal embeddings endpoint does not have the OpenAI request shape, so `client.embeddings.create` does not call it. +- **Language scope.** This recipe covers the Python OpenAI client only; it makes no TypeScript integration claim. + +## Privacy + +traceAI-openai follows the standard traceAI `TraceConfig` defaults: prompts, completions, tool arguments and results, and request parameters are captured unless you turn them off. Set these before the process starts, or pass the same settings to `TraceConfig`. + +```bash +export FI_HIDE_INPUTS=true +export FI_HIDE_OUTPUTS=true +export FI_HIDE_LLM_INVOCATION_PARAMETERS=true +``` + +With those three set, the model, token counts, status and timing still arrive and the content does not. Input and output fields hold `__REDACTED__`, the request parameters attribute is absent, and a unique marker placed in the system prompt, the user prompt, the model's answer, the tool-call arguments and the tool result appeared in no stored column. With the variables unset, the same marker appeared in the prompts, answers, tool arguments and results. Content is therefore on by default. To turn it back on after hiding it, unset the variables or set them to `false`. + +What the hide settings do not remove on traceAI-openai 0.1.10: + +- **Tool definitions.** The name, description and JSON schema of tools you pass are still on the span. +- **Session id.** The id you set, if any, stays. +- **Error text.** The status message, such as `AuthenticationError: Error code: 401 - ...`, includes the BytePlus error body and its request id. The exported `exception` event also carries the exception message and a stack trace with local file paths. The platform did not keep the stack trace in its span store in the runs documented here, but it is on the wire. + +If your errors can contain sensitive text, do not rely on the hide settings for them. + +## Run it yourself + +The example in [`examples/providers/byteplus`](https://github.com/future-agi/docs/tree/main/examples/providers/byteplus) contains the recipe as a runnable app, a ModelArk-shaped stand-in, and tests. Its `run_e2e.sh` installs the pinned versions in a virtual environment, runs the tests, and exports to your collector. + +```bash +cd examples/providers/byteplus +export FI_API_KEY="YOUR_API_KEY" FI_SECRET_KEY="YOUR_SECRET_KEY" +export FI_BASE_URL="http://localhost:4318" # your collector, if self-hosted +./run_e2e.sh # stand-in, no BytePlus account needed + +ARK_API_KEY="YOUR_BYTEPLUS_ARK_API_KEY" ARK_MODEL="your-model-or-endpoint-id" ./run_e2e.sh # real BytePlus, may cost money +``` + +With `ARK_API_KEY` set, the script calls BytePlus for `ARK_REGION` (`ap` by default, or `eu`). It never creates an account. + +## Troubleshooting + +
+ +| Symptom | Cause and fix | +| --- | --- | +| The span says `openai`, not BytePlus | Known display limit. Look for the project and model, not the provider name | +| HTTP 401 from BytePlus | A wrong key, or a key from the other region. BytePlus says API keys and endpoints are isolated by region, so use the base URL of the region that issued the key. Cross-region error status and body are outside this recipe's recorded evidence | +| No spans appear | Install the instrumentor with `tracer_provider=trace_provider` and use the `openai` package. Check `FI_API_KEY`, `FI_SECRET_KEY` and, if self-hosted, `FI_BASE_URL`. See [No traces appearing](/docs/observe/troubleshooting/no-traces-appearing) | +| No spans when using BytePlus's `Ark` client | traceAI-openai does not instrument `arkruntime.Ark`. Switch to `openai.OpenAI` | +| A streaming span has no token counts | Pass `stream_options={"include_usage": True}` | +| Bills do not match traces | Check the base URL. A base URL with `/api/coding` is the Coding Plan, which this page does not cover | +| `ai-evaluation is not installed, please install it to trace protect` printed at startup | Warning logged by `traceai-openai` on import. It concerns tracing Protect guardrails and did not affect chat tracing in these runs | +| `pip install` fails on Python 3.14 | traceAI-openai 0.1.10 requires Python below 3.14. Use 3.10 to 3.13 | + +
+ +## How this differs from the OpenAI page + +The [OpenAI integration](/docs/integrations/traceai/openai) assumes `api.openai.com`. This page assumes a different vendor's endpoint behind the same client: the regional base URLs, the key and region match, the Coding Plan URL to avoid, and what the spans do and do not show for a vendor that is not OpenAI. + +## Next steps + +- [OpenAI integration](/docs/integrations/traceai/openai) for everything traceAI-openai captures on `api.openai.com`. +- [No traces appearing](/docs/observe/troubleshooting/no-traces-appearing) if the export does not arrive.