Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions examples/providers/byteplus/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
.venv/
__pycache__/
*.pyc
80 changes: 80 additions & 0 deletions examples/providers/byteplus/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# BytePlus ModelArk tracing example

Runnable companion to the docs page `src/pages/docs/integrations/providers/byteplus.mdx`
(`/docs/integrations/providers/byteplus`). It traces BytePlus ModelArk chat completions made with the
stock `openai` Python client and the existing `traceAI-openai` instrumentor. There is no BytePlus
package and no `traceai-byteplus`.

Status: alpha. **No BytePlus account was used and no request was ever sent to a BytePlus host.** See
"What is real and what is a stand-in" below.

## Contents

| Path | What it is |
| --- | --- |
| `app.py` | The recipe (about 10 lines of integration) plus scenarios: `chat`, `stream`, `tools`, `thinking`, `error` |
| `run_e2e.sh` | Creates a venv, runs the tests, runs `app.py` against a collector, optionally runs the privacy comparison |
| `standin/byteplus_modelark_stub.py` | Local STAND-IN shaped like the ModelArk Chat API (`/api/v3/chat/completions`). Not BytePlus |
| `standin/connect_probe.py` | Refusing HTTP proxy that records which host the unmodified client dials. Opens no tunnel |
| `tests/` | pytest suite (see below) |
| `requirements.txt` | Pinned versions the example was tested with |

## Run it

Python 3.10 to 3.13 (traceAI-openai 0.1.10 does not support 3.14).

```bash
export FI_API_KEY=... FI_SECRET_KEY=...
export FI_BASE_URL=http://localhost:4318 # your Future AGI collector (run_e2e.sh defaults to this)
./run_e2e.sh # tests, then app.py against the stand-in. No BytePlus needed
PRIVACY=1 ./run_e2e.sh # also: content capture off vs default, with a unique marker
```

Against the real BytePlus API (needs your own account, activated ModelArk and a key; this can cost money,
and the script never creates an account):

```bash
export ARK_API_KEY=... # BytePlus ModelArk key, goes to the OpenAI client only
export ARK_MODEL=... # Model ID or endpoint ID from your ModelArk console
export ARK_REGION=ap # ap (Johor, default) or eu (Dublin): must match the region that issued the key
./run_e2e.sh
```

Any OpenAI-compatible server can stand in for the target: set `ARK_BASE_URL`, `ARK_API_KEY` and `ARK_MODEL`.
`ARK_BASE_URL` containing `/api/coding` is refused: that is the Coding Plan, not this recipe.

Run a single scenario with `APP_ARGS="--scenario stream"`. `python app.py --help` lists the flags.

## What the tests cover

| File | Covers |
| --- | --- |
| `tests/test_recipe.py` | Region URLs and default; Coding Plan URLs refused; trace shape (AGENT, LLM, TOOL, one session); model and usage; streaming usage; `thinking` in `extra_body`; error span; provider label `openai`; keys go only where they belong; uncompressed OTLP with `project_name`; an unreachable collector never breaks the request |
| `tests/test_regions.py` | With the documented AP and EU URLs, the unmodified client dials exactly `ark.ap-southeast.bytepluses.com:443` / `ark.eu-west.bytepluses.com:443`, never the other region; the span records neither host |
| `tests/test_privacy.py` | A unique marker in every prompt, answer, tool argument and tool result: present by default, absent from every exported byte with `FI_HIDE_*`; what the hide flags do not remove |
| `tests/test_page.py` | The Python snippet on the docs page runs as written (with the base URL swapped for the stand-in) and produces the span the page describes |
| `tests/test_stub.py` | The stand-in only claims fields BytePlus documents |
| `tests/test_ark_sdk.py` | Optional (`pip install arkruntime`): BytePlus's own `Ark` client is not traced by traceAI-openai |

```bash
.venv/bin/python -m pytest -q
```

## What is real and what is a stand-in

| Piece | Real | Stand-in |
| --- | --- | --- |
| `openai` SDK, `traceAI-openai`, `fi-instrumentation-otel`, OTLP export | yes | |
| Future AGI collector | yours | the tests use a recording collector |
| The ModelArk API | only if you set `ARK_API_KEY` without `ARK_BASE_URL` | `standin/byteplus_modelark_stub.py`, shaped from BytePlus's Chat API reference. Nothing in it was captured from a real BytePlus response. The `{"error": {...}}` envelope is an assumption |
| BytePlus hostnames | the client dials them in `tests/test_regions.py` | a refusing proxy answers 403; no tunnel opens |

What a live run is still needed for: the exact `model` value ModelArk returns for an endpoint id, whether
reasoning tokens are inside `usage.completion_tokens`, the real error body for a cross-region key, and
whether the real SSE stream matches the stand-in's.

## Versions

Tested 2026-10-15 on Python 3.13: traceAI-openai 0.1.10, fi-instrumentation-otel 1.1.0, openai 3.31.0 (latest)
and 1.109.1, opentelemetry-sdk 1.45.1. The suite also passes with traceAI PR 249 (open) overlaid; the two
assertions that differ are written to accept both.
209 changes: 209 additions & 0 deletions examples/providers/byteplus/app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,209 @@
#!/usr/bin/env python3
"""Trace BytePlus ModelArk chat completions with traceAI-openai.

This is the recipe from docs/integrations/providers/byteplus, plus a few small
scenarios so it can be run end to end:

chat one non-streaming chat completion
stream one streaming call (asks for the usage chunk BytePlus documents)
tools a tool round trip: model asks for a tool, the app runs it, the model answers
thinking a call that passes BytePlus's `thinking` field through extra_body
error a call with a wrong API key (the failure case)

The integration is only these lines (everything else is demo scaffolding):

trace_provider = register(project_type=ProjectType.OBSERVE, project_name=...)
OpenAIInstrumentor().instrument(tracer_provider=trace_provider)
client = OpenAI(api_key=<ARK_API_KEY>, base_url="https://ark.ap-southeast.bytepluses.com/api/v3")

Environment
ARK_API_KEY BytePlus ModelArk API key. Goes to the OpenAI client only.
ARK_MODEL Model ID (or endpoint ID) from your ModelArk console. Not frozen here.
ARK_REGION ap (default, Johor) or eu (Dublin). Must match the region that issued the key.
ARK_BASE_URL Override of the regional URL. Use it only to point at a local
stand-in; a real setup leaves it unset. A URL that contains
/api/coding is refused (that is the Coding Plan, not this recipe).
FI_API_KEY, FI_SECRET_KEY, FI_BASE_URL Future AGI. Go to the tracer only.
FI_PROJECT_NAME Observe project (default: byteplus).
FI_HIDE_INPUTS, FI_HIDE_OUTPUTS, FI_HIDE_LLM_INVOCATION_PARAMETERS
Set to true to keep prompts, completions and request
parameters out of the spans (traceAI TraceConfig).
"""
import argparse
import json
import os
import sys
import uuid

import openai
from fi_instrumentation import FITracer, TraceConfig, register, using_session
from fi_instrumentation.fi_types import FiSpanKindValues, ProjectType
from openai import OpenAI
from traceai_openai import OpenAIInstrumentor

# The two ModelArk inference regions BytePlus documents (docs: Region availability).
# They are not interchangeable: a key and an inference endpoint belong to one region.
REGION_URLS = {
"ap": "https://ark.ap-southeast.bytepluses.com/api/v3", # Johor, Asia Pacific, ap-southeast-1
"eu": "https://ark.eu-west.bytepluses.com/api/v3", # Dublin, Europe, eu-west-1
}
DEFAULT_REGION = "ap"

TOOLS = [
{
"type": "function",
"function": {
"name": "lookup_record",
"description": "Look up a record by a free-text query.",
"parameters": {
"type": "object",
"properties": {"query": {"type": "string"}},
"required": ["query"],
},
},
}
]


def lookup_record(query):
return f"record for {query}"


def resolve_base_url(env=None):
"""The regional URL, or the stand-in override. Returns (base_url, error_message)."""
env = os.environ if env is None else env
override = env.get("ARK_BASE_URL")
region = (env.get("ARK_REGION") or DEFAULT_REGION).lower()
if region not in REGION_URLS:
return None, f"ARK_REGION must be one of {sorted(REGION_URLS)}, got {region!r}"
base_url = override or REGION_URLS[region]
if "/api/coding" in base_url:
return None, (
"refusing a Coding Plan URL (contains /api/coding). That is a different BytePlus product and is "
"not covered by this recipe; use the ModelArk inference URL for your region."
)
return base_url, None


def _messages(prompt, system):
return ([{"role": "system", "content": system}] if system else []) + [{"role": "user", "content": prompt}]


def run_chat(client, model, prompt, system):
reply = client.chat.completions.create(model=model, messages=_messages(prompt, system))
return reply.choices[0].message.content


def run_stream(client, model, prompt, system, include_usage=True):
extra = {"stream_options": {"include_usage": True}} if include_usage else {} # documented in the ModelArk Chat API
stream = client.chat.completions.create(model=model, messages=_messages(prompt, system), stream=True, **extra)
return "".join(part.choices[0].delta.content or "" for part in stream if part.choices)


def run_thinking(client, model, prompt, system, mode="disabled"):
# `thinking` is a ModelArk field the OpenAI SDK does not know; extra_body carries it.
# BytePlus documents "enabled" and "disabled"; which values a model accepts varies by model.
reply = client.chat.completions.create(
model=model,
messages=_messages(prompt, system),
extra_body={"thinking": {"type": mode}},
)
return reply.choices[0].message.content


def run_tools(client, model, prompt, system, tracer):
messages = _messages(prompt, system)
first = client.chat.completions.create(
model=model,
messages=messages,
tools=TOOLS,
tool_choice={"type": "function", "function": {"name": "lookup_record"}},
)
message = first.choices[0].message
if not message.tool_calls:
return message.content
call = message.tool_calls[0]
# Your own spans carry only what you put on them. This one records no arguments.
with tracer.start_as_current_span("tool lookup_record", fi_span_kind=FiSpanKindValues.TOOL):
result = lookup_record(**json.loads(call.function.arguments))
messages += [
{"role": "assistant", "content": message.content or "", "tool_calls": [c.model_dump() for c in message.tool_calls]},
{"role": "tool", "tool_call_id": call.id, "content": result},
]
final = client.chat.completions.create(model=model, messages=messages, tools=TOOLS)
return final.choices[0].message.content


def run_error(base_url, model, prompt):
"""A deliberately wrong API key: ModelArk answers 401 AuthenticationError and the span is ERROR."""
bad = OpenAI(api_key="not-a-real-ark-key", base_url=base_url)
try:
bad.chat.completions.create(model=model, messages=[{"role": "user", "content": prompt}])
except openai.APIStatusError as exc:
return f"HTTP {exc.status_code}"
return "no error (unexpected)"


def main(argv=None):
ap = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
ap.add_argument("--scenario", choices=["all", "chat", "stream", "tools", "thinking", "error"], default="all")
ap.add_argument("--prompt", default="Say hello to BytePlus ModelArk in five words.")
ap.add_argument("--system", default="")
ap.add_argument("--thinking", choices=["disabled", "enabled"], default="disabled", help="value for the thinking scenario")
ap.add_argument("--no-stream-usage", action="store_true", help="stream without stream_options.include_usage (the span then has no token counts)")
ap.add_argument("--session-id", default=None)
a = ap.parse_args(argv)

api_key = os.environ.get("ARK_API_KEY")
model = os.environ.get("ARK_MODEL")
if not api_key or not model:
print("set ARK_API_KEY and ARK_MODEL (a Model ID or endpoint ID from your ModelArk console)", file=sys.stderr)
return 2
base_url, problem = resolve_base_url()
if problem:
print(problem, file=sys.stderr)
return 2

# 1. Future AGI tracing. The Future AGI key pair is read from FI_API_KEY / FI_SECRET_KEY.
trace_provider = register(
project_type=ProjectType.OBSERVE,
project_name=os.environ.get("FI_PROJECT_NAME", "byteplus"),
)
OpenAIInstrumentor().instrument(tracer_provider=trace_provider)

# 2. The OpenAI client, pointed at the BytePlus region that issued the key.
client = OpenAI(api_key=api_key, base_url=base_url)

tracer = FITracer(trace_provider.get_tracer("byteplus-example"), config=TraceConfig())
session_id = a.session_id or f"byteplus-{uuid.uuid4()}"
wanted = ["chat", "stream", "tools", "thinking", "error"] if a.scenario == "all" else [a.scenario]
# Optional app-level parent span and session, so the calls are grouped in Observe.
with using_session(session_id), tracer.start_as_current_span(
"byteplus-demo", fi_span_kind=FiSpanKindValues.AGENT
) as root:
trace_id = str(uuid.UUID(int=root.get_span_context().trace_id))
for name in wanted:
try:
if name == "chat":
out = run_chat(client, model, a.prompt, a.system)
elif name == "stream":
out = run_stream(client, model, a.prompt, a.system, include_usage=not a.no_stream_usage)
elif name == "tools":
out = run_tools(client, model, a.prompt, a.system, tracer)
elif name == "thinking":
out = run_thinking(client, model, a.prompt, a.system, a.thinking)
else:
out = run_error(base_url, model, a.prompt)
except openai.OpenAIError as exc: # keep the demo going; the span already records the failure
out = f"{type(exc).__name__}"
print(f"RESULT {name}: {out}")
trace_provider.force_flush()
trace_provider.shutdown()
print(f"base_url={base_url}")
print(f"trace_id={trace_id}")
print(f"session_id={session_id}")
return 0


if __name__ == "__main__":
raise SystemExit(main())
3 changes: 3 additions & 0 deletions examples/providers/byteplus/pytest.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[pytest]
testpaths = tests
addopts = -p no:cacheprovider
6 changes: 6 additions & 0 deletions examples/providers/byteplus/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Tested 2026-10-15 on Python 3.13 (traceAI-openai 0.1.10 requires Python < 3.14).
traceAI-openai==0.1.10
fi-instrumentation-otel==1.1.0
openai==3.31.0
# tests
pytest>=8
88 changes: 88 additions & 0 deletions examples/providers/byteplus/run_e2e.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
#!/usr/bin/env bash
# End-to-end run of the BytePlus ModelArk recipe against a Future AGI collector.
#
# FI_API_KEY=... FI_SECRET_KEY=... FI_BASE_URL=http://localhost:4318 ./run_e2e.sh
#
# What it does
# 1. creates ./.venv (Python 3.12 or 3.13: traceAI-openai needs < 3.14) and installs requirements.txt
# (set VENV_DIR to reuse an existing environment)
# 2. runs the tests (they use the local stand-in, a recording collector and a refusing CONNECT
# probe; no Future AGI and no BytePlus needed)
# 3. runs app.py (chat, stream, tools, thinking, error) with tracing on to FI_BASE_URL
# 4. PRIVACY=1: runs app.py twice more with a unique marker in the prompts, once with content
# capture off (FI_HIDE_*) and once with the default, into <project>-private and <project>-optin,
# and prints the markers so you can search your own Future AGI store for them
#
# Which API it calls
# default the local stand-in in standin/ (NOT BytePlus)
# ARK_API_KEY + ARK_MODEL the real BytePlus ModelArk API for ARK_REGION (ap by default).
# Needs a BytePlus account with ModelArk activated; this script
# never creates one, and a real run can cost money.
# ARK_BASE_URL=... any OpenAI-compatible base URL (for example a local proxy running a
# real model); combine with ARK_API_KEY and ARK_MODEL.
#
# Other knobs: FI_PROJECT_NAME (default byteplus-e2e), SKIP_TESTS=1, APP_ARGS="--scenario chat".
set -euo pipefail

HERE="$(cd "$(dirname "$0")" && pwd)"
cd "$HERE"
: "${FI_API_KEY:?set FI_API_KEY}" "${FI_SECRET_KEY:?set FI_SECRET_KEY}"
export FI_BASE_URL="${FI_BASE_URL:-http://localhost:4318}"
export FI_PROJECT_NAME="${FI_PROJECT_NAME:-byteplus-e2e}"

VENV_DIR="${VENV_DIR:-$HERE/.venv}"
if [ ! -x "$VENV_DIR/bin/python" ]; then
PY="${PYTHON:-$(command -v python3.13 || command -v python3.12 || true)}"
[ -n "$PY" ] || { echo "need python3.12 or python3.13 on PATH (or set PYTHON)"; exit 2; }
"$PY" -m venv "$VENV_DIR"
"$VENV_DIR/bin/python" -m pip install --quiet --upgrade pip
"$VENV_DIR/bin/python" -m pip install --quiet -r requirements.txt
fi
PYBIN="$VENV_DIR/bin/python"
"$PYBIN" - <<'PY'
import importlib.metadata as m, sys
names = ["traceai-openai", "fi-instrumentation-otel", "openai", "opentelemetry-sdk"]
print("versions: python", sys.version.split()[0], *(f"{n}=={m.version(n)}" for n in names))
PY

if [ "${SKIP_TESTS:-0}" != "1" ]; then
"$PYBIN" -m pytest -q
fi

STUB_PID=""
cleanup() { [ -n "$STUB_PID" ] && kill "$STUB_PID" 2>/dev/null || true; }
trap cleanup EXIT

if [ -n "${ARK_API_KEY:-}" ]; then
: "${ARK_MODEL:?set ARK_MODEL (a Model ID or endpoint ID from your ModelArk console)}"
if [ -n "${ARK_BASE_URL:-}" ]; then
echo "target: custom base URL $ARK_BASE_URL (not necessarily BytePlus)"
else
echo "target: REAL BytePlus ModelArk, region ${ARK_REGION:-ap}"
fi
else
PORT="$("$PYBIN" -c 'import socket; s=socket.socket(); s.bind(("127.0.0.1",0)); print(s.getsockname()[1])')"
"$PYBIN" standin/byteplus_modelark_stub.py --port "$PORT" --api-key e2e-stand-in-key >/dev/null 2>&1 &
STUB_PID=$!
for _ in $(seq 1 30); do curl -fs "http://127.0.0.1:$PORT/healthz" >/dev/null 2>&1 && break; sleep 0.2; done
export ARK_BASE_URL="http://127.0.0.1:$PORT/api/v3" ARK_API_KEY=e2e-stand-in-key ARK_MODEL="${ARK_MODEL:-example-model-id}"
echo "target: STAND-IN (not BytePlus) at $ARK_BASE_URL"
fi

echo "== app.py -> API, traces -> $FI_BASE_URL project=$FI_PROJECT_NAME"
"$PYBIN" app.py ${APP_ARGS:-} | grep -E '^(RESULT|base_url|trace_id|session_id)'

if [ "${PRIVACY:-0}" = "1" ]; then
for mode in private optin; do
MARK="PRIV-$(date +%s)-$RANDOM$RANDOM"
if [ "$mode" = "private" ]; then
export FI_HIDE_INPUTS=true FI_HIDE_OUTPUTS=true FI_HIDE_LLM_INVOCATION_PARAMETERS=true
else
unset FI_HIDE_INPUTS FI_HIDE_OUTPUTS FI_HIDE_LLM_INVOCATION_PARAMETERS
fi
echo "== privacy run ($mode), marker $MARK, project ${FI_PROJECT_NAME}-$mode"
FI_PROJECT_NAME="${FI_PROJECT_NAME}-$mode" "$PYBIN" app.py --prompt "find $MARK" --system "system note $MARK" ${APP_ARGS:-} \
| grep -E '^(RESULT|trace_id)'
echo "MARKER_${mode}=$MARK"
done
fi
Loading
Loading