Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/configuration/environment-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ export FI_PROJECT_VERSION_NAME="v1.2.0"
| `FI_HIDE_EMBEDDING_VECTORS` | `false` | Hide embedding vectors |
| `FI_HIDE_LLM_INVOCATION_PARAMETERS` | `false` | Hide model parameters |
| `FI_BASE64_IMAGE_MAX_LENGTH` | `32000` | Max chars for base64 images |
| `FI_HIDE_ERROR_DETAILS` | derived | Hide error messages and stacktraces (Python). Unset: `true` when any input/output hide flag above is `true` |

```bash
# Production privacy settings
Expand Down
45 changes: 44 additions & 1 deletion docs/configuration/trace-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ By default, traceAI captures all data including prompts, completions, and model
| `hide_embedding_vectors` | bool | `false` | Hide embedding vector values |
| `hide_llm_invocation_parameters` | bool | `false` | Hide model parameters (temperature, etc.) |
| `base64_image_max_length` | int | `32000` | Truncate large base64 images |
| `hide_error_details` | bool | derived | Hide error messages and stacktraces (Python). Unset: on when any of `hide_inputs`, `hide_outputs`, `hide_input_messages`, `hide_output_messages`, `hide_input_text`, `hide_output_text` is on |

## Python Usage

Expand All @@ -36,6 +37,7 @@ export FI_HIDE_OUTPUT_TEXT=true
export FI_HIDE_EMBEDDING_VECTORS=true
export FI_HIDE_LLM_INVOCATION_PARAMETERS=true
export FI_BASE64_IMAGE_MAX_LENGTH=16000
export FI_HIDE_ERROR_DETAILS=true
```

### Via Code
Expand All @@ -54,6 +56,7 @@ config = TraceConfig(
hide_embedding_vectors=True,
hide_llm_invocation_parameters=True,
base64_image_max_length=16000,
hide_error_details=True,
)
```

Expand Down Expand Up @@ -166,14 +169,54 @@ When data is hidden, it's replaced with `"__REDACTED__"`:
}
```

## Error Details

Error text is content too: provider error bodies can quote the prompt back and carry
request or session ids, and Python tracebacks carry local file paths.

With `hide_error_details` on (Python), a failed span keeps:
- status `ERROR`, with no description
- `error.type`: the provider error code (for example `model_not_found`) or the fully
qualified exception class
- exception events with `exception.type` (and `exception.escaped`) only

and drops `exception.message`, `exception.stacktrace`, `error.message` and
instrumentor keys ending in `.error.message` / `.error_message`. A failed LangChain run
also records no outputs: LangChain fills them with the provider's error response
(error body, response headers, request ids).

If you don't set `hide_error_details` (or `FI_HIDE_ERROR_DETAILS`), it is on whenever
any of `hide_inputs`, `hide_outputs`, `hide_input_messages`, `hide_output_messages`,
`hide_input_text` or `hide_output_text` is on. Set it to `false` explicitly to keep
error messages while hiding inputs and outputs:

```python
config = TraceConfig(
hide_inputs=True,
hide_outputs=True,
hide_error_details=False, # keep error messages for debugging
)
```

With it off, the status description is the exception message only, never a traceback;
the stacktrace is recorded on the exception event.

`error.type` is set only on spans that end with status `ERROR` (an exception recorded on
a span that still succeeds, such as a retried timeout, does not set it). It is the
provider error code for exceptions from openai, anthropic, groq, litellm,
botocore/boto3, google-genai, google-api-core and mistralai (for example
`model_not_found`, `not_found_error`, `ValidationException`, `NOT_FOUND`), the HTTP
status for httpx `HTTPStatusError` (`"404"`), and the fully qualified exception class
for everything else (for example `sqlalchemy.exc.OperationalError`).

## What's Always Captured

Even with maximum redaction, these are always captured:
- Span names and types
- Timestamps and duration
- Model names
- Token counts
- Error messages (consider additional error handling)
- Error status and `error.type` (error messages are hidden with `hide_error_details`)

## Best Practices

Expand Down
51 changes: 51 additions & 0 deletions python/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,54 @@
## [1.2.0] - 2026-10-18
### Changed
- **Behaviour change for users of the hide flags:** setting any of `hide_inputs`,
`hide_outputs`, `hide_input_messages`, `hide_output_messages`, `hide_input_text` or
`hide_output_text` (or their `FI_HIDE_*` env vars) now also hides error details.
Error text is content: provider error bodies quote the prompt back and carry request
and session ids, and tracebacks carry local paths. Failed spans keep status `ERROR`,
`error.type` and `exception.type`, but the status has no description and exception
events drop `exception.message` / `exception.stacktrace`. `error.message`,
`exception.message`, `exception.stacktrace` and instrumentor keys ending in
`.error.message` / `.error_message` are dropped. Set `hide_error_details=False`
(or `FI_HIDE_ERROR_DETAILS=false`) to keep the old behaviour.
- An `ERROR` status description is now the exception message only. The
`"<Type>: <message>"` form built by OpenTelemetry's `use_span` and most instrumentors
becomes `<message>` (the type is in `error.type`), also when one such line is joined
per exception (llama-index), and a Python traceback is never kept in the description.
The cleanup uses plain string operations, so a multi-megabyte exception message adds
about a millisecond to `set_status` (no regular expression is built from message text).

- **Content-hiding fix:** `hide_inputs` (`FI_HIDE_INPUTS`) now also redacts `raw.input`
(the raw request body, messages included) and the prompt template variables
(`llm.prompt_template.variables`, `gen_ai.prompt.template.variables`), and
`hide_outputs` (`FI_HIDE_OUTPUTS`) also redacts `raw.output`. Before, an instrumentation
that set these kept the prompt in the stored row while the hide flag was on. Same keys
as `@traceai/fi-core` 1.1.0.

### Feature
- New `TraceConfig(hide_error_details=...)` / `FI_HIDE_ERROR_DETAILS`. Unset, it is
derived from the content hide flags above; an explicit `true`/`false` wins.
`dataclasses.replace(config, hide_inputs=True)` derives it again from the new flags
unless it was set explicitly.
- Spans that end with status `ERROR` get `error.type` (an exception recorded on a span
that still ends OK, such as a recovered retry, does not set it). It is the provider
error code for exceptions from known provider SDKs: openai and groq
`code`/`type`, anthropic `error.type`, botocore/boto3 `Error.Code`, google-genai
`status`, google-api-core gRPC status, mistralai body `type`, and the HTTP status for
httpx `HTTPStatusError`. Every other exception gets its fully qualified class name
(for example `sqlalchemy.exc.OperationalError`, never its `.code`). An instrumentor's
own `error.type` is not overwritten. A failed span whose status is not about a
named exception (for example `set_status(ERROR, "guardrail rejected")` after a retried
timeout, outside an `except` block) gets `_OTHER` (OpenTelemetry semantic conventions)
instead of a stale recovered exception. litellm exceptions carry no `code`/`type`, so
they get their class name (for example `litellm.exceptions.NotFoundError`).

### Fixed
- `with tracer.start_span(...) as span:` now yields the FI span wrapper instead of the
raw SDK span, so attribute masking, span-kind ordering and the error handling above
also apply inside that block.
- `fi_instrumentation.instrumentation.version.__version__` now matches the package
version (it still said `0.1.19`).

## [1.1.0] - 2026-08-11
### Feature
- `register()` now accepts span-processor / exporter / provider tuning:
Expand Down
2 changes: 2 additions & 0 deletions python/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ config = TraceConfig(
hide_input_images=False, # Hide images in inputs
hide_embedding_vectors=False, # Hide embedding vectors
base64_image_max_length=32000, # Truncate large images
hide_error_details=None, # Hide error messages/stacktraces; None = on if any input/output flag is on
)
```

Expand All @@ -80,6 +81,7 @@ Or use environment variables:
- `FI_HIDE_OUTPUT_MESSAGES=true`
- `FI_HIDE_INPUT_IMAGES=true`
- `FI_HIDE_EMBEDDING_VECTORS=true`
- `FI_HIDE_ERROR_DETAILS=true|false` (unset: follows the input/output flags)

### Context Managers

Expand Down
9 changes: 9 additions & 0 deletions python/fi_instrumentation/fi_types.py
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,15 @@ class SpanAttributes:
"""Prompts provided to a completions API."""

# Input/Output values (OpenInference compat)
RAW_INPUT = "raw.input"
"""The raw request body (e.g. the full provider request, messages included)"""

RAW_OUTPUT = "raw.output"
"""The raw response body"""

LLM_PROMPT_TEMPLATE_VARIABLES = "llm.prompt_template.variables"
"""JSON of the values filled into a prompt template"""

OUTPUT_VALUE = "output.value"
OUTPUT_MIME_TYPE = "output.mime_type"
INPUT_VALUE = "input.value"
Expand Down
Loading
Loading