You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Bug]: Codex usage parser does not reconcile per-event and cumulative counters #5800
I searched existing issues and did not find a duplicate.
I included enough detail to reproduce or investigate the problem.
Area
apps/server
Steps to reproduce
Create a Codex rollout containing a turn_context followed by a valid token_count event whose total_token_usage advances by its last_token_usage.
Add another token_count event where the cumulative total_token_usage is unchanged but last_token_usage differs, as can happen when Codex recomputes current-context usage or re-emits token state alongside rate-limit updates.
Alternatively, make total_token_usage advance by a value that does not equal last_token_usage.
Scan the rollout from the Usage page.
The Codex protocol maintains total_token_usage by adding each real last_token_usage element-wise. An unchanged cumulative total therefore proves that a later event did not add billable usage, and a different cumulative delta proves that the two counters are inconsistent.
Expected behavior
The usage parser should use total_token_usage as an integrity signal. It should count a legitimate per-request last_token_usage once, suppress events whose cumulative total did not advance, and report inconsistent counter sequences as malformed rather than guessing which counter is correct.
Reasoning tokens must remain a subset of output tokens and must not be added to the processed-token total again.
Actual behavior
parseCodexLine never reads total_token_usage. It trusts each nonzero last_token_usage whose JSON signature differs from the immediately previous one.
This means a stale or recomputed last_token_usage can be counted even when the cumulative total did not advance, and a malformed per-event counter can be accepted even when it cannot reconcile with the cumulative counter. The Usage feature's original PR stated that these fields had been reconciled before last_token_usage was trusted, but that invariant is not implemented in the parser.
This is separate from copied parent history in forked rollout files (#5758): the issue here occurs within one rollout's counter sequence.
Implementation scope note: the focused parser change will count rejected/resynchronized counter sequences on CodexScanState.malformedRecords, but it will not add new RPC or source-status plumbing. Exposing those diagnostics through the Usage source summary belongs with the scan-status work tracked in #5798. The parser-output cache will be version-bumped so unchanged transcripts are reparsed under the corrected invariant.
Scope update after implementation review: I am revising the earlier note about keeping diagnostics parser-local. If rejected counter sequences only increment transient parser state, production scans still report the source as healthy and warm-cache reads lose the diagnostic entirely.
The focused fix will therefore carry only the Codex counter-reconciliation malformed count through the transcript reader and durable cache into UsageSource.malformedRecords, marking that source partial with a bounded message. General directory/stat/file I/O coverage remains separate in #5798.
Activity
caezium commentedon Aug 9, 2026
Implementation scope note: the focused parser change will count rejected/resynchronized counter sequences on
CodexScanState.malformedRecords, but it will not add new RPC or source-status plumbing. Exposing those diagnostics through the Usage source summary belongs with the scan-status work tracked in #5798. The parser-output cache will be version-bumped so unchanged transcripts are reparsed under the corrected invariant.caezium commentedon Aug 9, 2026
Scope update after implementation review: I am revising the earlier note about keeping diagnostics parser-local. If rejected counter sequences only increment transient parser state, production scans still report the source as healthy and warm-cache reads lose the diagnostic entirely.
The focused fix will therefore carry only the Codex counter-reconciliation malformed count through the transcript reader and durable cache into
UsageSource.malformedRecords, marking that sourcepartialwith a bounded message. General directory/stat/file I/O coverage remains separate in #5798.