Repository navigation
Python: fix(core): include tool trajectory details in summarizer input - #8087
Conversation
Introduce _format_summary_content as the dispatch point for per-content summary rendering and rewrite _format_summary_message to combine structured renderings with Message.text, keeping the legacy fallback to content types. Behavior is byte-identical for existing inputs; tool-call and tool-result branches follow in the next commit.
Render function_call contents with name, arguments, and call id, and function_result contents with result text, exception, and call id in _format_summary_content, so the summarizer LLM sees the tool trajectory instead of a bare content type. Text-only messages keep their legacy rendering byte-for-byte; messages mixing text and tool contents now include both.
Extend _format_summary_content with mcp_server_tool_call / mcp_server_tool_result (tool name, arguments, output, call id) and function_approval_request / function_approval_response (nested call name, approval id, decision) branches, completing the tool trajectory visible to the summarizer LLM. Fixes microsoft#8086
There was a problem hiding this comment.
🟡 Changes recommended
A critical crash path and two moderate rendering defects remain unresolved.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds structured function, MCP, and approval trajectory details to Python summarizer transcripts.
Changes:
- Renders tool calls, results, exceptions, IDs, and approval decisions.
- Preserves text-only formatting.
- Adds focused regression tests.
Required fixes:
- Critical (1 vote): Safely stringify non-JSON-serializable MCP result mappings to prevent compaction crashes.
- Moderate (2 votes): Preserve chronological ordering of text and structured content.
- Moderate (1 vote): Use
tool_namewhen rendering approvals wrapping MCP calls.
File summaries
| File | Description |
|---|---|
python/packages/core/tests/core/test_compaction.py |
Adds formatter regression coverage. |
python/packages/core/agent_framework/_compaction.py |
Adds structured summary rendering; contains the unresolved findings above. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 3
- Review effort level: Balanced
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
- Stringify non-JSON-serializable MCP result mappings with the same json.dumps(default=str) fallback used by Content.from_function_result, preventing compaction crashes inside _select_summary_input_groups. - Preserve time order of text and structured contents in _format_summary_message by emitting consecutive text blocks in place. - Use tool_name as the fallback identity for MCP tool calls nested in approval contents. Addresses review feedback on PR microsoft#8087; adds three regression tests.
|
Addressed the three findings in 6b5360d:
Added three regression tests (non-JSON MCP output, mixed-content ordering, MCP approval tool name); all 90 tests in \ ests/core/test_compaction.py\ pass, ruff clean. copilot-pull-request-reviewer please re-review. |
Ricky-7-Yan
left a comment
There was a problem hiding this comment.
Verified the current 6b5360da head independently against the exact two-file diff:
- the complete
tests/core/test_compaction.pysuite passes; - Ruff check and format check pass for both changed files;
- the three earlier findings are resolved: mapping outputs now use the established
default=strfallback, mixed text/tool contents retain their original order, and MCP approvals usetool_namewhennameis absent.
I also checked compatibility with the existing Message.text behavior for text-only, consecutive-text, text-plus-unrendered-content, and non-text fallback cases. No new blocking issue found.
|
JHf0912 please respond to each open PR review comments as to whether they're addressed or not. |
|
Thanks Evan Mattson (@moonbox3) — done. I've replied to all three open review threads. All three are addressed in 6b5360d, each with a dedicated regression test:
All 90 tests in \ ests/core/test_compaction.py\ pass; ruff clean. |
Prefer content.items when rendering function_result for the summarizer transcript, falling back to content.result for legacy results. Rich results (such as the error contents emitted by Monty and Hyperlight) store their payload in items while result only carries concatenated text, so failures previously rendered as blank results and the summarizer lost the error details. Adds a dedicated _format_summary_result_items helper rendering text and error items (message, error_details, error_code). Adds three regression tests; all 93 tests in tests/core/test_compaction.py pass; ruff clean. Addresses review feedback on PR microsoft#8087.
Add test_summarization_strategy_preserves_tool_trajectory_in_summary_input, matching the issue microsoft#8086 triage reproduction: a recording summarizer runs over a tool call/result pair with a named function, JSON arguments, and a shared call id, and the summarizer transcript must carry the tool name, arguments, result, and call id. Red against main (transcript contains only function_call / function_result placeholders) and green with the summarizer input fix. All 94 tests in tests/core/test_compaction.py pass; ruff clean.
Non-text rich result items (data, uri, hosted_file) now render safe metadata in the summarizer transcript instead of being skipped, so a tool that returns only an output file no longer reaches the summarizer as an empty function_result. Data contents never embed their payload: they are represented by media type and additional_properties path, while external uri contents include their uri, matching the existing observability stance of not emitting binary blobs. Adds three regression tests; all 97 tests in tests/core/test_compaction.py pass; ruff clean. Addresses review feedback on PR microsoft#8087.
Problem
SummarizationStrategyfeeds the summarizer LLM a transcript rendered by_format_summary_message(python/packages/core/agent_framework/_compaction.py), which only usesMessage.text. SinceMessage.text(python/packages/core/agent_framework/_types.py) concatenatesTextContentonly, tool rounds collapse to placeholders like2. [assistant] function_calland3. [tool] function_result. The summarizer therefore never sees the tool name, arguments, results, exceptions, orcall_id, and produces summaries that silently discard the tool trajectory.Fixes #8086
Solution
Add a per-content renderer
_format_summary_contentthat serializes tool trajectory contents into the summary input transcript, and route_format_summary_messagethrough it:function_call→function_call <name>(<arguments>) [call_id=<id>]function_result→function_result: <result> [call_id=<id>](witherror(<exception>)prefix when the call failed)mcp_server_tool_call/mcp_server_tool_result→ same shape with the MCP tool name and outputfunction_approval_request/function_approval_response→ nested call name, approval id, and decisionDesign decisions:
_compaction.pychange;SummarizationStrategy's trigger conditions, summary message shape, and trace links are untouched._select_summary_input_groupsstill selects whole groups by token budget; enriched groups simply carry their true payload, so the budget now reflects what the summarizer actually receives.Before → after (real output from the end-to-end reproduction):
Changes
python/packages/core/agent_framework/_compaction.py_format_summary_contentdispatch over tool-call / result / MCP / approval content types (reuses existing_tool_result_text)._format_summary_messagenow combines structured renderings withMessage.text, falling back to the legacy content-type list only when nothing else is available.python/packages/core/tests/core/test_compaction.pycall_id, mixed text/tool messages, text-only golden rendering, MCP tool details, approval request, approval response.Testing
SummarizationStrategyrun over a 6-message conversation with two tool rounds shows the summarizer input containing names, arguments, results, andcall_ids, and the summary message replacing the excluded originals with trace links intact.test_summarization_strategy_bounds_summary_input_to_complete_groups,test_summary_input_selection_does_not_retokenize_selected_transcript) pass unchanged; group-atomic selection semantics are preserved.Notes for Reviewer
include_tool_detailsopt-in/opt-out.text_reasoningrendering, structured JSON summary output, and no-call_idadjacency pairing ingroup_messagesare left unchanged; the last touches pairing semantics covered by specdocs/specs/004-python-function-calling-loop.mdand is proposed separately.before_run).