Repository navigation
fix(crewai): complete framework lifecycle and provider composition - #574
Merged
Merged
Conversation
Trace actual sync/async crew execution instead of lazy stream factories; observe native and structured tool entry points with scoped deduplication and privacy projection. Add real loopback HTTP regression contracts and retain the legacy floor. Co-Authored-By: Claude Code <noreply@anthropic.com>
Add version-gated Agent and Flow lifecycle spans, explicit result projection, and maintained OpenAI composition contracts on the declared floor and current CrewAI. Document provider usage accounting, lazy context, privacy, and native cancellation semantics. Co-Authored-By: Claude Code <noreply@anthropic.com>
Trace start, listener and router execution under the owning Flow with safe explicit inputs and outputs. Preserve native awaits, cancellation, suppression and exact hook restoration, excluding private AgentExecutor flows on both early and current CrewAI. Add real multi-step, branching, failure, concurrent and privacy regressions, and cover the 1.15.0 node-contract floor in CI. Co-Authored-By: Claude Code <noreply@anthropic.com>
Capture context at legacy async worker binding, preserving parentage and suppression without replacing native execution. Cover real HTTP task execution, provider usage, cancellation, errors, native wrappers and restoration. Co-Authored-By: Claude Code <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this fixes
The original repair preserves CrewAI's lazy streaming lifecycle and traces native tool execution that bypasses
ToolUsage._use. Release review then reproduced two additional coverage gaps: standalone Agent/Flow executions emitted no framework spans, and framework-only instrumentation provided no per-call LLM attribution.This follow-up addresses all three review findings, including the stale README inventory:
Agent.kickoff/kickoff_asynccoverage for CrewAI 1.15+. Itsakickoffalias and coroutine-factory path do not create duplicate spans.Flow.kickoff_asyncmethod, covering synchronous kickoff and async aliases without adding synthetic overrides to inherited classes. Skip privateis_crewai_internalexecutor Flows, but retain nested user Flow spans.LiteAgentOutput.raw.traceAI-openaiadapter rather than adding another LLM event listener. Actual OpenAI SDK requests provide per-call model/provider, content and reported usage beneath Task/Agent spans on both CrewAI 0.41.1 and 1.15.35.Native cancellation and version boundaries
Cancelling an Agent cannot forcibly terminate a tool already running in a native worker thread. The Agent span ends with ERROR; the tool span remains open until the worker actually completes. Tests compare uninstrumented and instrumented behavior and explicitly release/join the worker. The adapter does not change the application's cancellation semantics.
Standalone Agent hooks target CrewAI 1.15+. The declared 0.41.1 floor retains Crew/Task/ToolUsage coverage. For the intermediate CrewAI 0.152.0 LiteLLM transport, this OpenAI composition does not expose streamed per-call usage: use non-streaming for accounting or upgrade to the native OpenAI route. Four optional composition cases explicitly skip that documented limitation; the latest/floor streaming cases pass.
Framework-only instrumentation still does not create LLM spans by itself. Per-call attribution requires enabling both adapters as documented, with the same tracer provider and privacy settings. Other provider transports require their maintained provider adapter; this is not universal LLM-transport coverage.
Validation
All test execution ran in E2B with synthetic credentials, loopback provider servers, isolated homes and network namespaces with no outbound network during tests. No local test suite, real provider account, registry publishing, release action, merge, or main-branch change was performed.
Red-before evidence:
q1104-pr574-refix-r1-repro: independently reproduced all three missing runtime gates on 1.15.35 and the missing LLM gate on 0.41.1.q1104-pr574-refix-r1-regression-before: 19 latest Agent/Flow/documentation regression cases failed before the implementation; the maintained provider composition already supplied the missing LLM coverage.Passing validation:
q1104-pr574-refix-r1-finalq1104-pr574-refix-r1-pass310q1104-pr574-refix-r1-floor-locked3q1104-pr574-refix-r1-pass310q1104-pr574-refix-r1-finalThe final run also passed changed-file Ruff checks and dependency checks. The Python 3.10 runs preceded fixture-import cleanup and the 0.152.0-only skip/documentation update; runtime implementation is identical. Every sandbox was deleted and its disappearance verified.
Intermediate runs are not represented as passing: a version-import naming collision was reproduced and fixed; an initially unbounded cancellation test timed out, and independent native/instrumented controls established the worker-thread behavior before replacing it with bounded cleanup. The first optional 0.152.0 streaming composition run exposed the limitation documented above.
Competitor comparison
This closes the assigned lifecycle and attribution gates, not a claim of overall superiority or complete feature parity.
Follow-up into
devonly. Pushed for renewed qualification; not merged or released.Supported-floor async context follow-up (November 5)
Commit
b0ff09e7addresses the remaining independently verified floor blocker. CrewAI 0.41.1 starts async Task workers without copying context: the old head reproduced an orphan Task span and exported Task/LLM input/output even inside explicit suppression._execute_task_asyncas its worker target. Preserve OTel parentage, suppression, baggage and application ContextVars without replacingexecute_async, modifying Task instances or globally patching threads.Cloud evidence:
p1105-crewai-context-red-green-01reproduced 2 failed / 1 passed independent controls against the original implementation and 2 failed / 2 passed newly added suppression regressions before applying the patch. With the runtime fix, all eight independent controls pass on each of 0.41.1, 1.15.0 and latest 1.15.35. The three shared suites each pass 33 tests. The package matrix passed 414/489/489 cases, with 75 expected floor API skips and one new harness-only failure in each leg: comparing wrapt's bound view to the stored descriptor. The final commit corrects that assertion to inspect the class dictionary and corrects the fixture-import lint idiom; no assertion was removed or timeout raised. The runtime implementation is identical to the E2B-tested implementation.That E2B run is not represented as fully green. Final-commit GitHub compatibility run 41952376891 passed all six legs: 415 package + 33 shared passed, 75 expected API skips for each 0.41.1 leg; 490 package + 33 shared passed, no skips for each 1.15.0 and 1.15.35 leg. That is 2,988 passed / 150 expected skips / zero failures across Python 3.10.22 and 3.12.15/16 on Ubuntu 24.04. This validates the corrected descriptor-identity test on the exact pushed head. General Python run 41952601949, integration checks and the other latest workflows also passed; the earlier cancelled Python run and its failed aggregate are superseded, not hidden. Final Ruff was not separately rerun after the fixture-import correction. All E2B execution used synthetic credentials, isolated homes and loopback-only network namespaces. The single sandbox was killed and
get_info_404=trueverified. Prior qualification/security/competitor evidence above remains historical evidence, not a blanket current-head qualification. No merge or release performed.🤖 Generated with Claude Code