Add Spark physical plan only to completed spans

[aboitreaud]

What Does This Do

We send heartbeat spans every 2mins for accurate step and duration tracking. Physical plan is the largest span attr by far. We don't need to send it on running spans.

Motivation

Additional Notes

Contributor Checklist

• Format the title according to the contribution guidelines
• Assign the type: and (comp: or inst:) labels in addition to any other useful labels
• Avoid using close, fix, or any linking keywords when referencing an issue
  Use solves instead, and assign the PR milestone to the issue
• Update the CODEOWNERS file on source file addition, migration, or deletion
• Update public documentation with any new configuration flags or behaviors
• Once approved, use merge queue to merge the PR

Jira ticket: [PROJ-IDENT]

[dd-octo-sts]

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

• Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f778ae462c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[datadog-prod-us1-3]

More details

The retained SQL start event supplies the physical plan immediately before span completion, keeping it off running-span heartbeats without losing it from completed spans.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit f778ae4 · @DataDog review to ask questions}

[datadog-prod-us1-3]

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 68.82% (+9.73%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 6c12269 | Docs | Give us feedback!}

[dd-octo-sts]

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	13.98 s	14.04 s	[-1.2%; +0.4%] (no difference)
startup:insecure-bank:tracing:Agent	13.01 s	13.03 s	[-0.9%; +0.6%] (no difference)
startup:petclinic:appsec:Agent	17.01 s	16.82 s	[+0.2%; +2.1%] (maybe worse)
startup:petclinic:iast:Agent	16.97 s	17.07 s	[-1.5%; +0.3%] (no difference)
startup:petclinic:profiling:Agent	16.56 s	16.80 s	[-2.3%; -0.5%] (maybe better)
startup:petclinic:sca:Agent	16.97 s	16.98 s	[-1.1%; +0.9%] (no difference)
startup:petclinic:tracing:Agent	16.25 s	16.21 s	[-0.8%; +1.3%] (no difference)

Commit: 6c12269a · CI Pipeline · Benchmarking Platform UI

---

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

[dd-octo-sts]

/merge

[gh-worker-devflow-routing-ef8351]

View all feedbacks in Devflow UI.

2026-09-28 17:19:39 UTC ℹ️ Start processing command /merge

---

2026-09-28 17:19:43 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 1h (p90).

---

2026-09-28 17:59:31 UTC ❌ MergeQueue: The build pipeline contains failing jobs for this merge request

Build pipeline has failing jobs for 1bdb7e5:

• test_inst_latest: [21, 2/6]
• test_inst_latest: [17, 2/6]

⚠️ Do NOT retry failed jobs directly (why?).

What to do next?

• Investigate the failures and when ready, re-add your pull request to the queue!
• If your PR checks are green, try to rebase/merge. It might be because the CI run is a bit old.
• Any question, go check the FAQ.

Details

Since those jobs are not marked as being allowed to fail, the pipeline will most likely fail.
Therefore, and to allow other builds to be processed, this merge request has been rejected and the pipeline got canceled.

[dd-octo-sts]

/merge

[gh-worker-devflow-routing-ef8351]

View all feedbacks in Devflow UI.

2026-09-29 02:37:38 UTC ℹ️ Start processing command /merge

---

2026-09-29 02:37:43 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 1h (p90).

---

2026-09-29 03:46:39 UTC ℹ️ MergeQueue: This merge request was merged