Add CI smoke tests that each service boots successfully #6220

Task Summary

CI builds each service's distributable and runs its unit tests, but never starts the packaged service. Runtime classpath problems that compile cleanly and pass unit tests — but crash the moment main runs — slip through. Two recent dependency upgrades did exactly this on main:

• texera-web — after chore(deps): upgrade Apache Hadoop to 3.4.3 to resolve hadoop-common CVEs #6201 bumped Hadoop to 3.4.3, texera-web crashed on boot with NoClassDefFoundError: com/sun/jersey/core/provider/jaxb/AbstractRootElementProvider (Hadoop 3.4 ships jersey-json via com.github.pjfanning, evading the existing com.sun.jersey exclusions; the Jersey 1.x providers shadow Dropwizard's Jersey 2). Reported in texera-web fails to start after Hadoop 3.4.3 upgrade: NoClassDefFoundError from pjfanning jersey-json #6204, fixed by fix(deps): exclude pjfanning jersey-json from hadoop to unbreak service startup #6205.
• computing-unit-managing-service — after fix(deps): upgrade Apache Arrow to 19.0.0 and Netty to 4.2.15.Final #6185 upgraded Arrow to 19, the service crashed at startup with JsonMappingException: Scala module 2.18.8 requires Jackson Databind >= 2.18.0 and < 2.19.0 — Found 2.21.0. Reported in computing-unit-managing-service fails to start: jackson-databind 2.21.0 incompatible with jackson-module-scala 2.18.8 #6206, fixed by fix(deps): pin jackson-databind to 2.18.8 in computing-unit-managing-service #6209.

Both were found and fixed manually after landing on main. A per-service boot check in CI would have caught them before merge.

Current state

The platform job builds each service (sbt "${sbt_project}/dist" "${sbt_project}/jacoco"), unzips the dist, and validates LICENSE/NOTICE — but there is no step that launches the built service and confirms it starts. The affected services: config-service, access-control-service, file-service, computing-unit-managing-service, workflow-compiling-service, notebook-migration-service, plus the amber texera-web app.

Suggested direction

Add a CI step/job that launches each packaged service from its built dist and asserts it reaches a started state (health endpoint responds, or a "server started" log line, or simply the process survives N seconds) rather than crashing on boot.

• Both incidents crashed early in initialization (Dropwizard Jersey setup / ComputingUnitManagingService.initialize), before touching a database — so even a shallow "launch and assert the process doesn't die with a linkage / NoClassDefFoundError / module error within N seconds" check catches them, without a fully provisioned backing environment.
• Services that need backing infra to start fully (postgres, minio, …) can reuse what the platform / amber-integration jobs already provision, or run a fail-fast boot mode.

References — incidents: #6204 / #6205 (texera-web), #6206 / #6209 (computing-unit-managing-service); triggering upgrades: #6201 (Hadoop), #6185 (Arrow).

Task Type

• Refactor / Cleanup
• DevOps / Deployment / CI
• Testing / QA
• Documentation
• Performance
• Other