Repository navigation
dev: include the HTML documentation build in the local lint suite - #25586
Conversation
`./dev/rust_lint.sh` ran every non-functional CI check except the documentation website build. Contributors found Sphinx warnings only after pushing. Add `ci/scripts/check_docs_html.sh`, a wrapper around the existing `docs/build.sh`, register it in the local lint suite, and call it from the "Test doc build" job. The wrapper resolves the repository root from its own location, reports a missing `uv`, `cargo`, `cargo-depgraph`, `dot`, or `make` before it starts, and installs nothing. It has no write mode. Part of apache#21048.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #25586 +/- ##
==========================================
+ Coverage 82.42% 82.45% +0.02%
==========================================
Files 1139 1140 +1
Lines 435372 436589 +1217
Branches 435372 436589 +1217
==========================================
+ Hits 358845 359979 +1134
- Misses 54826 54849 +23
- Partials 21701 21761 +60 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
2010YOUY01
left a comment
There was a problem hiding this comment.
Tested it locally and it's 👌🏼 . Thank you.
| [`dev/update_function_docs.sh`]: https://github.com/apache/datafusion/blob/main/dev/update_function_docs.sh | ||
| [`ci/scripts/check_generated_docs.sh`]: https://github.com/apache/datafusion/blob/main/ci/scripts/check_generated_docs.sh | ||
|
|
||
| ## Documentation HTML Build |
There was a problem hiding this comment.
Small optional advice, we can do it after finishing all scripts.
This testing.md is an entry-point documentation for testing, so I think we don't have to explain all non-functional tests in detail. The detailed documentation could go to individual scripts.
Maybe it's enough to list all of them, and briefly introduce them in 1-2 sentences here.
There was a problem hiding this comment.
sounds good! I will open a new pr after all of the scripts are merged.
The testing guide is an entry point for contributors, so it now introduces the check in a few sentences. The tools the check needs and the files it writes moved into the help text of `ci/scripts/check_docs_html.sh`.
|
Thank you @2010YOUY01 for the review |
…ache#25586) ## Which issue does this PR close? - Part of apache#21048. ## Rationale for this change `./dev/rust_lint.sh` runs the non-functional CI checks locally, but not the documentation website build. Sphinx builds with `-W`, so one warning fails CI. A contributor sees that failure only after a push. ## What changes are included in this PR? - `ci/scripts/check_docs_html.sh` runs the existing `docs/build.sh` through uv. It works from any directory. It reports a missing `uv`, `cargo`, `cargo-depgraph`, `dot`, or `make` before the build starts, and installs nothing. It has no write mode. - `dev/rust_lint.sh` runs the new script as a read-only step. - The "Test doc build" job runs the same script. The path filters of the job also watch the script. - `docs/source/contributor-guide/testing.md` documents the command. `docs/build.sh` still owns the Sphinx and dependency-graph commands. The Python dependencies and the Sphinx settings do not change. ## What is the testing strategy for this PR? Local checks with command stubs and a real build: - The script starts the same build from the repository root and from other directories. - `-h`, `--help`, `--write`, `--allow-dirty`, and an unknown flag all fail before any tool runs. - Each missing tool gives one message and a non-zero exit. - Failures in `uv`, `cargo depgraph`, and `dot` keep their exit code. - A real build writes `docs/build/html/index.html` and changes no tracked file. - One Sphinx warning, a page outside every toctree, fails the build. `uv run ./dev/rust_lint.sh` passes with the new step. ## Are there any user-facing changes? No. This changes the contributor tooling only.
…ache#25586) ## Which issue does this PR close? - Part of apache#21048. ## Rationale for this change `./dev/rust_lint.sh` runs the non-functional CI checks locally, but not the documentation website build. Sphinx builds with `-W`, so one warning fails CI. A contributor sees that failure only after a push. ## What changes are included in this PR? - `ci/scripts/check_docs_html.sh` runs the existing `docs/build.sh` through uv. It works from any directory. It reports a missing `uv`, `cargo`, `cargo-depgraph`, `dot`, or `make` before the build starts, and installs nothing. It has no write mode. - `dev/rust_lint.sh` runs the new script as a read-only step. - The "Test doc build" job runs the same script. The path filters of the job also watch the script. - `docs/source/contributor-guide/testing.md` documents the command. `docs/build.sh` still owns the Sphinx and dependency-graph commands. The Python dependencies and the Sphinx settings do not change. ## What is the testing strategy for this PR? Local checks with command stubs and a real build: - The script starts the same build from the repository root and from other directories. - `-h`, `--help`, `--write`, `--allow-dirty`, and an unknown flag all fail before any tool runs. - Each missing tool gives one message and a non-zero exit. - Failures in `uv`, `cargo depgraph`, and `dot` keep their exit code. - A real build writes `docs/build/html/index.html` and changes no tracked file. - One Sphinx warning, a page outside every toctree, fails the build. `uv run ./dev/rust_lint.sh` passes with the new step. ## Are there any user-facing changes? No. This changes the contributor tooling only.
Which issue does this PR close?
Rationale for this change
./dev/rust_lint.shruns the non-functional CI checks locally, but not thedocumentation website build. Sphinx builds with
-W, so one warning fails CI.A contributor sees that failure only after a push.
What changes are included in this PR?
ci/scripts/check_docs_html.shruns the existingdocs/build.shthrough uv.It works from any directory. It reports a missing
uv,cargo,cargo-depgraph,dot, ormakebefore the build starts, and installsnothing. It has no write mode.
dev/rust_lint.shruns the new script as a read-only step.also watch the script.
docs/source/contributor-guide/testing.mddocuments the command.docs/build.shstill owns the Sphinx and dependency-graph commands. The Pythondependencies and the Sphinx settings do not change.
What is the testing strategy for this PR?
Local checks with command stubs and a real build:
directories.
-h,--help,--write,--allow-dirty, and an unknown flag all failbefore any tool runs.
uv,cargo depgraph, anddotkeep their exit code.docs/build/html/index.htmland changes no tracked file.uv run ./dev/rust_lint.shpasses with the new step.Are there any user-facing changes?
No. This changes the contributor tooling only.