Skip to content

dev: include the HTML documentation build in the local lint suite - #25586

Merged
kumarUjjawal merged 2 commits into
apache:mainfrom
kumarUjjawal:dev/21048-local-docs-html-check
Sep 24, 2026
Merged

kumarUjjawal merged 2 commits into
apache:mainfrom
kumarUjjawal:dev/21048-local-docs-html-check

Conversation

@kumarUjjawal

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

./dev/rust_lint.sh runs the non-functional CI checks locally, but not the
documentation website build. Sphinx builds with -W, so one warning fails CI.
A contributor sees that failure only after a push.

What changes are included in this PR?

  • ci/scripts/check_docs_html.sh runs the existing docs/build.sh through uv.
    It works from any directory. It reports a missing uv, cargo,
    cargo-depgraph, dot, or make before the build starts, and installs
    nothing. It has no write mode.
  • dev/rust_lint.sh runs the new script as a read-only step.
  • The "Test doc build" job runs the same script. The path filters of the job
    also watch the script.
  • docs/source/contributor-guide/testing.md documents the command.

docs/build.sh still owns the Sphinx and dependency-graph commands. The Python
dependencies and the Sphinx settings do not change.

What is the testing strategy for this PR?

Local checks with command stubs and a real build:

  • The script starts the same build from the repository root and from other
    directories.
  • -h, --help, --write, --allow-dirty, and an unknown flag all fail
    before any tool runs.
  • Each missing tool gives one message and a non-zero exit.
  • Failures in uv, cargo depgraph, and dot keep their exit code.
  • A real build writes docs/build/html/index.html and changes no tracked file.
  • One Sphinx warning, a page outside every toctree, fails the build.

uv run ./dev/rust_lint.sh passes with the new step.

Are there any user-facing changes?

No. This changes the contributor tooling only.

`./dev/rust_lint.sh` ran every non-functional CI check except the
documentation website build. Contributors found Sphinx warnings only after
pushing.

Add `ci/scripts/check_docs_html.sh`, a wrapper around the existing
`docs/build.sh`, register it in the local lint suite, and call it from the
"Test doc build" job. The wrapper resolves the repository root from its own
location, reports a missing `uv`, `cargo`, `cargo-depgraph`, `dot`, or `make`
before it starts, and installs nothing. It has no write mode.

Part of apache#21048.
@github-actions github-actions Bot added documentation Improvements or additions to documentation development-process Related to development process of DataFusion labels Sep 22, 2026
@codecov-commenter

codecov-commenter commented Sep 22, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.45%. Comparing base (0576a0b) to head (9452e63).
⚠️ Report is 24 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #25586      +/-   ##
==========================================
+ Coverage   82.42%   82.45%   +0.02%     
==========================================
  Files        1139     1140       +1     
  Lines      435372   436589    +1217     
  Branches   435372   436589    +1217     
==========================================
+ Hits       358845   359979    +1134     
- Misses      54826    54849      +23     
- Partials    21701    21761      +60     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@2010YOUY01 2010YOUY01 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested it locally and it's 👌🏼 . Thank you.

[`dev/update_function_docs.sh`]: https://github.com/apache/datafusion/blob/main/dev/update_function_docs.sh
[`ci/scripts/check_generated_docs.sh`]: https://github.com/apache/datafusion/blob/main/ci/scripts/check_generated_docs.sh

## Documentation HTML Build

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small optional advice, we can do it after finishing all scripts.

This testing.md is an entry-point documentation for testing, so I think we don't have to explain all non-functional tests in detail. The detailed documentation could go to individual scripts.

Maybe it's enough to list all of them, and briefly introduce them in 1-2 sentences here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good! I will open a new pr after all of the scripts are merged.

The testing guide is an entry point for contributors, so it now introduces
the check in a few sentences.

The tools the check needs and the files it writes moved into the help text
of `ci/scripts/check_docs_html.sh`.
@kumarUjjawal

Copy link
Copy Markdown
Contributor Author

Thank you @2010YOUY01 for the review

@kumarUjjawal
kumarUjjawal added this pull request to the merge queue Sep 23, 2026
@kumarUjjawal
kumarUjjawal removed this pull request from the merge queue due to a manual request Sep 23, 2026
@kumarUjjawal
kumarUjjawal added this pull request to the merge queue Sep 24, 2026
Merged via the queue into apache:main with commit 63ab2a8 Sep 24, 2026
42 checks passed
@kumarUjjawal
kumarUjjawal deleted the dev/21048-local-docs-html-check branch September 24, 2026 06:38
diegoQuinas pushed a commit to diegoQuinas/datafusion that referenced this pull request Sep 24, 2026
…ache#25586)

## Which issue does this PR close?

- Part of apache#21048.

## Rationale for this change

`./dev/rust_lint.sh` runs the non-functional CI checks locally, but not
the
documentation website build. Sphinx builds with `-W`, so one warning
fails CI.
A contributor sees that failure only after a push.

## What changes are included in this PR?

- `ci/scripts/check_docs_html.sh` runs the existing `docs/build.sh`
through uv.
  It works from any directory. It reports a missing `uv`, `cargo`,
`cargo-depgraph`, `dot`, or `make` before the build starts, and installs
  nothing. It has no write mode.
- `dev/rust_lint.sh` runs the new script as a read-only step.
- The "Test doc build" job runs the same script. The path filters of the
job
  also watch the script.
- `docs/source/contributor-guide/testing.md` documents the command.

`docs/build.sh` still owns the Sphinx and dependency-graph commands. The
Python
dependencies and the Sphinx settings do not change.

## What is the testing strategy for this PR?

Local checks with command stubs and a real build:

- The script starts the same build from the repository root and from
other
  directories.
- `-h`, `--help`, `--write`, `--allow-dirty`, and an unknown flag all
fail
  before any tool runs.
- Each missing tool gives one message and a non-zero exit.
- Failures in `uv`, `cargo depgraph`, and `dot` keep their exit code.
- A real build writes `docs/build/html/index.html` and changes no
tracked file.
- One Sphinx warning, a page outside every toctree, fails the build.

`uv run ./dev/rust_lint.sh` passes with the new step.

## Are there any user-facing changes?

No. This changes the contributor tooling only.
Omega359 pushed a commit to Omega359/arrow-datafusion that referenced this pull request Oct 11, 2026
…ache#25586)

## Which issue does this PR close?

- Part of apache#21048.

## Rationale for this change

`./dev/rust_lint.sh` runs the non-functional CI checks locally, but not
the
documentation website build. Sphinx builds with `-W`, so one warning
fails CI.
A contributor sees that failure only after a push.

## What changes are included in this PR?

- `ci/scripts/check_docs_html.sh` runs the existing `docs/build.sh`
through uv.
  It works from any directory. It reports a missing `uv`, `cargo`,
`cargo-depgraph`, `dot`, or `make` before the build starts, and installs
  nothing. It has no write mode.
- `dev/rust_lint.sh` runs the new script as a read-only step.
- The "Test doc build" job runs the same script. The path filters of the
job
  also watch the script.
- `docs/source/contributor-guide/testing.md` documents the command.

`docs/build.sh` still owns the Sphinx and dependency-graph commands. The
Python
dependencies and the Sphinx settings do not change.

## What is the testing strategy for this PR?

Local checks with command stubs and a real build:

- The script starts the same build from the repository root and from
other
  directories.
- `-h`, `--help`, `--write`, `--allow-dirty`, and an unknown flag all
fail
  before any tool runs.
- Each missing tool gives one message and a non-zero exit.
- Failures in `uv`, `cargo depgraph`, and `dot` keep their exit code.
- A real build writes `docs/build/html/index.html` and changes no
tracked file.
- One Sphinx warning, a page outside every toctree, fails the build.

`uv run ./dev/rust_lint.sh` passes with the new step.

## Are there any user-facing changes?

No. This changes the contributor tooling only.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

development-process Related to development process of DataFusion documentation Improvements or additions to documentation v56.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants