Skip to content

dev: include Markdown link checks in the local lint suite - #25287

Merged
kumarUjjawal merged 2 commits into
apache:mainfrom
kumarUjjawal:dev/21048-local-markdown-link-check
Sep 14, 2026
Merged

kumarUjjawal merged 2 commits into
apache:mainfrom
kumarUjjawal:dev/21048-local-markdown-link-check

Conversation

@kumarUjjawal

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

dev/rust_lint.sh is the local mirror of the CI lint jobs, but it does not run the Markdown link check. CI runs ci/scripts/markdown_link_check.sh in dev.yml. A developer can push a broken internal link after a clean local lint run and then see the failure only in CI. This PR adds the existing checker to the local suite.

What changes are included in this PR?

  • dev/rust_lint.sh runs ci/scripts/markdown_link_check.sh as a read-only step, after the workflow install check and before the Rust documentation build. The --write and --allow-dirty flags never reach it.
  • The runner loads LYCHEE_VERSION from ci/scripts/utils/tool_versions.sh and installs that version when lychee is missing. An installed lychee is used as is, which is the runner's existing policy for other tools.
  • ci/scripts/markdown_link_check.sh becomes executable, because the runner invokes each registered script directly. The script body, lychee.toml, the file selection, and the GitHub workflow do not change.
  • docs/source/contributor-guide/testing.md documents the new behavior and keeps the standalone instructions.

What is the testing strategy for this PR?

  • A disposable fixture with stubbed steps and tools: the checker runs once with no arguments in check mode and in both write modes, a missing lychee triggers exactly the pinned install command, a present lychee triggers no install, and a checker failure stops the suite before the later steps.
  • A disposable clone with an injected broken internal link: the standalone checker and ./dev/rust_lint.sh both fail with the same exit code and name the file. After the fix, the full suite passes.
  • The real checker and the full ./dev/rust_lint.sh pass on this branch with lychee 0.23.0.

Are there any user-facing changes?

No. Developers get the link check in the local lint suite. CI is unchanged.

🤖 Generated with Claude Code

`dev/rust_lint.sh` now runs `ci/scripts/markdown_link_check.sh` as a
read-only step, after the workflow install check and before the Rust
documentation build. The runner loads `LYCHEE_VERSION` from
`ci/scripts/utils/tool_versions.sh` and installs that pinned version
when `lychee` is missing. An already installed `lychee` is used as is,
matching the runner's existing policy for other tools.

The checker becomes executable so the runner can invoke it directly, as
it does for every other registered script. Its body, configuration,
file selection, and the GitHub workflow are unchanged, and `--write`
never reaches the link check.

Partial progress on apache#21048.
@github-actions github-actions Bot added documentation Improvements or additions to documentation development-process Related to development process of DataFusion labels Sep 14, 2026
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.90%. Comparing base (c2cf289) to head (0c1ac9d).

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #25287      +/-   ##
==========================================
- Coverage   81.90%   81.90%   -0.01%     
==========================================
  Files        1134     1134              
  Lines      425261   425261              
  Branches   425261   425261              
==========================================
- Hits       348325   348320       -5     
- Misses      56295    56297       +2     
- Partials    20641    20644       +3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@kumarUjjawal

Copy link
Copy Markdown
Contributor Author

Thank you @martin-g for the review.

@kumarUjjawal
kumarUjjawal added this pull request to the merge queue Sep 14, 2026
Merged via the queue into apache:main with commit afaee3d Sep 14, 2026
42 checks passed
@kumarUjjawal
kumarUjjawal deleted the dev/21048-local-markdown-link-check branch September 14, 2026 15:25
Omega359 pushed a commit to Omega359/arrow-datafusion that referenced this pull request Oct 11, 2026
## Which issue does this PR close?

- Part of apache#21048. The issue stays open for the other checklist items.

## Rationale for this change

`dev/rust_lint.sh` is the local mirror of the CI lint jobs, but it does
not run the Markdown link check. CI runs
`ci/scripts/markdown_link_check.sh` in `dev.yml`. A developer can push a
broken internal link after a clean local lint run and then see the
failure only in CI. This PR adds the existing checker to the local
suite.

## What changes are included in this PR?

- `dev/rust_lint.sh` runs `ci/scripts/markdown_link_check.sh` as a
read-only step, after the workflow install check and before the Rust
documentation build. The `--write` and `--allow-dirty` flags never reach
it.
- The runner loads `LYCHEE_VERSION` from
`ci/scripts/utils/tool_versions.sh` and installs that version when
`lychee` is missing. An installed `lychee` is used as is, which is the
runner's existing policy for other tools.
- `ci/scripts/markdown_link_check.sh` becomes executable, because the
runner invokes each registered script directly. The script body,
`lychee.toml`, the file selection, and the GitHub workflow do not
change.
- `docs/source/contributor-guide/testing.md` documents the new behavior
and keeps the standalone instructions.

## What is the testing strategy for this PR?

- A disposable fixture with stubbed steps and tools: the checker runs
once with no arguments in check mode and in both write modes, a missing
`lychee` triggers exactly the pinned install command, a present `lychee`
triggers no install, and a checker failure stops the suite before the
later steps.
- A disposable clone with an injected broken internal link: the
standalone checker and `./dev/rust_lint.sh` both fail with the same exit
code and name the file. After the fix, the full suite passes.
- The real checker and the full `./dev/rust_lint.sh` pass on this branch
with `lychee` 0.23.0.

## Are there any user-facing changes?

No. Developers get the link check in the local lint suite. CI is
unchanged.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

development-process Related to development process of DataFusion documentation Improvements or additions to documentation v56.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants