Add Local Scripts to Reproduce Full CI

Is your feature request related to a problem or challenge?

It would be great to have a local script to run full CI, for all functional and non-functional(e.g. lints) tests, this helps development process for human and AI agents.

./dev/ci.sh      # Run full CI locally

Here is a previous work that provides a single script to run some non-functional tests

Add Local Scripts to Reproduce Full CI and Perform Auto-Fixes #19227

The remaining work is to

Continue supporting more non-functional tests
Create a similar script for all functional tests (rust tests, sqllogictests, extended tests, ...)

(might use xtask instead of scripts, see #23414)
~~### Tracker for non-functional tests~~

./dev/rust_lint.sh should fully reproduce non-functional CI locally.
./dev/rust_lint.sh --write --allow-dirty should do best-effort fixing where applicable.

Today the script covers cargo fmt, clippy, Cargo.toml formatting, license headers, typos, markdown prettier, and cargo doc.

Remaining CI checks that could be migrated similarly:

Potential Improvements to `rust_lint.sh`

Better naming.
Support running subsets, so developers and AI agents can run only checks likely affected by local changes.

feat(CI): Introduce `xtask` to keep local test and CI commands in sync (

ci: share extended test commands through xtask (apache#25256)

dev: include Markdown link checks in the local lint suite (apache#25287)

dev: include ASF status checks in the local lint suite (apache#25313)

Future improvements #25359 (review)

dev: include security auditing in the local lint suite (apache#25359)

dev: include dependency checks in the local lint suite (apache#25399)

I'd like to take the Large file check item.

Plan, following #25359: move the size check out of the inline run: block in .github/workflows/large_files.yml into ci/scripts/check_large_files.sh. The script takes a base ref and defaults to the merge base with origin/main, so it can run on a local branch; CI passes the PR or merge-queue SHAs it already has. The workflow's run: line becomes a call to the script so CI and the local suite share one definition, and dev/rust_lint.sh gets it as a read-only step.