[dezren39]

Fixes #510

Problem

PR #508 (#507) added an up-front host check to the container crawler's validate_crawl_url (pkgs/lab-ops-hub/container/app.py), but it stopped at ord(c) <= 0x20 — C0 controls and space only. The rest of the WHATWG forbidden-host set still fell through and was certified as "a public DNS name", while the Worker's validateCrawlUrl refuses every one of them:

• DEL (0x7F) — also one codepoint short of http.client's own disallowed set, so the crawl died deep inside the fetcher instead of at the validator
• C1 controls (0x80–0x9F)
• NBSP (0xA0)
• <, >, ^, |

Reproduced on main 025794c8 by driving the real Python validator: all nine forms ALLOW, while Node's WHATWG URL parser (the TS validator's first step) rejects every one.

Solution

Extend the existing up-front check in validate_crawl_url to refuse a host containing any character with ord(c) <= 0x20, 0x7F <= ord(c) <= 0xA0, or one of <>^|, with a clear validator error. Nothing else changes: non-ASCII above NBSP (valid IDN hosts, e.g. examéple.com), exam_ple.com, ordinary DNS names, valid IPv6 literals and public targets are still accepted on both sides.

Verification

• New regression test test_refuses_del_c1_nbsp_and_forbidden_host_characters covers each refused class (DEL, C1 low/NEL/high, NBSP, < > ^ |) plus IDN/underscore controls.
• npm test in pkgs/lab-ops-hub: TS 19/19 + Python 14/14 pass.
• Ablation: with only container/app.py reverted, the new test fails (ValueError not raised for the DEL host); restored, the suite is green.
• Differential probe driving both real validators over the issue's full battery (every printable ASCII character, 0x7F–0xA0, é, a combining character, plus the control URLs): full TS/Python parity post-fix.
• ruff format applied (the repo's treefmt Python formatter); diff is exactly the validator check + the new test.

Single-repo: the validator pair lives only in developing-today/code (per #510's cross-repo check).

Note: an extended probe beyond the issue's battery (Unicode space separators, bidi/format controls, U+FFFD) shows a further IDNA-class divergence — filed separately as a follow-up issue rather than expanding this fix.

---

^{Need help on this PR? Tag @codesmith-bot with what you need.}

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 52 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

• Configuration used: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: 9ed0f582-f28c-434a-8427-b446bab24234

📥 Commits

Reviewing files that changed from the base of the PR and between c7a00ee and 177d3c8.

📒 Files selected for processing: 2

• pkgs/lab-ops-hub/container/app.py
• pkgs/lab-ops-hub/test/test_container_validate.py

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-22T00:12:54.081620Z	177d3c8	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.