Skip to content

Fix: Make LocalFileSystem.match() work for bare relative globs in the current working directory - #41388

Open
Naman-Katiyar wants to merge 3 commits into
apache:masterfrom
Naman-Katiyar:fix-localfilesystem-relative-glob
Open

Naman-Katiyar wants to merge 3 commits into
apache:masterfrom
Naman-Katiyar:fix-localfilesystem-relative-glob

Conversation

@Naman-Katiyar

Copy link
Copy Markdown

Fixes #41375

LocalFileSystem._list() returned nothing for an empty directory prefix because os.path.exists('') is False, so FileSystem.match() found no files for bare relative globs such as 'users.json*' in the current working directory. This change normalises an empty prefix to '.' and strips the leading './' from the yielded paths so they line up with the caller's relative pattern. A regression test matching a bare relative glob in the current working directory is included.

… current working directory

Fixes apache#41375

LocalFileSystem._list() returned nothing for an empty directory prefix because os.path.exists('') is False, so FileSystem.match() found no files for bare relative globs such as 'users.json*' in the current working directory. Normalize an empty prefix to '.' and strip the leading './' from the yielded paths so they line up with the caller's relative pattern.

Signed-off-by: Naman Katiyar <namankatiyar120@gmail.com>
…rking directory

Adds a regression test for the LocalFileSystem fix: matching a bare relative glob such as 'users.json*' in the current working directory previously returned no files. The test reuses the existing tmpdir_as_cwd helper.

Signed-off-by: Naman Katiyar <namankatiyar120@gmail.com>
Copilot AI balanced review requested due to automatic review settings November 16, 2026 06:02

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@github-actions github-actions Bot added python io size/s S: 10-49 lines changed labels Nov 16, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📋 Review Summary

This PR fixes LocalFileSystem._list() when matching bare relative globs in the current working directory (where dir_or_prefix is '') by normalizing the empty prefix to '.' and stripping the leading './' from yielded paths. However, there is a critical IndentationError in sdks/python/apache_beam/io/localfilesystem.py that prevents the module from being imported.

🔍 General Feedback

  • The approach of normalizing '' to '.' in LocalFileSystem._list() and stripping the leading '.' + os.sep prefix cleanly resolves the issue and preserves relative paths for FileSystem.match().
  • Please fix the indentation inside the try block in LocalFileSystem._list() so that the module parses valid Python and yields FileMetadata unconditionally regardless of strip_dot_prefix.

💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.

try:
for f in list_files(dir_or_prefix):
try:
if strip_dot_prefix and f.startswith('.' + os.sep):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Critical: if strip_dot_prefix ... is indented at the same level (8 spaces) as try: on line 133, which raises an IndentationError on import. It needs to be indented to 10 spaces (and line 135 to 12 spaces).

Suggested change
if strip_dot_prefix and f.startswith('.' + os.sep):
if strip_dot_prefix and f.startswith('.' + os.sep):

for f in list_files(dir_or_prefix):
try:
if strip_dot_prefix and f.startswith('.' + os.sep):
f = f[len('.' + os.sep):]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Critical: Along with indenting the if statement on line 134 to 10 spaces, this body statement must be indented to 12 spaces so that yield FileMetadata(...) on line 136 remains outside the if block and executes when strip_dot_prefix is False.

Suggested change
f = f[len('.' + os.sep):]
f = f[len('.' + os.sep):]

@github-actions

Copy link
Copy Markdown
Contributor

Assigning reviewers:

R: @tvalentyn for label python.
Backup reviewer: shunping (if the main reviewer is unresponsive, you can escalate to them by tagging their username)

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

Indent the strip_dot_prefix check and its body inside the inner try block, leaving the yield outside the if so it still runs when strip_dot_prefix is False.

Signed-off-by: Naman Katiyar <namankatiyar120@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: [Python] LocalFileSystem.match() returns empty results for relative glob patterns in the current working directory

2 participants