Skip to content

programs: keep zstd -l DictID at 0 after a dictionary ID mismatch - #4986

Open
aldo01 wants to merge 1 commit into
meta:devfrom
aldo01:fix-list-dictid-mismatch
Open

aldo01 wants to merge 1 commit into
meta:devfrom
aldo01:fix-list-dictid-mismatch

Conversation

@aldo01

@aldo01 aldo01 commented Nov 18, 2026

Copy link
Copy Markdown

What and why

Fixes #4985.

FIO_analyzeFrames() used dictID == 0 both for "no frame seen yet" and for "frames disagree". After a mismatch reset it to 0, the next frame's ID was accepted again, so zstd -lv printed the wrong value right after warning "Showing dictID 0 instead":

Frames Before After
A, B, C warning, DictID: C warning, DictID: 0
A, B, A warning, DictID: A warning, DictID: 0
no-dict, A no warning, DictID: A warning, DictID: 0
A, no-dict warning, DictID: 0 warning, DictID: 0
A, A DictID: A DictID: A

The fix takes the first frame's ID when numActualFrames == 0 and records a mismatch in a separate multipleDictIDs flag. As a result, DictID stays 0 whatever the frame order, and the warning is printed once. The warning also gets the missing trailing newline; without it, the warning ran into the next line of output.

Only --list output changes. Compression and decompression are unaffected.

Checklist

  • I searched open pull requests and issues: this problem is not already being addressed.
  • Bug fix: it comes with a reproducer or a test that fails without the fix.
  • Performance change: not applicable.
  • I stated below whether AI or other automated tools were used.

Test

New tests/cli-tests/dictionaries/list-dictid.sh uses the suite's two trained dictionaries plus a frame without a dictionary. It checks the reported DictID for the orders a b, a b a, none a, a none and a a none, which must all give 0, and for a a, which must give A's ID. It also checks that the warning appears exactly once and on its own line.

On macOS arm64, default make flags:

Tools used

GitHub Copilot (AI) helped find the problem while auditing zstd --list on concatenated multi-frame files, and helped write the fix and the test. The reproducer and the test results above were checked on a local build. The commit carries a Co-authored-by: Copilot trailer.

FIO_analyzeFrames() used dictID 0 both for "no frame seen yet" and for
"frames disagree". After a mismatch reset it to 0, the next frame's ID was
accepted again, so frames A,B,C reported C and A,B,A reported A, right
after warning "Showing dictID 0 instead". A no-dictionary frame followed by
a dictionary frame was not detected as a mismatch at all.

Take the first frame's ID from numActualFrames == 0 and record the mismatch
in a separate flag, so DictID stays 0 regardless of frame order and the
warning is printed once. Also terminate the warning with a newline.

Fixes meta#4985

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@meta-cla

meta-cla Bot commented Nov 18, 2026

Copy link
Copy Markdown

Hi @aldo01!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

zstd -l -v reports the wrong DictID for multi-frame files with different dictionaries

1 participant