Repository navigation
fix: layer_config for lm_head takes effect in model-free mode - #2182
Conversation
Co-authored-by: xin3he <83260933+xin3he@users.noreply.github.com>
|
/azp run Unit-Test-CUDA-AutoRound |
|
Azure Pipelines successfully started running 1 pipeline(s). |
…nfig keys Co-authored-by: xin3he <83260933+xin3he@users.noreply.github.com>
Co-authored-by: xin3he <83260933+xin3he@users.noreply.github.com>
Co-authored-by: xin3he <83260933+xin3he@users.noreply.github.com>
|
Extended the lm_head name variants in |
|
@copilot model free应该还有其他地方涉及到lm_head的处理,都考虑全面各种已知的名称,优化代码。 |
Co-authored-by: xin3he <83260933+xin3he@users.noreply.github.com>
Updated in commit All remaining hardcoded |
|
@copilot resolve the merge conflicts in this pull request |
…fig-priority # Conflicts: # auto_round/compressors/model_free.py Co-authored-by: xin3he <83260933+xin3he@users.noreply.github.com>
Merge conflicts resolved. The |
|
@copilot
|
Co-authored-by: xin3he <83260933+xin3he@users.noreply.github.com>
Both issues addressed in the latest commit:
|
|
@copilot
|
…er.should_ignore Co-authored-by: xin3he <83260933+xin3he@users.noreply.github.com>
Fixed. The previous approach ( The fix is at the |
|
/azp run Unit-Test-CUDA-AutoRound |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run Unit-Test-CUDA-AutoRound |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Description
In model-free quantization, explicitly setting
lm_headinlayer_config(e.g.--layer_config "{lm_head:{bits:4}}"or--layer_config "{lm_head:{scheme:W4A16}}") was silently ignored —lm_headwas always skipped unless--quant_lm_headwas also set.Root cause:
_build_ignore_patterns()unconditionally appended both"lm_head"and"head"(DeepSeek v4 compatibility) toignore_patternswheneverquant_lm_head=False. Since"head"is a substring pattern that matches"lm_head", fixing only the"lm_head"entry was insufficient.Fix:
_build_ignore_patterns(), check whether any key inself.layer_configcontains"lm_head"before appending either"lm_head"or"head"to the ignore list — user-specifiedlayer_confignow takes highest priority over defaultstest_layer_config_lm_head_bits_takes_effectandtest_layer_config_lm_head_scheme_takes_effectType of Change
Bug fix
Related Issues
Checklist Before Submitting
/azp run Unit-Test-CUDA-AutoRound.