Skip to content

improve "compressed-tensors" import error raise - #2263

Merged
changwangss merged 3 commits into
mainfrom
wangchang/fix_importerror
Aug 31, 2026
Merged

changwangss merged 3 commits into
mainfrom
wangchang/fix_importerror

Conversation

@changwangss

@changwangss changwangss commented Aug 31, 2026 •

Copy link
Copy Markdown
Contributor

Description

the import error raised too late when "compressed-tensors" didn't install.
issue #2173
the origin.

(.venv) root@5090D:/mnt/disk3/changwa1# CUDA_VISIBLE_DEVICES=0 auto-round --model FLUX.1-dev/ --scheme mxfp8 --format llm_compres
sor --disable_opt_rtn --iters 0
2026-08-31 10:16:10 INFO main.py L299: start to quantize FLUX.1-dev
2026-08-31 10:16:10 INFO autoround.py L221: Auto-routing to model-free quantization (iters=0, disable_opt_rtn=True, supported sch
eme). Pass disable_model_free=True to use the regular flow.
2026-08-31 10:16:10 INFO model_free.py L501: `torch.compile` is disabled, as RTN/OPT-RTN quantizes each layer in a single pass
2026-08-31 10:16:10 INFO model_free.py L645: Model-free source decision: local/snapshot (input is an existing local directory)
2026-08-31 10:16:10 INFO model_free.py L674: Path selected: local directory mode. Reading shards from local path.
2026-08-31 10:16:10 INFO model_free.py L687: Detected diffusion model (no root config.json, found transformer/ subfolder). Only t
he transformer component will be quantized; other sub-components are skipped.
... ...
  Ignored layers (0):
Processing shards:  67%|███████████████████████████████████████████████▎                       | 2/3 [00:48<00:22, 22.05s/shard]2
026-08-31 10:17:29 INFO model_free.py L1004: Memory usage: 'peak_ram': 4.48GB
2026-08-31 10:17:29 INFO model_free.py L1008: Shard 3/3 (diffusion_pytorch_model-00002-of-00003.safetensors):
  Quantized layers (206): single_transformer_blocks.[0-23].attn.to_k, single_transformer_blocks.[0-23].attn.to_q, single_transfor
mer_blocks.[0-23].attn.to_v, single_transformer_blocks.[0-23].proj_out, single_transformer_blocks.[0-24].norm.linear, single_tran
sformer_blocks.[0-24].proj_mlp, transformer_blocks.14.ff.net.0.proj, transformer_blocks.14.ff.net.2, transformer_blocks.14.ff_con
text.net.0.proj, transformer_blocks.14.ff_context.net.2, transformer_blocks.15.attn.to_out.0, transformer_blocks.15.ff.net.0.proj
, transformer_blocks.15.ff.net.2, transformer_blocks.15.ff_context.net.0.proj, transformer_blocks.15.ff_context.net.2, transforme
r_blocks.16.attn.to_out.0, transformer_blocks.16.ff.net.0.proj, transformer_blocks.16.ff.net.2, transformer_blocks.16.ff_context.
net.0.proj, transformer_blocks.16.ff_context.net.2, transformer_blocks.17.attn.to_out.0, transformer_blocks.17.ff.net.0.proj, tra
nsformer_blocks.17.ff.net.2, transformer_blocks.17.ff_context.net.0.proj, transformer_blocks.17.ff_context.net.2, transformer_blo
cks.18.attn.to_out.0, transformer_blocks.18.ff.net.0.proj, transformer_blocks.18.ff.net.2, transformer_blocks.18.ff_context.net.0
.proj, transformer_blocks.18.ff_context.net.2, transformer_blocks.[15-18].attn.add_k_proj, transformer_blocks.[15-18].attn.add_q_
proj, transformer_blocks.[15-18].attn.add_v_proj, transformer_blocks.[15-18].attn.to_add_out, transformer_blocks.[15-18].attn.to_
k, transformer_blocks.[15-18].attn.to_q, transformer_blocks.[15-18].attn.to_v, transformer_blocks.[15-18].norm1.linear, transform
er_blocks.[15-18].norm1_context.linear
  Ignored layers (0):
Processing shards: 100%|███████████████████████████████████████████████████████████████████████| 3/3 [01:18<00:00, 26.26s/shard]
2026-08-31 10:17:29 ERROR config.py L38: Please install compressed-tensors via 'pip install compressed-tensors' to save as llm-compressor format
2026-08-31 10:17:29 ERROR config.py L38: Please install compressed-tensors via 'pip install compressed-tensors' to save
llm_compressor format
Traceback (most recent call last):
  File "/mnt/disk3/changwa1/nunchaku-torch-cu130-ubuntu22/.venv/bin/auto-round", line 10, in <module>
    sys.exit(run())
             ^^^^^
  File "/mnt/disk3/changwa1/auto-round/auto_round/cli/main.py", line 513, in run
    start(argv=command_argv)
  File "/mnt/disk3/changwa1/auto-round/auto_round/cli/main.py", line 244, in start
    tune(args)
  File "/mnt/disk3/changwa1/auto-round/auto_round/cli/main.py", line 408, in tune
    model, folders = autoround.quantize_and_save(  # pylint: disable=no-member
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/disk3/changwa1/auto-round/auto_round/compressors/model_free.py", line 1769, in quantize_and_save
    out_path = self.run()
               ^^^^^^^^^^
  File "/mnt/disk3/changwa1/auto-round/auto_round/compressors/model_free.py", line 1379, in run
    self._write_config_files()
  File "/mnt/disk3/changwa1/auto-round/auto_round/compressors/model_free.py", line 1215, in _write_config_files
    quantization_config = _build_quantization_config(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/disk3/changwa1/auto-round/auto_round/utils/model_free_utils.py", line 2011, in _build_quantization_config
    return _build_mxfp_quantization_config(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/disk3/changwa1/auto-round/auto_round/utils/model_free_utils.py", line 1585, in _build_mxfp_quantization_config
    group_scheme, fmt = _get_mxfp_group_scheme_and_format(actual_bits, actual_dt, ignore)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/disk3/changwa1/auto-round/auto_round/utils/model_free_utils.py", line 1515, in _get_mxfp_group_scheme_and_format
    tmp_qconfig = initialize_quantization(scheme=scheme_name, ignore=ignore)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/disk3/changwa1/auto-round/auto_round/export/export_to_llmcompressor/config.py", line 68, in initialize_quantization
    check_compressed_tensors_supported(raise_error=True)
  File "/mnt/disk3/changwa1/auto-round/auto_round/export/export_to_llmcompressor/config.py", line 40, in check_compressed_tensors
_supported
    raise ImportError(msg) from None
ImportError: Please install compressed-tensors via 'pip install compressed-tensors' to save as llm-compressor format

after this pr.

(.venv) root@5090D:/mnt/disk3/changwa1# CUDA_VISIBLE_DEVICES=0 auto-round --model FLUX.1-dev/ --scheme mxfp8 --format llm_compressor --disable_opt_rtn --iters 0
2026-08-31 10:45:57 ERROR main.py L286: The 'llm_compressor' output format requires the optional dependency 'compressed-tensors'. Install it with: pip install compressed-tensors

Type of Change

Bug fix

Related Issues

Fixes or relates to #

Checklist Before Submitting

  • My code has been tested locally.
  • Documentation has been updated as needed.
  • New or updated tests are included where applicable.
  • The CUDA CI has passed. You can trigger it by commenting /azp run Unit-Test-CUDA-AutoRound.

@changwangss
changwangss marked this pull request as ready for review August 31, 2026 10:29
@changwangss
changwangss requested a review from wenhuach21 August 31, 2026 10:36
changwangss and others added 2 commits August 31, 2026 10:39
Signed-off-by: Wang, Chang <chang1.wang@intel.com>
@AutoRoundBot

Copy link
Copy Markdown
Collaborator

/azp run Unit-Test-CUDA-AutoRound

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@wenhuach21

Copy link
Copy Markdown
Contributor

better log the key info at first, e.g. Please pip install xxx since xxx

@changwangss

Copy link
Copy Markdown
Contributor Author

better log the key info at first, e.g. Please pip install xxx since xxx

improved as following
Please pip install compressed-tensors since the 'llm_compressor' output format requires the optional dependency 'compressed-tensors'.

Signed-off-by: Wang, Chang <chang1.wang@intel.com>
@AutoRoundBot

Copy link
Copy Markdown
Collaborator

/azp run Unit-Test-CUDA-AutoRound

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines will not run the associated pipelines, because the pull request was updated after the run command was issued. Review the pull request again and issue a new run command.

@changwangss

Copy link
Copy Markdown
Contributor Author

/azp run Unit-Test-CUDA-AutoRound

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@changwangss
changwangss merged commit 672eb8e into main Aug 31, 2026
52 checks passed
@changwangss
changwangss deleted the wangchang/fix_importerror branch August 31, 2026 13:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants