Skip to content

[Wan2.2 VAE, Qwen-Image-2.1 VAE] Drop redundant input clone in residual up/down blocks - #15029

Open
luca-888 wants to merge 1 commit into
huggingface:mainfrom
luca-888:vae-drop-residual-input-clone
Open

luca-888 wants to merge 1 commit into
huggingface:mainfrom
luca-888:vae-drop-residual-input-clone

Conversation

@luca-888

@luca-888 luca-888 commented Oct 14, 2026 •

Copy link
Copy Markdown

What does this PR do?

Fixes #15040

WanResidualDownBlock and WanResidualUpBlock (and their QwenImage21 copies) do x_copy = x.clone() and only use x_copy for the avg_shortcut at the end. Nothing in between modifies x in place, so the clone is unnecessary. It only keeps an extra activation-sized buffer alive for the whole block. This PR replaces it with x_copy = x.

Outputs are bitwise identical, peak memory goes down, and speed is unchanged:

A100-80GB, released weights, torch 2.10 peak before peak after
Qwen-Image-2.1 decode 1024², bf16 7.20 GiB 6.63 GiB
Qwen-Image-2.1 decode 2048², bf16 26.87 GiB 24.62 GiB
Wan2.2 decode 121×704×1280, fp32 21.94 GiB 20.22 GiB
Wan2.2 decode 121×704×1280, bf16 12.71 GiB 11.85 GiB

Tests

Existing VAE tests, on CPU (torch 2.14.1), same result as on the parent commit 08d6977:

$ pytest tests/models/autoencoders/test_models_autoencoder_wan.py tests/models/autoencoders/test_models_autoencoder_kl_qwenimage21.py
35 passed, 45 skipped, 5 warnings in 14.29s

The skips are the usual CPU ones (accelerator-only, nightly, auth token, unsupported features).

Copies and style:

$ python utils/check_copies.py
(no output, exit 0)
$ ruff check src/diffusers/models/autoencoders/autoencoder_kl_wan.py src/diffusers/models/autoencoders/autoencoder_kl_qwenimage21.py
All checks passed!

Before/after on released weights (A100-80GB, torch 2.10.0+cu130), equal is torch.equal on the outputs:

qi21_bf16_enc_1024            equal=True  peak  2.90 ->  2.72 GiB  t 0.07 -> 0.06 s
qi21_bf16_dec_1024            equal=True  peak  7.20 ->  6.63 GiB  t 0.21 -> 0.21 s
qi21_bf16_enc_2048            equal=True  peak  9.70 ->  8.95 GiB  t 0.24 -> 0.23 s
qi21_bf16_dec_2048            equal=True  peak 26.87 -> 24.62 GiB  t 0.84 -> 0.84 s
wan22_fp32_enc_1f_704x1280    equal=True  peak  4.28 ->  4.15 GiB  t 0.59 -> 0.59 s
wan22_fp32_dec_121f_704x1280  equal=True  peak 21.94 -> 20.22 GiB  t 248.32 -> 248.22 s
wan22_bf16_dec_121f_704x1280  equal=True  peak 12.71 -> 11.85 GiB  t 14.00 -> 13.95 s
wan22_fp32_enc_121f_704x1280  equal=True  peak 11.12 -> 10.58 GiB  t 53.11 -> 53.07 s

Self-review notes

Posted as a comment below.

Before submitting

  • Used an AI agent (Claude Code); ran self-review, notes in the comment below.
  • Read the contributor guideline.

Who can review?

@yiyixuxu @dg845

…al up/down blocks

`WanResidualDownBlock`/`WanResidualUpBlock` (and the QwenImage21 copies)
clone the block input before running the main path, only to feed it to the
`avg_shortcut` branch. Nothing in the main path writes to its input in place,
so the clone just holds an extra activation-sized buffer alive for the whole
block. Alias it instead.

Outputs are bitwise identical; peak activation memory drops by ~10%.
@github-actions github-actions Bot added models size/S PR with diff < 50 LOC labels Oct 14, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Hi @luca-888, thanks for the PR! It does not appear to link an issue it fixes. If this PR addresses an existing issue, please add a closing keyword (e.g. Fixes #1234) to the PR description so the issue is linked. See the contribution guide for more details. If this PR intentionally does not fix a tracked issue, a maintainer can add the no-issue-needed label to silence this reminder.

Please note that PRs without a linked issue are likely to be automatically closed 10 days after this notice.

Once the PR links an issue (or gets the no-issue-needed label), you can ignore this message — it stays here as a comment, but it no longer applies.

@luca-888

Copy link
Copy Markdown
Author

Self-review notes

READY, no blocking issues. Nothing in these blocks writes to x in place, so x_copy = x is safe.

Left for the reviewer:

  1. No new test: outputs don't change, and existing VAE tests cover both blocks.
  2. The feat_cache[idx] = x.clone() calls are left alone; they are a separate case.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

models size/S PR with diff < 50 LOC

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Wan2.2 VAE, Qwen-Image-2.1 VAE] Redundant x.clone() in residual up/down blocks keeps an extra activation alive

1 participant