You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Matrix-Game-3: --fa_version 0 (SDPA) is not implemented; attention() asserts a flash-attn install instead of falling back #86
Matrix-Game-3/wan/modules/attention.py no longer has the SDPA fallback that the original Wan attention module ships. The non-FA3 branch of flash_attention() is:
Without a flash-attn install, every attention call dies on that assertion, so the model cannot run at all on a machine where flash-attn does not build (the CLI's --fa_version 0 suggests SDPA is meant to be selectable).
With flash-attn 2 installed, --fa_version 0 still runs the flash-attn 2 kernel: version=0 is neither 3 nor handled anywhere, so it falls through to the same branch. --fa_version 3 with only FA2 installed does degrade with a warning, which is the configuration that worked for us on an H100.
Suggested fix: restore the torch.nn.functional.scaled_dot_product_attention path for version == 0 (and as the fallback when neither flash-attn build is importable), with the warning the upstream Wan module prints. Happy to open a PR if that is the intended behaviour.
Related, still open as of today: the multi-GPU path passes fa_version= to attention() whose parameter is named version (#80, fix in #61), so --ulysses_size > 1 raises TypeError on a clean checkout.
Activity