SIGN IN SIGN UP

Pretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.

0 0 89 Python

Warn when FSDP auto-wrap policy splits tied weights (#21613)

* Add tied weight detection warning for FSDP auto-wrap policy

Detect shared parameters that would be placed in separate FSDP units
by the auto-wrap policy and emit a warning before wrapping. This turns
a cryptic RuntimeError (size mismatch) into an actionable message.

Applies to both Fabric and PyTorch Lightning FSDP strategies.

Closes #21403

* Add tests for FSDP tied weight detection warning

Cover four scenarios: tied weights across units (warns), tied weights
in same unit (no warn), no shared params (no warn), no policy (no warn).

---------

Co-authored-by: Deependu <deependujha21@gmail.com>
C
c-pozzi committed
4ea9b0105dadd7c9e42e5fd341d9c3246c0f9989
Parent: 7e3c0cd
Committed by GitHub <noreply@github.com> on 4/13/2026, 8:01:04 PM