Pretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.
Warn when FSDP auto-wrap policy splits tied weights (#21613)
* Add tied weight detection warning for FSDP auto-wrap policy Detect shared parameters that would be placed in separate FSDP units by the auto-wrap policy and emit a warning before wrapping. This turns a cryptic RuntimeError (size mismatch) into an actionable message. Applies to both Fabric and PyTorch Lightning FSDP strategies. Closes #21403 * Add tests for FSDP tied weight detection warning Cover four scenarios: tied weights across units (warns), tied weights in same unit (no warn), no shared params (no warn), no policy (no warn). --------- Co-authored-by: Deependu <deependujha21@gmail.com>
C
c-pozzi committed
4ea9b0105dadd7c9e42e5fd341d9c3246c0f9989
Parent: 7e3c0cd
Committed by GitHub <noreply@github.com>
on 4/13/2026, 8:01:04 PM