Pretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.
Added a check to validate that wrapped FSDP models are used while initializing optimizers (#15301)
Co-authored-by: awaelchli <aedu.waelchli@gmail.com>
R
Rohit Gupta committed
0886e6352e13756d800a49e2a000b595742650ca
Parent: 18f7f2d
Committed by GitHub <noreply@github.com>
on 11/8/2022, 2:10:35 AM