Pretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.
Make `all_reduce` consistent for both NCCL and GLOO (#18235)
Co-authored-by: Carlos Mocholí <[email protected]>
A
Adrian Wälchli committed
70e31b64800a89e91ff46b631670cf8944d10ab9
Parent: 27d9125
Committed by GitHub <[email protected]>
on 8/9/2023, 9:39:57 PM