Pretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.
(7/n) Support 2D Parallelism - TP Fabric Docs (#19884)
Co-authored-by: Sebastian Raschka <[email protected]> Co-authored-by: Carlos Mocholí <[email protected]>
A
awaelchli committed
987c2c4093ea4dbebc0fd41e503fd1743f054933
Parent: 7e87ce0
Committed by GitHub <[email protected]>
on 5/22/2024, 10:20:40 AM