Pretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.
docs: update `optimizer_zero_grad` order, and the backward pass. (#21144)
(cherry picked from commit 06bed20190c2e428c00ef73c6aa70ab423b2a47a)
G
GdoongMathew committed
0df60336f8d83f257ccf55f40cbf2adcdd65e91e
Parent: 8f79734
Committed by Luca Antiga <[email protected]>
on 9/5/2025, 1:14:02 PM