SIGN IN SIGN UP
hpcaitech / ColossalAI UNCLAIMED

Making large AI models cheaper, faster and more accessible

0 0 38 Python

[pipeline]: fix p2p comm, add metadata cache and support llama interleaved pp (#5134)

* test: add more p2p tests

* fix: remove send_forward_recv_forward as p2p op list need to use the same group

* fix: make send and receive atomic

* feat: update P2PComm fn

* feat: add metadata cache in 1f1b

* feat: add metadata cache in interleaved pp

* feat: modify is_xx_stage fn

* revert: add _broadcast_object_list

* feat: add interleaved pp in llama policy

* feat: set NCCL_BUFFSIZE in HybridParallelPlugin
W
Wenhao Chen committed
4fa689fca1ecf50b8e905cf1c74d4a2c08219daf
Parent: af95267
Committed by GitHub <noreply@github.com> on 12/22/2023, 2:44:00 AM