SIGN IN SIGN UP
hpcaitech / ColossalAI UNCLAIMED

Making large AI models cheaper, faster and more accessible

0 0 96 Python

[Inference/Kernel] Add Paged Decoding kernel, sequence split within the same thread block (#5531)

* feat flash decoding for paged attention

* refactor flashdecodingattention

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
S
Steve Luo committed
be396ad6cc102fa610731291bf28e531a5641c7a
Parent: 56b222e
Committed by GitHub <noreply@github.com> on 4/18/2024, 8:45:07 AM