Add quantized kv cache to llama (#5664)
Summary: Pull Request resolved: https://github.com/pytorch/executorch/pull/5664 This diff adds - quantized kv cache imlementation and apply corresponding source transforms - add support for quant/dequant per token in quantized kernels ghstack-source-id: 245703480 exported-using-ghexport //oss complaining of internal lint and build failure bypass-github-export-checks exported-using-ghexport Reviewed By: malfet, digantdesai Differential Revision: D62301844 fbshipit-source-id: d89679b5793b73de4e85cb1111414e41ec2e7faf
K
Kimish Patel committed
055bed53ae12295d8446edae8fa01de230d44b88
Parent: 8ddb846
Committed by Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com>
on 10/2/2024, 12:51:40 AM