SIGN IN SIGN UP

Add quantized kv cache to llama (#5664)

Summary:
Pull Request resolved: https://github.com/pytorch/executorch/pull/5664

This diff adds
- quantized kv cache imlementation and apply corresponding source transforms
- add support for quant/dequant per token in quantized kernels
ghstack-source-id: 245703480
exported-using-ghexport

//oss complaining of internal lint and build failure
bypass-github-export-checks
exported-using-ghexport

Reviewed By: malfet, digantdesai

Differential Revision: D62301844

fbshipit-source-id: d89679b5793b73de4e85cb1111414e41ec2e7faf
K
Kimish Patel committed
055bed53ae12295d8446edae8fa01de230d44b88
Parent: 8ddb846
Committed by Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com> on 10/2/2024, 12:51:40 AM