SIGN IN SIGN UP

modify model to use sdpa_with_kv_cache op (#2154)

Summary:
Pull Request resolved: https://github.com/pytorch/executorch/pull/2154

This diff adds new option of `use_sdpa_with_kv_cache` to do model surgery and
make Attention layer use sdpa_with_kv_cache op.

NB: This will likely conflict with some upcoming refacotring.

NB: Model exported with this op is not runnable in aten mode
ghstack-source-id: 216820234
exported-using-ghexport

validated oss ci is clean. have to by pass because ci think its needs internal linter to pass.
bypass-github-export-checks

Reviewed By: mergennachin

Differential Revision: D54075828

fbshipit-source-id: 3f89c77c3315fbf0f0ed49f9a4cc5ddb231cbe6c
K
Kimish Patel committed
c67f0ff57993239c5881d979e267a2ec560f208f
Parent: 92ac890
Committed by Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com> on 2/29/2024, 5:50:29 PM