refactor runner to fix bugs (#2752)
Summary: Pull Request resolved: https://github.com/pytorch/executorch/pull/2752 Lots of bugs before with kv cache implementation of runner. Trying this out The bugs were mostly various ways that the prompt sequence was getting trashed in kv cache mode Reviewed By: iseeyuan Differential Revision: D55496044 fbshipit-source-id: c98cbb26ce022981eb7eddbadd57037bad94739a
J
Jacob Szwejbka committed
05b0d3add83f3f0ac78f124e96ad147955094372
Parent: a624345
Committed by Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com>
on 3/29/2024, 6:35:31 AM