fix runner max seq len (#2688)
Summary: Pull Request resolved: https://github.com/pytorch/executorch/pull/2688 Max seq len arg when not passed uses the max seq len from the model. THis means, num tokens generated should be equal to kv cache size. However, generate loop tries to generate one more token because pos, 0 based index, is taken for num tokens Reviewed By: mergennachin, digantdesai Differential Revision: D55369776 fbshipit-source-id: 7beb38177a23449649e96184b0b0a0bb507c199f
K
Kimish Patel committed
cde514ca0898ba93d95158b3dbe9ecd2a2b9f32c
Parent: 61b1b83
Committed by Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com>
on 3/26/2024, 6:04:19 PM