Add seq_len to llama runner for early stopping (#2051)
Summary: Pull Request resolved: https://github.com/pytorch/executorch/pull/2051 By default, the llama runner will continue generating until max_seq_len. This is a property embedded in the model metadata. We want a way to limit the number of tokens generated. Reviewed By: Jack-Khuu, larryliu0820 Differential Revision: D53873431 fbshipit-source-id: bf2df65f74d60b381ef6f938c242f29cb4e58261
V
Varun Puri committed
33ba5638f4ae0c01df5022b448d5cd0c84ca555f
Parent: c4a1e95
Committed by Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com>
on 2/23/2024, 5:17:40 AM