SIGN IN SIGN UP

Add seq_len to llama runner for early stopping (#2051)

Summary:
Pull Request resolved: https://github.com/pytorch/executorch/pull/2051

By default, the llama runner will continue generating until max_seq_len. This is a property embedded in the model metadata. We want a way to limit the number of tokens generated.

Reviewed By: Jack-Khuu, larryliu0820

Differential Revision: D53873431

fbshipit-source-id: bf2df65f74d60b381ef6f938c242f29cb4e58261
V
Varun Puri committed
33ba5638f4ae0c01df5022b448d5cd0c84ca555f
Parent: c4a1e95
Committed by Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com> on 2/23/2024, 5:17:40 AM