update llama runner to decode single token (#6768)
Pull Request resolved: https://github.com/pytorch/executorch/pull/6703 Right now, we don't print the generated response in the eager runner until all tokens are generated. This is not good experience as we need to wait until all tokens are generated to see the response. This PR updates it to decode each new token immediately after it is generated. ghstack-source-id: 252924039 Differential Revision: [D65578306](https://our.internmc.facebook.com/intern/diff/D65578306/) Co-authored-by: Lunwen He <lwhecser@gmail.com>
P
pytorchbot committed
671f9c50ca1d581d4d34e38dbe8509c8040d5323
Parent: 6887ae9
Committed by GitHub <noreply@github.com>
on 11/11/2024, 10:09:50 PM