Add warmup for Llama (#5756)
Summary: Load the model. Run the everything twice. Reset stats in between two runs. Also decrease logging level for warm up. Notes: Tested on Android and Mac. With Llama2 and Llama3 - with temperature=0 produces same output. This warm up option is disabled by default. This is inspired from llama.cpp options [[1](https://github.com/ggerganov/llama.cpp/blob/ea9c32be71b91b42ecc538bd902e93cbb5fb36cb/common/common.cpp#L897-L929), [2](https://github.com/ggerganov/llama.cpp/blob/ea9c32be71b91b42ecc538bd902e93cbb5fb36cb/examples/llama-bench/llama-bench.cpp#L1595-L1602)]. Sample [runs](https://www.internalfb.com/phabricator/paste/view/P1613261035) Pull Request resolved: https://github.com/pytorch/executorch/pull/5756 Reviewed By: mcr229, metascroy Differential Revision: D63642723 Pulled By: digantdesai fbshipit-source-id: 39ff257eda182fff423f90582a9f32387cfdb253
D
Digant Desai committed
660ef7761087668e78d9ec97d500b1c6cd486983
Parent: 26dc9fd
Committed by Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com>
on 10/1/2024, 2:02:18 AM