Qualcomm AI Engine Direct - Suport batch prefill mode for llama3.2 (#6983)
* Qualcomm AI Engine Direct - Suport bert mode for llama3.2 - Enable bert mode - Change input sequence of static_llama - Tag bert output as uint8 - Unify both 1b and 3b in 1 runner - Add hybrid IO memory for llama3_2 runner - Align timer with llama * Rebase and minor fix - Fix rebase conflict - Change input dtype of calibration function * Change bert to batch prefill * Fix compile error * Fix lint - Fix transformers version - Refine pass quantization tagging function - Rebase * Add one line in the end of CmakeList * Remove trailing line of CmakeList * Move noqa to correct line number --------- Co-authored-by: Joey Tsai <chunit@qti.qualcomm.com>
C
Chun-I Tsai committed
d89d3e71bc469cabdd2f7d4c2d3885855c509f57
Parent: 22a75be
Committed by GitHub <noreply@github.com>
on 12/3/2024, 4:31:32 PM