SIGN IN SIGN UP

Qualcomm AI Engine Direct - Suport batch prefill mode for llama3.2 (#6983)

* Qualcomm AI Engine Direct - Suport bert mode for llama3.2

- Enable bert mode
- Change input sequence of static_llama
- Tag bert output as uint8
- Unify both 1b and 3b in 1 runner
- Add hybrid IO memory for llama3_2 runner
- Align timer with llama

* Rebase and minor fix

- Fix rebase conflict
- Change input dtype of calibration function

* Change bert to batch prefill

* Fix compile error

* Fix lint

- Fix transformers version
- Refine pass quantization tagging function
- Rebase

* Add one line in the end of CmakeList

* Remove trailing line of CmakeList

* Move noqa to correct line number

---------

Co-authored-by: Joey Tsai <chunit@qti.qualcomm.com>
C
Chun-I Tsai committed
d89d3e71bc469cabdd2f7d4c2d3885855c509f57
Parent: 22a75be
Committed by GitHub <noreply@github.com> on 12/3/2024, 4:31:32 PM