SIGN IN SIGN UP

Qualcomm AI Engine Direct - Enable AR-N model for prompt processing in hybrid mode (#8210)

* Qualcomm AI Engine Direct - Enable AR-N mode to process prompt in hybrid
mode

Summary:
- Add `max_seq_len` to refer to maximum number of tokens that the model can process & consider at once to generate predictions/responses.
- Add `prefill_ar_n` to determine the number of tokens to consume and the number of logits to produce for prompt processor in hybrid mode.
- Remove prefill mode

* fixed CI

* Add the figure to readme and fixed unused variable

* fixed linting
S
shewu-quic committed
9484c011c441d30af78908fc676f674ed2dcf74f
Parent: f965746
Committed by GitHub <noreply@github.com> on 2/25/2025, 8:38:11 AM