Qualcomm AI Engine Direct - Enable AR-N model for prompt processing in hybrid mode (#8210)
* Qualcomm AI Engine Direct - Enable AR-N mode to process prompt in hybrid mode Summary: - Add `max_seq_len` to refer to maximum number of tokens that the model can process & consider at once to generate predictions/responses. - Add `prefill_ar_n` to determine the number of tokens to consume and the number of logits to produce for prompt processor in hybrid mode. - Remove prefill mode * fixed CI * Add the figure to readme and fixed unused variable * fixed linting
S
shewu-quic committed
9484c011c441d30af78908fc676f674ed2dcf74f
Parent: f965746
Committed by GitHub <noreply@github.com>
on 2/25/2025, 8:38:11 AM