SIGN IN SIGN UP

Qualcomm AI Engine Direct - Add QNN support for to_edge_transform_and_lower (#9643)

Summary:

- Support `to_edge_transform_and_lower`
- Replace capture_program with new API
`to_edge_transform_and_lower_to_qnn`
- Replace capture_program with to_edge_transform_and_lower_to_qnn for
unit_test
- Replace capture_program with to_edge_transform_and_lower_to_qnn for
examples
- Replace capture_program with to_edge_transform_and_lower_to_qnn for
llama
- Add QnnPassManager to manage all passes in different stage
  - Deprecated _transform in export_llama_lib with qnn_pass_manager
- Add transform_for_export_pipeline for LiftConstantScalarOperands to
avoid creating temporary tensors in the operation builder. However, this
pass will create a get_attr node, which should be converted into a
lifted tensor constant by the lift_constant_tensor_pass. If placed in
the to_edge_transform_passes, it will be executed after the
lift_constant_tensor_pass, causing the operation builder to fail to
correctly retrieve the parameter by the get_parameter for get_attr node.
- Refactor the passes
  - Fix the output dtype doesn't match in runtime after build quant io
  - Combine constant_i64_to_i32 and tensor_i64_to_i32 into i64_to_i32
  - Replace convert_to_linear pass with fixed_linear_keep_dim pass
- Since QNN has no keep dims for linear op, we will need to add squeeze
and unsqueeze around linear node
- Add TagQuantIO pass to tag io nodes to avoid inserting q/dq in
qnn_preprocess
  - Add prelu, leaky_relu, linear, rms_norm into decompose_table
    - Remove recompose_prelu.py
- Remove unused variable in insert_requantize.py, and
replace_index_put_input.py

- Support aten.split_with_sizes_copy.default
- Support leaky_relu with inplace=True
S
shewu-quic committed
2f408dd79d9656c8bfb90b1e8fd990ed326ea36f
Parent: 753a88e
Committed by GitHub <noreply@github.com> on 4/2/2025, 11:56:01 PM