Qualcomm AI Engine Direct - Add QNN support for to_edge_transform_and_lower (#9643)
Summary:
- Support `to_edge_transform_and_lower`
- Replace capture_program with new API
`to_edge_transform_and_lower_to_qnn`
- Replace capture_program with to_edge_transform_and_lower_to_qnn for
unit_test
- Replace capture_program with to_edge_transform_and_lower_to_qnn for
examples
- Replace capture_program with to_edge_transform_and_lower_to_qnn for
llama
- Add QnnPassManager to manage all passes in different stage
- Deprecated _transform in export_llama_lib with qnn_pass_manager
- Add transform_for_export_pipeline for LiftConstantScalarOperands to
avoid creating temporary tensors in the operation builder. However, this
pass will create a get_attr node, which should be converted into a
lifted tensor constant by the lift_constant_tensor_pass. If placed in
the to_edge_transform_passes, it will be executed after the
lift_constant_tensor_pass, causing the operation builder to fail to
correctly retrieve the parameter by the get_parameter for get_attr node.
- Refactor the passes
- Fix the output dtype doesn't match in runtime after build quant io
- Combine constant_i64_to_i32 and tensor_i64_to_i32 into i64_to_i32
- Replace convert_to_linear pass with fixed_linear_keep_dim pass
- Since QNN has no keep dims for linear op, we will need to add squeeze
and unsqueeze around linear node
- Add TagQuantIO pass to tag io nodes to avoid inserting q/dq in
qnn_preprocess
- Add prelu, leaky_relu, linear, rms_norm into decompose_table
- Remove recompose_prelu.py
- Remove unused variable in insert_requantize.py, and
replace_index_put_input.py
- Support aten.split_with_sizes_copy.default
- Support leaky_relu with inplace=True S
shewu-quic committed
2f408dd79d9656c8bfb90b1e8fd990ed326ea36f
Parent: 753a88e
Committed by GitHub <noreply@github.com>
on 4/2/2025, 11:56:01 PM