Qualcomm AI Engine Direct - Fold_Quant Enabled (#1773)
Summary: - Enabled fold_quant as default option during convert_pt2e - Embedding OP Removed: 1. Embedding OP has input index data type in int64, which is not supported by QNN. 2. When enabling "fold_quant" option, the parameters from fallback embedding will go through dequantization process to get its floating value. This operation will lead to a significant loss in accuracy. 3. Therefore, we have decided to remove this operation and its quantization config. This can largely benefit models that is sensitive to parameters (e.g., mobilebert) by using mix precision. Pull Request resolved: https://github.com/pytorch/executorch/pull/1773 Reviewed By: cccclai Differential Revision: D53369090 Pulled By: kimishpatel fbshipit-source-id: adedbf8f4277d96ee671f9660951dd878ed38baa
W
winskuo-quic committed
4e7a6fddb7c2135bbc66b3c6f82f32fa6f90b51c
Parent: 63489e8
Committed by Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com>
on 2/9/2024, 6:01:10 AM