SIGN IN SIGN UP

Qualcomm AI Engine Direct - Observer Fix and remove unused passes (#6225)

Summary:
- `ConvertToLinear()` is redundant in `qnn_preprocess.py` since this pass is already called in `executorch/backends/qualcomm/utils/utils.py`

- Some models are experiencing a significant drop in accuracy, with a few models having 0% accuracy. Adding new conditions to perform requantization and change ptq_per_channel_quant_config's IO from MinMaxObserver to MovingAverageMinMaxObserver to resolve the issue.

 1. Why adding new conditions to do requantization? We noticed this change in PyTorch PR (https://github.com/pytorch/pytorch/commit/b8eef500a63e18c8594f70b9794daa4333538629#diff-976c3b0c6f85048d3db01a0c394ce8eb16e2f7541f0983d0f4ef549baa4be822L152). Before this PR, quantization spec only checks whether 2 qspecs were same by comparing `dtype` and `is_dynamic`. After this change, it checks for more attributes such as `scale`, `zero_point`, etc. This causes some nodes having an extra pair of QDQ nodes. As shown in the image below, there are 2 pairs of QDQ nodes after the PyTorch PR, and these 2 pairs of QDQ nodes have different scale and offset. For QNN lowering process, node will only save the quant info right after the node output. For example, `cat` op below will use `quantize_per_tensor_default_18`'s scale and offset as the node's quant attribute, and all other quant and dequant nodes will be ignored.
This causes an accuracy drop, but by inserting a requantize node, we can see an improvement in accuracy for most models. Taking inceptionv3 as an example, the average top1 accuracy 0%->~75%. I have checked a couple other models and see accuracy either stays the same or have improvements.

I have also provided the option for users to skip this requant optimization if they preferred not to use it.

**Before:**
![image](https://github.com/user-attachments/assets/e6048b24-347c-4a5b-8406-c11dc14d33ae)

___
**After**
![image](https://github.com/user-attachments/assets/200cca57-f4f7-48bc-83fb-fc1595935569)

2. Why change ptq_per_channel_quant_config's IO from MinMaxObserver to MovingAverageMinMaxObserver?
After the above change, it seems like there is an inference speed drop due to requantization. By switching to MovingAverageMinMaxObserver, I observed an improvement in inference speed for some models such as inceptionv3.

Pull Request resolved: https://github.com/pytorch/executorch/pull/6225

Reviewed By: kirklandsign

Differential Revision: D64413835

Pulled By: cccclai

fbshipit-source-id: a8be66b034c69ff403f9f2985f2b584695f3798b
W
winskuo-quic committed
dc4be7c74bb9112552d5dc05e7b1430436ecb446
Parent: 423f65d
Committed by Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com> on 10/16/2024, 6:25:13 PM