SIGN IN SIGN UP

fuse conv and batch_norm (#3769)

Summary:
Pull Request resolved: https://github.com/pytorch/executorch/pull/3769

When `batchnorm` is applied after `conv` in a model, we can fuse the weight and bias of `batchnorm` into `conv` and thereafter remove the `batchnorm` node. We implement this fusion through graph transforms and apply it in `vulkan_preprocess.py`.

This change can reduce both the latency and memory. We illustrate the performance improvement with Mobilenet_v2.

- The model has 52 conv+batch_norm instances. After fusing, when we export the model as in D57475757, `_native_batch_norm_legit_no_training` doesn't show up anymore.
- The performance has been improved as below. In particular, inference latency has been reduced from 161 ms to 148 ms.

| fuse  | Loading(ms) | vmRss(KB) | vmaBlock(KB) | Inference(ms) | vmRss(KB) | vmaBlock(KB) |
| -------- | ------- | ------- | ------- | ------- |
| Yes  | 380 | 22928 | 65536 | 148 | 24296 | 65536 |
| No  | 473 | 26036 | 65536 | 161 | 27416 | 65536 |

Reviewed By: SS-JIA

Differential Revision: D57895439

fbshipit-source-id: 7ca48fafaca88b5053fa09a0dd489d6766e0eb1e
W
Wei Lu committed
4f2f7e08bb16c0b712e1ee3fdd8edd0e7159e452
Parent: 870c753
Committed by Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com> on 5/29/2024, 11:42:09 PM