Executorch][xnnpack] Hack to speedup dqlinear lowering (#1893)
Summary:
Pull Request resolved: https://github.com/pytorch/executorch/pull/1893
XNNPACK preprocess is very very slow due to running many transform passed. In
the specific instance convert_bilinear_upsamples was taking extreme amount of
time.
This should really be solved with _to_edge_and_transform API, but in the
meantime I really want to increase my, maybe others, iteration speed. This hack
is a way to get there. Potentially in future we also allow partitioner to
highlight what was partitioned so we dont run unrelated transform passes.
Result:
Run time buck2 run mode/opt mode/inplace
executorch/examples/models/llama2:export_llama -- -c
executorch/examples/models/llama2/params/demo_rand_params.pth -p
executorch/examples/models/llama2/params/demo_config.json --pt2_quantize
"xnnpack_dynamic" -m '{"get_bos_id": 3, "get_eos_id": 3, "get_n_bos": 1,
"get_n_eos": 2}' -2 -kv
Before: ~7m
After: little more than a minute
for profiling runtime try
time buck2 run mode/opt mode/inplace
executorch/examples/models/llama2:export_llama -- -c
executorch/examples/models/llama2/params/demo_rand_params.pth -p
executorch/examples/models/llama2/params/demo_config.json --pt2_quantize
"xnnpack_dynamic" -m '{"get_bos_id": 3, "get_eos_id": 3, "get_n_bos": 1,
"get_n_eos": 2}' -2 -kv -prof "llama2.html"
There is still a signifcant amount of time spent in partitioning
ghstack-source-id: 215452812
exported-using-ghexport
Reviewed By: digantdesai, mcr229
Differential Revision: D53584078
fbshipit-source-id: a80ecb95a449e2d80c3f73453ba3ad6ad9eecb8f K
Kimish Patel committed
b601b49b21de7caac782512713474672ba1c5a3f
Parent: 800de21
Committed by Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com>
on 2/16/2024, 5:44:16 PM