Skip quantizing output linear layer (#1644)
Summary: Pull Request resolved: https://github.com/pytorch/executorch/pull/1644 Skip quantizing output linear layer Required memory for activation in bytes: [0, x] | Configuration. | memory size | exec log | checkpoint name | |--|--|--|--| |FP32 activation & weights |x = 5.5 GB | P1042687346 | new_llama_checkpoint.pte [ ] | |FP32 activation, qint8 weights: |x = 3 GB | P1042717078 | new_llama_checkpoint_q.pte [-Q] | |FP16 activation, qint8 weights: |x = 1.9 GB | P1042645801 | new_llama_checkpoint_qh.pte [-Q -H] | |FP16 activation & weights |x = 1.5 GB | P1042661553 | new_llama_checkpoint_h.pte [-H] | |(old) FP32 activation, FP16 scales, qint8 weights |x = 3 GB | P1042678025 | new_llama_checkpoint_qhf.pte [-Q ] | Reviewed By: hl475 Differential Revision: D52827084 fbshipit-source-id: 8c0b6aebdcbd1f48ca1c7ce704c9f77248e944a3
M
Michael Gschwind committed
1468f57005baa45f450173befd13b49ca88935c8
Parent: a4b54f7
Committed by Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com>
on 1/19/2024, 9:15:47 PM