Add custom `VkInt4WeightOnlyQuantizer` for vulkan (#6234)
Summary: Pull Request resolved: https://github.com/pytorch/executorch/pull/6234 ## Context This diff adds the `VkInt4WeightOnlyQuantizer` class which enables 4-bit quantization of linear layers via source transformation. This quantizer class is copied from `torchao.quantization.GPTQ.WeightOnlyInt4Linear` with some minor changes as annotated in the implementation. Note that the pt2e quantization flow does not yet support groupwise quantization, so source transformation is the only way to perform groupwise quantization at the moment. ghstack-source-id: 248349848 exported-using-ghexport Reviewed By: jorgep31415 Differential Revision: D64406457 fbshipit-source-id: 7d638d58414f2eb048caba60ebb11b7783bb3b7a
S
Stephen Jia committed
58ee33d3f43b19a9239e8cad32e126850eff2bc4
Parent: 841277a
Committed by Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com>
on 10/16/2024, 7:48:15 PM