Relax dtype requirements for int4 and float8 quants in autoquant (#1571)
* Relax dtype requirements for int4 quants in autoquant Summary: Some of the int4 quant only works with bfloat16/float16, previously we require the model to be in correct dtype to apply these in autoquant, this PR relaxes the constraints by converting weight and activation to compatible dtypes Test Plan: python test/integration/test_integration.py -k test_autoquant_int4wo Reviewers: Subscribers: Tasks: Tags: * remove prints * add float8 * run pre-commit * run pre-commit * manual format * enable bias=True test * remove print
J
Jerry Zhang committed
cf453360dd3e09394657172f8e8d8da23cfbf043
Parent: f520c91
Committed by GitHub <noreply@github.com>
on 1/17/2025, 1:11:45 AM