SIGN IN SIGN UP

Fix quantization for input to reference model (#2317)

Summary:
Add the zerpoint instead of subtracting. This worked since the tests so far used the ones as inputs which quantize to a zp of -128 which gives the same np.int8 result in both cases since the int8 wraps. Also needs to round and clip the scaled values to the int8 range.

Signed-off-by: Per Åstrand <per.astrand@arm.com>

Pull Request resolved: https://github.com/pytorch/executorch/pull/2317

Reviewed By: mergennachin

Differential Revision: D55201623

Pulled By: digantdesai

fbshipit-source-id: 81a07186f3ebc4adb75af28cee109ab6ed4d0de8
P
Per Åstrand committed
d06ccd27a4c5433eb7c7e13909b1dac63f9a158d
Parent: 9c7bb45
Committed by Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com> on 3/22/2024, 3:35:26 AM