Arm backend: Explicitly convert quantized value to int64
Previously, dequantizing a value with dequantize_value() in backends/arm/tosa_quant_utils.py could result in integer overflow when using numpy 2.1.3. The offending part of the formula is `qx - qargs.zp`. If the subtraction results in a value outside of the range of the dtype of `qx` the following warning is printed: "RuntimeWarning: overflow encountered in scalar subtract" With numpy 1.21.3 the dtype is implicitly convert to a dtype that can store the correct value. However, in numpy 2.1.3 there's no such conversion, leading the function to return an incorrect value. Here's a concrete example: ``` import numpy as np a = np.int8(127) b = -128 print(a-b) ``` Numpy 1.21.3: a - b = 255 Numpy 2.1.3: a - b = -1 To remedy this, explicitly convert qx to int64. Change-Id: Ie0e9e7745a424103ce650e2d58fe1a1a4cbd30e1
S
Sebastian Larsson committed
80f1c1b8b0201a297caa1a968982699ed4aa61e2
Parent: 3f7eb3b
Committed by Fredrik Knutsson <fredrik.knutsson.hunnebo@gmail.com>
on 12/13/2024, 1:03:59 PM