SIGN IN SIGN UP

Migrate quantized conv2d tests (#1387)

Summary:
Pull Request resolved: https://github.com/pytorch/executorch/pull/1387

Migrate convolution tests from test_xnnpack_quantized.py to ops/*.py and update to use new test harness.

Add calibration run to quantize step. This is needed to calibrate the quantization observers.

Update test logic to use quantization scale for absolute tolerance when comparing reference and model outputs for quantized tests. This fixes an issue where quantized tests intermittently fail due to a difference of +/-1 quantized step, which is within expected deviation.

Reviewed By: mcr229

Differential Revision: D51874474

fbshipit-source-id: f990f324c988070b237ab483656d6b6e46a0efe5
G
Gregory Comer committed
c8a1f3bf8fa180120758ee1a6c79532dd9d7574a
Parent: f0499a3
Committed by Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com> on 12/18/2023, 11:12:00 PM