[rocm-libraries] ROCm/rocm-libraries#8762 (commit fe88750)
[CK] Fix flaky test `test_batched_gemm_b_scale_wmma` Loosen up the tolerance and add better logging in case of failure TLDR: `profile_batched_gemm_b_scale_impl.hpp` (lines 390-392) uses `rtol = atol = 1e-1`, while the sibling non-batched B-scale profiler uses `2e-2` in `profile_gemm_b_scale_impl.hpp` (lines 361-364). `KBatch > 1` adds an F16 atomic accumulation step, so the batched path is stricter on the noisier arithmetic path. Loosening up the tolerances should fix the flaky test. I've also added better loging in case of failures. ## Motivation `test_batched_gemm_b_scale_wmma` can fail on gfx1201 when `KBatch > 1` because that path uses split-K F16 atomic accumulation. The existing batched B-scale tolerance was stricter than the non-batched B-scale profiler, even though the batched split-K path has additional rounding noise from native F16 atomics. This PR aligns the batched tolerance with the comparable non-batched path and improves failure diagnostics so future mismatches identify the exact shape, KBatch value, instance, strides, and operator. ## Technical Details Updated [projects/composablekernel/profiler/include/profiler/profile_batched_gemm_b_scale_impl.hpp](projects/composablekernel/profiler/include/profiler/profile_batched_gemm_b_scale_impl.hpp) to: - Change non-FP8 verification tolerance from `1e-2` to `2e-2`. - Match the tolerance already used by the non-batched B-scale profiler. - Build a detailed failure message with: - instance index - `kbatch` - `M`, `N`, `K`, and `BatchSize` - `StrideA`, `StrideB`, and `StrideC` - selected operator name - Move operator name construction earlier so it is available for diagnostics. ## Test Plan Configured the CK gfx1201 build with device instances enabled and XDL disabled to isolate WMMA coverage. Built and ran the focused batched GEMM B-scale WMMA test target: ```bash ninja -C projects/composablekernel/build-gfx950-gfx1201 test_batched_gemm_b_scale_wmma ``` Ran the focused `MidLargeM` case across WMMA instances and then ran the full WMMA-only gtest target. ## Test Result The WMMA-only gfx1201 validation passed. - `MidLargeM` passed across 21 WMMA instances. - Full `test_batched_gemm_b_scale_wmma` passed all 3 gtests. - No incorrect-result lines were reported. ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests. Signed-off-by: Chris Tsiaousis <chris.tsiaousis@streamhpc.com>
C
chris-tsiaousis-hpc committed
e503e6277a47060d84f9a976665b6f2a97c84e3d
Parent: 137f2a9
Committed by assistant-librarian[bot] <assistant-librarian[bot]@users.noreply.github.com>
on 6/24/2026, 4:41:50 PM