SIGN IN SIGN UP

Add Warp Size metric for alternative SM workload distribution (#4305)

Summary:
Pull Request resolved: https://github.com/pytorch/executorch/pull/4305

Here's where things start to get a little bit tricky. As you can see in the graph below from an Adreno, looking only at the latency increases has the problem that there is actually two jumps: every 64 threads and every 128 threads.

 {F1762497531}

This may be due to the fact that, although the hardware has a limit of 64 threads per warp, each SM could potentially simulate concurrency of 128 threads. Also, as ArchProbe mentions, basing ourselves solely on latency can have issues due to how the SMs distribute workloads when there are fewer threads than the maximum:

In Case 1, like ARM Mali, the SM can assign dummy workloads to fill empty threads and maintain a uniform workload.
In Case 2, like in Adreno, the driver might decide to pack multiple works together and dispatch them at once.

Therefore we need two different methods for warp size calculation, one that evidences the physical limitations, and one that evidences the actual distribution by the driver.

This diff introduces a second alternative way to measure warp size that makes use of the scheduler. The kernel utilizes an atomic counter that is shared by all the threads of the warp, and each run will increase the counter by one and save its value in its corresponding spot. Since the scheduler will place the runs in ascending order, the output buffer should have its values filled in ascending order, i.e:
  [0, 1, 2, 3, ... n]

If we increase the warp size by one on each run, there should be a point where two warps are running at the same time. Since each warp has its own scheduler and shared variables, and each scheduler is not aware of each other, the buffer output will no longer be in ascending order at one point, i.e:

  [... 126, 127, 0, 1, 2, ...]

The point where the order resets should be the size of our warp.

On this chart we can see the warps racing each other with their own unique values for the counter, and this repeats every 128 threads.
{F1762495824}

More information can be found [here](https://www.microsoft.com/en-us/research/uploads/prod/2022/02/mobigpu_mobicom22_camera.pdf) on page 5.

Reviewed By: jorgep31415

Differential Revision: D59926726

fbshipit-source-id: 8fbde95137ff112fbef70dd6c5be3216da86adde
E
Esteban Padilla Cerdio committed
f0ebfa20f3328927b70c4f5697f41fe79f2c1d83
Parent: 3269e61
Committed by Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com> on 7/23/2024, 3:43:30 PM