SIGN IN SIGN UP

Specialize for non-mixed-dtype in elementwise_util (#9388)

Mixed dtype should be uncommon. Here is how we can specialize for the
common case. Prepares us to tackle #9241 .

Test Plan: automated tests on this PR verify we didn't break the
now-deprecated runtime_out_dtypes mode; tests on the next PR will verify
that everything works after migration. Also included migration for
exactly one operator, op_mul, to verify that the new code compiles.

To check performance, I edited examples/models/toy_model/model.py so
that MulModule used inputs of size 3000, 2000 instead of 3, 2. I
exported it with `python3 -m examples.portable.scripts.export
--model_name mul` and saved the resulting `mul.pte`. Then I built in
release mode with optimized kernels on, but with mul.out removed from
kernels/optimized/optimized.yaml, so that we would use the
optimized_portable_kernels build of kernels/portable/op_mul.cpp.
Finally, I ran 3 trials on my M1 Macbook Pro using
`cmake-out/executor_runner --model_path mul3kby2k.pte --num_executions
1000 --cpu_threads 2`. Resulting times for 1000 iterations in ms:

Previous diff: 8295, 8187, 8139
This diff: 2953, 2806, 2861

(For comparison, the actual optimized mul kernel took around 1000 ms to
run 1000 iterations, and #9432 later in the stack arrived at similar
numbers.)
S
Scott Wolchok committed
208a341e0b04e835c3111d45dce8076c2291bfed
Parent: 59870c5
Committed by GitHub <noreply@github.com> on 4/23/2025, 1:05:28 AM