Disable `f16` on Aarch64 without `neon`

LLVM has crashes at some `half` operations when built with assertions
enabled if fp-armv8 is not available [1]. Things seem to usually work,
but we are reaching LLVM undefined behavior so this needs to be
disabled.

[1]: https://github.com/llvm/llvm-project/issues/129394