Auto merge of #150056 - Kivooeo:trying-to-address-perf, r=davidtwco

Perf regression fix

The only thing changed from the previous PR is that I removed `output_is_inhabited` from hot path, and hide it behind condition, so now it will compute it less frequent

follow up on https://github.com/rust-lang/rust/pull/149664