audio: separate fallbacks for upmix and downmix cases

We always want to prefer upmix to downmix, as long as it makes sense.
Even if the upmix is not "perfect" (not just adding channels), we want
to prefer the upmix.

Cleanup for commit d3c7fd9d.