[CI] Remove cache clean action

This action has never worked due to permission issues, and because
of how GitHub handles caches it wouldn't reliably help cache anyway.