[core] Gate ChildOf unparent cleanup on diff flags to avoid spurious depth recompute