Fix broadcast semantics
Summary: According to https://pytorch.org/docs/stable/notes/broadcasting.html, for two tensors, "the dimension sizes must either be equal, or one of them is 1". When one of them is 1, broadcast happens, and the size is changed to match the other tensor. The doc states that for each dimension size, the resulting dimension size is the max of the sizes of x and y along that dimension. However, when the other tensor has size 0, the size of both tensors are broadcasted into 0, not 1 (max between 0 and 1). This is how numpy behaves. ATen has the same behavior as numpy. ``` >>> np.add(np.zeros([1, 0]), torch.ones([1, 1, 1])) tensor([], size=(1, 1, 0), dtype=torch.float64) >>> torch.zeros([1, 0]) + torch.ones([1, 1, 1]) tensor([], size=(1, 1, 0)) ``` Therefore, `max` is not always true in this case. It's always changing the tensor with size 1 to match the other. Reviewed By: manuelcandales Differential Revision: D48167767 fbshipit-source-id: 9d65e7fad3f9fae1db184041f72a4c77e150ee39
H
Hansong Zhang committed
943e4ec9f6ad7e151e3d7473cd9955ae2a3174b2
Parent: 333dfb4
Committed by Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com>
on 8/8/2023, 11:20:36 PM