Control over broadcasting semantics


In torch, when I try to do this, I get

import torch
y_rel = torch.randn(16,2,8)  # (Batch, n, seq_len)
y_start = torch.randn(16,2)
y_rel + y_start

This throws RuntimeError: The size of tensor a (8) must match the size of tensor b (2) at non-singleton dimension 2

In Julia, I can do something like this. Is there anything like this in Pytorch?

y_rel = randn(16,2,8)  # (Batch, n, seq_len)
y_start = randn(16,2)
y_rel .+ y_start

tl;dr : Is there a .+ type of operator in Pytorch?

If you add dim2 to y_start the tensor will be automatically broadcasted:

y_rel + y_start.unsqueeze(2)
1 Like