How to add a smaller tensor to a bigger tensor out of place, just like padded with zero?
This can be done either by
pad+add or by
split+add+cat, but both of them wastes time copying data around.
Another possible method is
torch.Tensor.put, but I don’t really have or want to generate a index tensor. This would result in wasting time indexing around.
The third method I can think of, is to manually create output tensor, and do
torch.add(out=xx) stuff. This theoretically does not waste any time, but might it affect auto-optimization / autograd? This is also not really elegant.
Any better ideas?