How to add a smaller tensor to a bigger tensor out of place, just like padded with zero?
This can be done either by pad+add
or by split+add+cat
, but both of them wastes time copying data around.
Another possible method is torch.Tensor.put
, but I don’t really have or want to generate a index tensor. This would result in wasting time indexing around.
The third method I can think of, is to manually create output tensor, and do torch.add(out=xx)
stuff. This theoretically does not waste any time, but might it affect auto-optimization / autograd? This is also not really elegant.
Any better ideas?