Autocorrelation using existing methods

Hi all,

What would be the most efficient way (using existing methods) for convolving a tensor with itself up to a predefined offset? I would like to be able to convolve a 3D tensor with itself, in all 3 dimensions.

For example, let’s say I have a tensor of size [1, 1024, 64, 64] and convolve it with itself with offsets of ±4 for dimensions 2 and 3, and an offset of ±1 in the first dimension. This should lead an output tensor of size [1, 2*(1)+1, 2*(4)+1, 2*(4)+1] which is [1, 3, 9, 9].

AFAIU, using Conv2D or Conv3D is problematic because convolving the tensor with itself would require treating the tensor as filter weights meaning that training would alter them. My layer has no learned weights.


NVIDIA created a correlation module for its FlowNet2 implementation.
Have a look at this feature request for some explanation.