Convolution backward path for CUDA

C++
1

Hi,
I want to write a convolution backward function , and it would call Pytorch’s cuda path for default. Basically it has the following definition:

std::tuple<at::Tensor,at::Tensor,at::Tensor>  
custom_convolution_backward(const at::Tensor & grad_output, 
                    const at::Tensor & input, 
                    const at::Tensor & weight, 
                    at::IntArrayRef stride, 
                    at::IntArrayRef padding, 
                    at::IntArrayRef dilation,
                    bool transposed, 
                    at::IntArrayRef output_padding, 
                    int64_t groups, 
                    std::array<bool,3> output_mask);

I explore the source code in PyTorch and find that CUDA uses slow_conv2d_backward_out_cuda.

My question is :

  1. Is slow_conv2d_backward_out_cuda the right function I need to use?
  2. The function has args of finput and fgrad_input finput, what are these two? I can’t find docs nor comments on them.
  3. Why can’t I find the convolution backward function in Convolution.cpp ? Isn’t it should write the backward path as in cudnn ?
2

I have figured this out and I may answer the question myself. Hope it will offer some help for coming visitors.

  1. The function that needs to be used is thnn_conv2d_backward, the method to find this is to read the derivatives.yaml. This file will record the forward and backward function.
    {Note} If you come to this and find that there is no THNN function, you may refer to this issue, because the pytorch teams are migrating to newer ATen function. The ATen is modern implementation for it.

  2. The finput and fgradinput are just two temporary buffer. Which is cols and ones if you refer to Convolution implementation papers. Sorry I forget the paper’s name so you may search by your self.
    The thing is, you don’t have to worry about these two, just pass empty tensor is enough, they serve as only a buffer. You may find the usage like

    at::Tensor finput = at::empty({0},input.options()); 
at::Tensor fgrad_input = at::empty({0}, input.options());
auto kernel_size = weight.sizes().slice(2);
auto &&result = at::thnn_conv2d_backward(grad_output, input, weight,kernel_size , stride, padding, 
finput, fgrad_input, output_mask);

  3. This could be find on derivatives.yaml as stated before.