Where is "conv1d” implemented?


(Aerin Kim) #1

Hello, I wanted to see how the conv1d module is implemented
https://pytorch.org/docs/stable/_modules/torch/nn/modules/conv.html#Conv1d. So I looked at functional.py but still couldn’t find the looping and cross-correlation computation.

Then I searched Github by keyword ‘conv1d’, checked conv.cpp https://github.com/pytorch/pytorch/blob/eb5d28ecefb9d78d4fff5fac099e70e5eb3fbe2e/torch/csrc/api/src/nn/modules/conv.cpp but still couldn’t locate where the computation is happening.

My question is two-fold.

  1. Where is the source code that "conv1d” is implemented?

  2. In general, if I want to check how the modules are implemented, where is the best place to find? Any pointer to the documentation will be appreciated. Thank you.


(Aerin Kim) #2

I got a clear answer from Stackoverflow: https://stackoverflow.com/q/53927358/3907250

  1. It depends on the backend (GPU, CPU, distributed etc) but in the most interesting case of GPU it’s pulled from cuDNN which is released in binary format and thus you can’t inspect its source code. It’s a similar story for CPU MKLDNN. I am not aware of any place where PyTorch would “handroll” it’s own convolution kernels, but I may be wrong.
  2. It’s difficult without knowing how PyTorch is structured. A lot of code is actually being autogenerated based on various markup files, as explained here. Figuring this out requires a lot of jumping around. For instance, the conv.cpp file you’re linking uses torch::conv1d , which is defined here and uses at::convolution which in turn uses at::_convolution , which dispatches to multiple variants, for instance at::cudnn_convolution . at::cudnn_convolution is, I believe, created here via a markup file and just plugs in directly to cuDNN implementation (though I cannot pinpoint the exact point in code when that happens).

(Anuvabh) #3

I believe the “handroll”-ed convolution is defined here: https://github.com/pytorch/pytorch/blob/master/aten/src/THNN/generic/SpatialConvolutionMM.c

The NN module implementations are here: https://github.com/pytorch/pytorch/tree/master/aten/src
The GPU version is in THCUNN and the CPU version in THNN