Extension of convolution layer using CFFI

1

I want to extend or make my new convolution layer, using C FFI. Even if I saw this page https://github.com/pytorch/extension-ffi , it is hard to get started.

It seems that I can start from some sources from pytorch/torch/lib/…

Could I just know what sources I have to use or a big picture of getting started?

2

The extension-ffi example showcases implementing a small function to add two tensors in https://github.com/pytorch/extension-ffi/blob/master/package/my_package/src/my_lib.c

What additional information would get you started?

You can look at how existing ops are implemented in C here:
https://github.com/pytorch/pytorch/tree/master/torch/lib/THNN/generic

3

You could also see how the extension FFI has been used to implement CTC loss wrappers in this repo.

4

Thanks for your information.

In /THNN/docs I found a description of

module = nn.SpatialConvolution(nInputPlane, nOutputPlane, kW, kH, [dW], [dH], [padW], [padH])

But it seems that there is no SpatialConvolution in THNN/generic.

I am just curious:

  1. Where can I see the convolution code used in PyTorch?

  2. PyTorch doesn’t use any codes in /pytorch/torch/lib, right? For example, there is no dilated convolution in PyTorch docs. So I wonder it.

5
  1. THNN/generic/SpatialConvolutionMM and THNN/generic/SpatialDilatedConvolution
  2. It uses codes there. Dilated convolution is provided as a keyword argument in convXd http://pytorch.org/docs/nn.html#conv2d
1 Like
6

How can I call the im2col function within my own convolution layer?
In SpatialDilatedConvolution it is called in the following way:

THNN_(col2im)(
  THTensor_(data)(gradColumns),
  nInputPlane, inputHeight, inputWidth, outputHeight, outputWidth,
  kH, kW, padH, padW, dH, dW,
  dilationH, dilationW,
  THTensor_(data)(gradInput_n)
);

But I’m only considering Float type, so can how can I call this function? Something like THNN_Float_im2col?
Are there any header files I need to import?

7

Hi,

from greping through the pytorch source, I would seem that there is no header with the col2im definition, so you would need to add that somewhere and it might not be considered part of the official api…
In the library, the function is THNN_Floatcol2im
(says objdump -t torch/lib/build/THNN/libTHNN.so.1 | grep col2im).

Best regards

Thomas

8

Thanks!
So what you mean is that I need to declare it in my code and link to libThNN.so.1 right?
Also can you tell the full definition from the lib file (void THNN_Floatcol2im?, what are the parameters included )?

9

You can use the one from SpatialFullConvolution.c after substituting real–>float (if it took THTensor_ you would need to expand that, too, but it seems to take the float* buffers.

10

Sorry if I didn’t make it clear.
The function I’m going to use is:

THNN_Floatim2col(THFloatTensor_data(input_h_w), 1, inputHeight, inputWidth, kH, kW, padH, padW, dH, dW, dilationH, dilationW, THFloatTensor_data(columns));

But when I compile the file, the warning comes out:

warning: implicit declaration of function 'THNN_Float_im2col' is invalid in C99 [-Wimplicit-function-declaration]
            THNN_Float_im2col(THFloatTensor_data(input_h_w), 1, inputHeight, inputWidth, kH, kW, padH, padW, dH, dW, dilationH, dilationW, THFloatTensor_data(columns))

I wonder this might because I need to declare it first. So the question is, what is the full declaration for THNN_Float_im2col?

I’m new to c, so if I understand this wrong or there is an other way to solve this problem, please let me know.

Many thanks.

11

I think the declaration is more

static void THNN_Floatim2col(const float* data_im, const int channels,
      const int height, const int width, const int kernel_h, const int kernel_w,
      const int pad_h, const int pad_w,
      const int stride_h, const int stride_w,
      const int dilation_h, const int dilation_w,
      float* data_col);

or so (by literally copying the declaration in the file and substituting).

From the message, you seem to have an extra underscore somewhere.

Best regards

Thomas

12

Where did you copy this?

13

Hi!

Sorry, I was outdated by a week… :confused:
The commit seems to remove it. I don’t know whether it will be in the .so on master, but I would have doubts.
But you might be able to get it with pytorch 0.2.

Best regards

Thomas

1 Like
14

I see, thanks.
I tried to add the declaration on top of my C code and got the following warning:

THNN_Floatim2col' has internal linkage but is not defined

If I removed the declaration I got:

implicit declaration of function 'THNN_Floatim2col' is invalid in C99

I will continue to check if it is possible to get the header files that defines THNN_Floatim2col.

Thanks for you help.
Best
Z

15

@smth @apaszke Guys could you please give some suggestions?

16

hi
I recently implemented a convolution with the PyTorch extension here and the code is based on here.
It is worth mentioning that the convolution achieved by PyTorch is very inefficient:

3_11
3_111302×716 182 KB

I hope this helps you !

1 Like
17

thanks for sharing :slight_smile:, but pytorch’s convolutions is way faster than the extension. Do you think it would be possible to speed it up?