I am going to create a custom layer that can be pre-processed for each layer

1

I am trying to create a layer that is executed every batch in a layer using the torch.
Previously, I tried to write it with TensorFlow and Keras, but NumPy was not possible, so I used PyTorch. When I used NumPy in pytroch, it was run with cpu and it was too slow.
So, in the end, I wrote NumPy by replacing it with the function of the torch.
I am currently training, but it is too slow and the accuracy is 10%. (take 5hour per a epoch)
The sequence of the code is as follows:

class my_custom_layer(nn.Module):
    def __init__(self, D_in, D_out, kernel_size):
        super(my_custom_layer, self).__init__()
        self.BConv_layer = nn.Conv2d(D_in, D_out, kernel_size, padding=(1))
    def forward(self, x):      
        with torch.no_grad(): 
            y = my_function(x)
            #preprocess batch using histogram data
            #make histogram of img -> Partially scaling according to the characteristics of the image -> return scaling factor for result of conv layer

        results = F.conv(x, kernel)
        results = torch.multiply(results, y)
        return results

Sometimes it runs on the CPU, but most of the time it runs on the GPU.
I will replace this layer with VGG’s conv layer.
Is it correct to write a custom layer like this?

2

Your custom layer isn’t using the initialized self.BConv_layer and is using undefined arguments, such as kernel and y, instead, so this won’t work.

Besides that, I’m unsure if you need the no_grad block. If my_function isn’t using any parameters, you could probably remove it.
Also, if you can replace all numpy operations with their PyTorch equivalent, you should be able to use the GPU for all operations. If that’s not possible, note that each data transfer to the CPU would be blocking and might yield a slowdown.

1 Like
3

I have a question about F.conv.
In general, nn.conv is declared in intial and used in forward. On the other hand, should F.conv be declared in intial before written? if it’s true, how to init F.conv?
and in my_function, there are preprocessing input data using torch’s function. but in my opinion, for my_function, I used no_grad because I thought that the parameters they used were unnecessary for calculating gradients.

4

F.convXX is a functional form of convolution where you will manually have to put filters and bias tensors and manage their operation during testing and training time, which is useful for custom implementation of functionality and it does not require initialisation in the __init__ while if you are just using it as the generic convolution I suggest you use the one present in nn.ConvXX if you will look at its source code in the back it also calls f.convXX but manages the filters and bias tensors for you and their operations during test and train time and it can make use of features of nn.Module like self.training flag for example.

1 Like
5

The reason I used f.conv is to convolution each channel of input with kernel and then multiply it by a scaling factor.
Is this also possible using nn.conv?

6

Yes! You can, in forward function you pass the features through a convolution layer of (nn.CovXX) and then multiply it with a scaling factor an example will be like:

class MyCustomLayer(nn.Module):
    def __init__(self, D_in, D_out, kernel_size):
        super(my_custom_layer, self).__init__()
        self.BConv_layer = nn.Conv2d(D_in, D_out, kernel_size, padding=(1))
    def forward(self, x):

        results = y * self.BConv_layer(x) # where y is your scaling factor
        return results
1 Like