Implement custom layer: local binary patern

vision
1

(I changed the code so one can make it run on their own machine easily…)

Hi together,

I want to implement Local Binary Pattern on pytorch tensors. At the moment my code looks like this:

import numpy as np
import torch
import torch.nn as nn
import torchvision
from PIL import Image
import torchvision.transforms as transforms
from torchvision.transforms import ToTensor

class LBP(nn.Module):

def __init__(self):
    super(LBP, self).__init__()
    self.multiply = torch.tensor([[1,2,4],[128, 0, 8], [64, 32, 16]]).to("cuda")
    self.padding = transforms.Pad(1)

def get_value_of_crop(self, crop):
    crop_center = crop[1,1]
    comparison = crop <= crop_center
    result_matrix =  comparison * self.multiply
    result = torch.sum(result_matrix)
    return result

def forward(self, grey_image):
    lbp_tensor = torch.zeros_like(grey_image)
    padded_image = self.padding(grey_image)
    for i in np.arange(0, padded_image.shape[1] - 2):
        for j in np.arange(0, padded_image.shape[1] - 2):
            print(f"{i}, {j}")
            lbp_tensor[0, i, j] = lbp_machine.get_value_of_crop(padded_image[0, i:i + 3, j:j + 3])

    return lbp_tensor

lbp_machine = LBP()

data = torchvision.datasets.CIFAR10(“.”, download= True)
testimage = data.getitem(0)[0].convert(“L”)
tensortransform = ToTensor()
testimage = tensortransform(testimage).to(“cuda”)
lbp_machine(testimage)

It is not really performant as there are two for-loops. Any ideas about how to avoid these two for-loops and make the layer more performant?

Thanks a lot for any feedback!

Bernhard

P.s.: I know I could use packages like scikit-image, but they dont work on pytorch-tensors and by transforming my tensors to numpy-arrays I lose information like “grad-fn” that I will need later…