How to calculate the gradient of images?

vision
1

In my network, I have a output variable A which is of size hw3, I want to get the gradient of A in the x dimension and y dimension, and calculate their norm as loss function. How should I do it?

Thanks.

3 Likes
2

I guess you could represent gradient by a convolution with sobel filters. Maybe implemented with Convolution 2d filter with require_grad=false (where you set the weights to sobel filters).

1 Like
3

@Michael have you been able to implement it? Is it possible to show the code snippet?

4

Here is the code. It works perfectly.

from PIL import Image
import torch.nn as nn
import torch
import numpy as np
from torchvision import transforms
from torch.autograd import Variable
#img = Image.open(’/home/soumya/Documents/cascaded_code_for_cluster/RGB256FullVal/frankfurt_000000_000294_leftImg8bit.png’).convert(‘LA’)
img = Image.open(’/home/soumya/Downloads/PhotographicImageSynthesis_master/result_256p/final/frankfurt_000000_000294_gtFine_color.png.jpg’).convert(‘LA’)
#img.save(‘greyscale.png’)
T=transforms.Compose([transforms.ToTensor()])
P=transforms.Compose([transforms.ToPILImage()])

ten=torch.unbind(T(img))
x=ten[0].unsqueeze(0).unsqueeze(0)

a=np.array([[1, 0, -1],[2,0,-2],[1,0,-1]])
conv1=nn.Conv2d(1, 1, kernel_size=3, stride=1, padding=1, bias=False)
conv1.weight=nn.Parameter(torch.from_numpy(a).float().unsqueeze(0).unsqueeze(0))

G_x=conv1(Variable(x)).data.view(1,256,512)

b=np.array([[1, 2, 1],[0,0,0],[-1,-2,-1]])
conv2=nn.Conv2d(1, 1, kernel_size=3, stride=1, padding=1, bias=False)
conv2.weight=nn.Parameter(torch.from_numpy(b).float().unsqueeze(0).unsqueeze(0))
G_y=conv2(Variable(x)).data.view(1,256,512)

G=torch.sqrt(torch.pow(G_x,2)+ torch.pow(G_y,2))
X=P(G)
X.save(‘fake_grad.png’)

6 Likes
5

Thanks ! this worked.
We could simplify it a bit, since we don’t want to compute gradients, but the outputs look great :+1:t4:

#Black and white input image x, 1x1xHxW
a = torch.Tensor([[1, 0, -1],
[2, 0, -2],
[1, 0, -1]])

a = a.view((1,1,3,3))
G_x = F.conv2d(x, a)

b = torch.Tensor([[1, 2, 1],
[0, 0, 0],
[-1, -2, -1]])

b = b.view((1,1,3,3))
G_y = F.conv2d(x, b)

G = torch.sqrt(torch.pow(G_x,2)+ torch.pow(G_y,2))
res = P(G)

2 Likes
6

I have one of the simplest differentiable solutions. The idea comes from the implementation of tensorflow. The basic principle is:

dx = I(x+1) - I(x), dy = I(y+1) - I(y),

The code is as follows:

def gradient_loss(gen_frames, gt_frames, alpha=1):

    def gradient(x):
        # idea from tf.image.image_gradients(image)
        # https://github.com/tensorflow/tensorflow/blob/r2.1/tensorflow/python/ops/image_ops_impl.py#L3441-L3512
        # x: (b,c,h,w), float32 or float64
        # dx, dy: (b,c,h,w)

        h_x = x.size()[-2]
        w_x = x.size()[-1]
        # gradient step=1
        left = x
        right = F.pad(x, [0, 1, 0, 0])[:, :, :, 1:]
        top = x
        bottom = F.pad(x, [0, 0, 0, 1])[:, :, 1:, :]

        # dx, dy = torch.abs(right - left), torch.abs(bottom - top)
        dx, dy = right - left, bottom - top 
        # dx will always have zeros in the last column, right-left
        # dy will always have zeros in the last row,    bottom-top
        dx[:, :, :, -1] = 0
        dy[:, :, -1, :] = 0

        return dx, dy

    # gradient
    gen_dx, gen_dy = gradient(gen_frames)
    gt_dx, gt_dy = gradient(gt_frames)
    #
    grad_diff_x = torch.abs(gt_dx - gen_dx)
    grad_diff_y = torch.abs(gt_dy - gen_dy)

    # condense into one tensor and avg
    return torch.mean(grad_diff_x ** alpha + grad_diff_y ** alpha)
4 Likes
7

hi! you can also use kornia.spatial_gradient to compute gradients of an image.
See: https://kornia.readthedocs.io/en/latest/filters.html#kornia.filters.SpatialGradient

4 Likes
8

You defined h_x and w_x, however you do not use these in the defined function. May I ask what the purpose of h_x and w_x are?