How to calculate the gradient of images?

In my network, I have a output variable A which is of size hw3, I want to get the gradient of A in the x dimension and y dimension, and calculate their norm as loss function. How should I do it?

Thanks.

3 Likes

I guess you could represent gradient by a convolution with sobel filters. Maybe implemented with Convolution 2d filter with require_grad=false (where you set the weights to sobel filters).

1 Like

@Michael have you been able to implement it? Is it possible to show the code snippet?

Here is the code. It works perfectly.

from PIL import Image
import torch.nn as nn
import torch
import numpy as np
from torchvision import transforms
from torch.autograd import Variable
#img = Image.open(’/home/soumya/Documents/cascaded_code_for_cluster/RGB256FullVal/frankfurt_000000_000294_leftImg8bit.png’).convert(‘LA’)
img = Image.open(’/home/soumya/Downloads/PhotographicImageSynthesis_master/result_256p/final/frankfurt_000000_000294_gtFine_color.png.jpg’).convert(‘LA’)
#img.save(‘greyscale.png’)
T=transforms.Compose([transforms.ToTensor()])
P=transforms.Compose([transforms.ToPILImage()])

ten=torch.unbind(T(img))
x=ten[0].unsqueeze(0).unsqueeze(0)

a=np.array([[1, 0, -1],[2,0,-2],[1,0,-1]])
conv1=nn.Conv2d(1, 1, kernel_size=3, stride=1, padding=1, bias=False)
conv1.weight=nn.Parameter(torch.from_numpy(a).float().unsqueeze(0).unsqueeze(0))

G_x=conv1(Variable(x)).data.view(1,256,512)

b=np.array([[1, 2, 1],[0,0,0],[-1,-2,-1]])
conv2=nn.Conv2d(1, 1, kernel_size=3, stride=1, padding=1, bias=False)
conv2.weight=nn.Parameter(torch.from_numpy(b).float().unsqueeze(0).unsqueeze(0))
G_y=conv2(Variable(x)).data.view(1,256,512)

G=torch.sqrt(torch.pow(G_x,2)+ torch.pow(G_y,2))
X=P(G)
X.save(‘fake_grad.png’)

6 Likes

Thanks ! this worked.
We could simplify it a bit, since we don’t want to compute gradients, but the outputs look great :+1:t4:

#Black and white input image x, 1x1xHxW
a = torch.Tensor([[1, 0, -1],
[2, 0, -2],
[1, 0, -1]])

a = a.view((1,1,3,3))
G_x = F.conv2d(x, a)

b = torch.Tensor([[1, 2, 1],
[0, 0, 0],
[-1, -2, -1]])

b = b.view((1,1,3,3))
G_y = F.conv2d(x, b)

G = torch.sqrt(torch.pow(G_x,2)+ torch.pow(G_y,2))
res = P(G)

2 Likes

I have one of the simplest differentiable solutions. The idea comes from the implementation of tensorflow. The basic principle is:

dx = I(x+1) - I(x), dy = I(y+1) - I(y),

The code is as follows:

def gradient_loss(gen_frames, gt_frames, alpha=1):

    def gradient(x):
        # idea from tf.image.image_gradients(image)
        # https://github.com/tensorflow/tensorflow/blob/r2.1/tensorflow/python/ops/image_ops_impl.py#L3441-L3512
        # x: (b,c,h,w), float32 or float64
        # dx, dy: (b,c,h,w)

        h_x = x.size()[-2]
        w_x = x.size()[-1]
        # gradient step=1
        left = x
        right = F.pad(x, [0, 1, 0, 0])[:, :, :, 1:]
        top = x
        bottom = F.pad(x, [0, 0, 0, 1])[:, :, 1:, :]

        # dx, dy = torch.abs(right - left), torch.abs(bottom - top)
        dx, dy = right - left, bottom - top 
        # dx will always have zeros in the last column, right-left
        # dy will always have zeros in the last row,    bottom-top
        dx[:, :, :, -1] = 0
        dy[:, :, -1, :] = 0

        return dx, dy

    # gradient
    gen_dx, gen_dy = gradient(gen_frames)
    gt_dx, gt_dy = gradient(gt_frames)
    #
    grad_diff_x = torch.abs(gt_dx - gen_dx)
    grad_diff_y = torch.abs(gt_dy - gen_dy)

    # condense into one tensor and avg
    return torch.mean(grad_diff_x ** alpha + grad_diff_y ** alpha)
3 Likes

hi! you can also use kornia.spatial_gradient to compute gradients of an image.
See: https://kornia.readthedocs.io/en/latest/filters.html#kornia.filters.SpatialGradient

3 Likes

You defined h_x and w_x, however you do not use these in the defined function. May I ask what the purpose of h_x and w_x are?