In my network, I have a output variable A which is of size h*w*3, I want to get the gradient of A in the x dimension and y dimension, and calculate their norm as loss function. How should I do it?

Thanks.

In my network, I have a output variable A which is of size h*w*3, I want to get the gradient of A in the x dimension and y dimension, and calculate their norm as loss function. How should I do it?

Thanks.

3 Likes

I guess you could represent gradient by a convolution with sobel filters. Maybe implemented with Convolution 2d filter with require_grad=false (where you set the weights to sobel filters).

1 Like

Here is the code. It works perfectly.

from PIL import Image

import torch.nn as nn

import torch

import numpy as np

from torchvision import transforms

from torch.autograd import Variable

#img = Image.open(’/home/soumya/Documents/cascaded_code_for_cluster/RGB256FullVal/frankfurt_000000_000294_leftImg8bit.png’).convert(‘LA’)

img = Image.open(’/home/soumya/Downloads/PhotographicImageSynthesis_master/result_256p/final/frankfurt_000000_000294_gtFine_color.png.jpg’).convert(‘LA’)

#img.save(‘greyscale.png’)

T=transforms.Compose([transforms.ToTensor()])

P=transforms.Compose([transforms.ToPILImage()])

ten=torch.unbind(T(img))

x=ten[0].unsqueeze(0).unsqueeze(0)

a=np.array([[1, 0, -1],[2,0,-2],[1,0,-1]])

conv1=nn.Conv2d(1, 1, kernel_size=3, stride=1, padding=1, bias=False)

conv1.weight=nn.Parameter(torch.from_numpy(a).float().unsqueeze(0).unsqueeze(0))

G_x=conv1(Variable(x)).data.view(1,256,512)

b=np.array([[1, 2, 1],[0,0,0],[-1,-2,-1]])

conv2=nn.Conv2d(1, 1, kernel_size=3, stride=1, padding=1, bias=False)

conv2.weight=nn.Parameter(torch.from_numpy(b).float().unsqueeze(0).unsqueeze(0))

G_y=conv2(Variable(x)).data.view(1,256,512)

G=torch.sqrt(torch.pow(G_x,2)+ torch.pow(G_y,2))

X=P(G)

X.save(‘fake_grad.png’)

6 Likes

Thanks ! this worked.

We could simplify it a bit, since we don’t want to compute gradients, but the outputs look great

#Black and white input image x, 1x1xHxW

a = torch.Tensor([[1, 0, -1],

[2, 0, -2],

[1, 0, -1]])

a = a.view((1,1,3,3))

G_x = F.conv2d(x, a)

b = torch.Tensor([[1, 2, 1],

[0, 0, 0],

[-1, -2, -1]])

b = b.view((1,1,3,3))

G_y = F.conv2d(x, b)

G = torch.sqrt(torch.pow(G_x,2)+ torch.pow(G_y,2))

res = P(G)

2 Likes

I have one of the simplest differentiable solutions. The idea comes from the implementation of tensorflow. The basic principle is:

dx = I(x+1) - I(x), dy = I(y+1) - I(y),

The code is as follows:

```
def gradient_loss(gen_frames, gt_frames, alpha=1):
def gradient(x):
# idea from tf.image.image_gradients(image)
# https://github.com/tensorflow/tensorflow/blob/r2.1/tensorflow/python/ops/image_ops_impl.py#L3441-L3512
# x: (b,c,h,w), float32 or float64
# dx, dy: (b,c,h,w)
h_x = x.size()[-2]
w_x = x.size()[-1]
# gradient step=1
left = x
right = F.pad(x, [0, 1, 0, 0])[:, :, :, 1:]
top = x
bottom = F.pad(x, [0, 0, 0, 1])[:, :, 1:, :]
# dx, dy = torch.abs(right - left), torch.abs(bottom - top)
dx, dy = right - left, bottom - top
# dx will always have zeros in the last column, right-left
# dy will always have zeros in the last row, bottom-top
dx[:, :, :, -1] = 0
dy[:, :, -1, :] = 0
return dx, dy
# gradient
gen_dx, gen_dy = gradient(gen_frames)
gt_dx, gt_dy = gradient(gt_frames)
#
grad_diff_x = torch.abs(gt_dx - gen_dx)
grad_diff_y = torch.abs(gt_dy - gen_dy)
# condense into one tensor and avg
return torch.mean(grad_diff_x ** alpha + grad_diff_y ** alpha)
```

3 Likes

hi! you can also use `kornia.spatial_gradient`

to compute gradients of an image.

See: https://kornia.readthedocs.io/en/latest/filters.html#kornia.filters.SpatialGradient

3 Likes

You defined h_x and w_x, however you do not use these in the defined function. May I ask what the purpose of h_x and w_x are?