Hi,

I am trying to compute the gradient of output from model with respect to the input, which is a symmetric matrix. I expect the output of gradient calculation to be symmetric. Can someone help explain why the gradient output is not symmetric? This is the code I have been trying.

SM = Variable(matrix,requires_grad=True)

out = model(SM)

output = F.softmax(out)

model.zero_grad()

out_class = output.max(1)[1].data[0]

one_hot_output = torch.zeros(output.size())

one_hot_output[0, output_class] = 1

output.backward(gradient=one_hot_output)

grad_out_in = SM.grad.data[0]

The output of above code is for example looks like this:

tensor([[[-0.0465, -0.0377, -0.0602, …, 0.0117, 0.0323, -0.0426],

[ 0.0018, -0.0711, 0.0594, …, 0.0302, -0.1398, 0.0213],

[-0.0123, 0.0490, 0.0875, …, -0.0314, 0.0194, -0.0434],

…,

[ 0.0363, 0.0121, 0.0098, …, 0.0108, 0.0057, 0.0425],

[ 0.0596, -0.0743, 0.0541, …, 0.0290, -0.1835, 0.1229],

[-0.0395, 0.0283, -0.0428, …, -0.0294, 0.1277, -0.0793]]]

Using Pytorch 1.4 with cuda 9.2