Hi, I have some issues about autograd. I try to implement a network whose nodes in the hidden layer takes diverse noisy inputs. However, it hindered when computing the gradients w.r.t all the parameters.

num_hid = 1

nr_classes = 10

w1 = torch.rand(1024,num_hid,requires_grad = True)-0.5

w1_bias = torch.rand(num_hid,requires_grad = True)-0.5

w2 = torch.rand(num_hid,nr_classes,requires_grad = True)-0.5

w2_bias = torch.rand(nr_classes,requires_grad = True)-0.5

noise_gt_data1 = GenerateNoisyData(n)

linear_output_first_layer = torch.matmul(noise_gt_data1.reshape(-1,1024),w1)+w1_bias

sigmoid = nn.Sigmoid()

activation_output_first_layer = sigmoid(linear_output_first_layer)

out = Variable(torch.matmul(activation_output_first_layer,w2),requires_grad=True)+w2_bias

net = Net(num_hid)

net.body[0].weight.data = w1

net.body[0].bias.data = w1_bias

net.fc[0].weight.data = w2

net.fc[0].bias.data = w2_bias

y = criterion(out, gt_onehot_label)

gradients=torch.autograd.grad(y,net.parameters(),create_graph=True,retain_graph=True,allow_unused=True)

print(gradients)

It shows gradients are all None. I tried both backward() and autograd.grad(). Neither works.