Loss.backward() error-loss(my_func(output),label)

1

RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn
I got this RuntimeError

def generate_3d_dipole_kernel(data_shape, voxel_size, b_vec):
    #fov = np.array(data_shape) * np.array(voxel_size)
    fov = torch.tensor(data_shape)*torch.tensor(voxel_size)

    ry, rx, rz = torch.meshgrid(torch.arange(-data_shape[1] // 2, data_shape[1] // 2),
                             torch.arange(-data_shape[0] // 2, data_shape[0] // 2),
                             torch.arange(-data_shape[2] // 2, data_shape[2] // 2))

    rx, ry, rz = rx / fov[0], ry / fov[1], rz / fov[2]

    sq_dist = rx ** 2 + ry ** 2 + rz ** 2
    sq_dist[sq_dist == 0] = 1e-6
    d2 = ((b_vec[0] * rx + b_vec[1] * ry + b_vec[2] * rz) ** 2) / sq_dist
    kernel = (1 / 3 - d2)

    return kernel

def forward_convolution_1(chi_sample):
    scaling = np.sqrt(chi_sample.shape[0]*chi_sample.shape[1]*chi_sample.shape[2])
    chi_fft = torch.fft.fftshift(torch.fft.fftn(torch.fft.fftshift(chi_sample))) / scaling
    
    chi_fft_t_kernel = chi_fft * generate_3d_dipole_kernel(chi_sample.shape, voxel_size=1, b_vec=[0, 0, 1])
   
    tissue_phase = torch.fft.fftshift(torch.fft.ifftn(torch.fft.fftshift(chi_fft_t_kernel)))
    tissue_phase = torch.real(tissue_phase * scaling)

    return tissue_phase

optimizer = torch.optim.Adam(p, lr=0.01)
num_iter=100
for j in range(num_iter):
    optimizer.zero_grad()
    out = net(net_input)
    out1=out.clone().detach().cpu()
    outt = forward_convolution_1(out1[0][0]).cuda()
    total_loss = mse(outt, img_noisy_torch)
    total_loss.backward()
    optimizer.step()

I have to put the output from model into forward_convolution(self-made function).
But it continues making error of no gradient…
How can I solve it?

2

You could try doing the following:

  • Include the steps of forward_convolution_1 in the forward function of net, so that net(net_input) does not require any further processing before you pass it to mse.
  • Use torch.sqrt (or some combination of PyTorch functions) instead of np.sqrt.
3

Thank you for your advice but torch.sqrt input should be tensor…:joy:

4

As far as I understand, PyTorch won’t be able to track gradients if you compute your output using non-PyTorch functions such as numpy.sqrt. I could be wrong, though.

1 Like
5

You are manually detaching the model output in this line of code:

out1=out.clone().detach().cpu()

so the error message is expected, since the detached tensor is not attached to any computation graph.