import torch data = torch.tensor([[1.,2.,3.,4.], [5.,6.,7.,8.]], requires_grad=True) batch = data.shape t_data = data.reshape(batch, 2, 2) tf_data = torch.zeros((batch, 3, 2, 2)) for i in range(batch): tf_data[i] = t_data[i].expand_as(tf_data[i]) loss = torch.sum(tf_data) print('tf_data\n', tf_data) print('data\n', data) tf_data.register_hook(print) data.register_hook(print) loss.backward()
I want to know the gradient flow for some tensor operation like reshape or expand_as, so I wrote above code to test. I got below output.
the grad of data and tf_data is all 3 and 1 no matter how I changed data, can anyone explain the gradient flow for this case in pytorch? How does the pytorch calculate the gradient for this case?