Trying to run the first model in https://pytorch.org/tutorials/beginner/nn_tutorial.html using GPU

1

Hello,

I am trying to run the first model in https://pytorch.org/tutorials/beginner/nn_tutorial.html on GPU, but could not.

The notebook is at https://github.com/soumitrak/nn_tutorial_cuda/blob/master/nn_tutorial_cuda.ipynb

The code works if ‘dev = torch.device(“cpu”)’, but for ‘dev = torch.device(“cuda”)’, weights.grad is None.

I browsed around and found that for non-leaf Variables grad is not stored, but here bias, and weights are leaf level Tensors, so don’t know why it fails.

Thanks in advance.
-Soumitra.

2

Same code works on “cpu”, but on “cuda”, it has following error:

TypeError Traceback (most recent call last)
in
16 loss.backward()
17 with torch.no_grad():
—> 18 weights -= weights.grad * lr
19 bias -= bias.grad * lr
20 weights.grad.zero_()

TypeError: unsupported operand type(s) for *: ‘NoneType’ and ‘float’

3

Check if both lr and weights.grad is present and is not None in the code.

4

Thanks Ashok, I check lr is not None, but weights.grad is None. The same code works well if the device is “cpu”.