Is it possible to back propagate in a TorchScript?

jit
1

PyTorch 1.5.0

Hello, I am using PyTorch to minimise a cost function. No training involved. I was wondering whether using TorchScript rather than pure Python could have led to a speed up. However, I am not sure whether it is possible to back propagate in TorchScript.

Let us consider the following toy class:

class Example(nn.Module):
  def forward(self, x):
    y = torch.tensor([0], dtype=x.dtype)
    y.requires_grad = True
    return y

If I call example_scripted = torch.jit.script(Example()), I get the following error:

RuntimeError:
Tried to set an attribute: grad on a non-class: Tensor:

The issue seems the requires_grad. I am wondering whether it is possible to use TorchScript to back propagate. Is it?

Please see also https://github.com/pytorch/pytorch/issues/40561

Thank you.

2

Yeah, backprop works, but some python code won’t compile. In your (contrived) case, tensor(…).requires_grad_(True) would compile, but you don’t usually need to be explicit about requires_grad, as it propagates with operations, and there is also .detach().

3

Yes you can train see this book:
https://pytorch.org/deep-learning-with-pytorch

Picture1
Picture11552×309 109 KB

1 Like