API change for `tensor.data.set_(...)` in torch-nightly

1

Hi!
I test the package against torch nightly builds and got a strange error today

Seems like torch nightly has just changed the behaviour for tensor.data.set_(...) and now raises an error

https://travis-ci.com/ferrine/geoopt/jobs/167379249#L1406
How do I now write the same function without api change from my side?

    def proj_(self):
>       self.data.set_(self.manifold.projx(self.data))
E       RuntimeError: set_storage is not allowed on Tensor created from .data or .detach()
2

I asked around, they said

tensor.set_ should be okay (instead of tensor.data.set_)
if tensor.set_ is not ok, wrapping in a with torch.no_grad(): block will make it ok

2 Likes
3

The error message is to alert you to a backwards incompatible change; our apologies that the error message isn’t clear, we will correct that.

Essentially the issue is that previously self.data would give you the same underlying Tensor as self (this is legacy behavior from when Variable’s “wrapped” Tensors), so metadata changes (e.g. size, storage, etc.) made to .data would also change self. Now self.data gives you a (non-differentiable) alias with its own copy of the metadata, so metadata changes made to self.data will not be reflected in self.

The fix depends on what you were trying to achieve. If you were trying to change self without autograd knowing about it, you should put the the change in a torch.no_grad() block and stop referencing .data.

Here’s a simplified example.

Before:

>>> x=torch.randn((2,3,4), requires_grad=True)
>>> y=x.sum(dim=2)
>>> z = torch.randn(7)
>>> y.data.set_(z)
>>> y
tensor([[-2.8514,  0.5746, -1.3324],
        [ 1.1557,  0.7676,  1.8141]], grad_fn=<SumBackward2>)

Now (error):

>>> x=torch.randn((2,3,4), requires_grad=True)
>>> y=x.sum(dim=2)
>>> z = torch.randn(7)
>>> y.data.set_(z)
RuntimeError: set_storage is not allowed on Tensor created from .data or .detach()

Corrected:

>>> x=torch.randn((2,3,4), requires_grad=True)
>>> y=x.sum(dim=2)
>>> z = torch.randn(7)
>>> with torch.no_grad():
            y.set_(z)
>>> y
tensor([[-2.8514,  0.5746, -1.3324],
        [ 1.1557,  0.7676,  1.8141]], grad_fn=<SumBackward2>)
4 Likes
4

Yeah, meta data issue sounds like a reasonable argument to change the behaviour. What benefits do tou have with that change from the developer perspective?

5

The benefits are two-fold, although perhaps won’t make a huge difference at the python level:

  1. The C++ API still has the Variable/Tensor distinction which adds a lot of complexity for C++ users / library writers. This change is part of the larger project to make the C++ API match the Python API.
  2. The change allows us to remove a number of virtual function calls for common accessors (mainly the metadata), which should translate to lower overhead.
1 Like
6

Hi @gchanan. I get similar error with resize_ function.
But the solution with.torch.no_grad() seems no working.

Here is the example:
if we have a with torch.Size([64, 3, 7, 7])
and b with torch.Size([58, 3, 7, 7])

a.resize_(b.size())

RuntimeError: set_sizes_contiguous is not allowed on Tensor created from .data or .detach()

1 Like