<UnsafeViewBackward> in grad_fn

1

Hi,
While tracking down my gradient functions, I came accross this label . As my model fails to learn, I was wondering whether that could be related to that.
And more generally speaking, I havent found much information related to that, what does this “Unsafe” account for ?
Thanks

1 Like
2

I also realized I have a variable with grad_fn=UnsafeViewBackward after using torchviz. I think that this is related to the function _unsafe_view, which is defined in https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/TensorShape.cpp. There is a comment just before the definition that might be helpful. In my case this happened to the variable x_hat, which is created using the following line

x_hat = F.linear(Z.view(batch_size,1,-1),self.D)

I found that _unsafe_view () is called inside matmul(), which is in turn called by F.linear, being F just torch.nn.functional. See here: https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/LinearAlgebra.cpp

1 Like
3

From the comment in the code:

// _unsafe_view() differs from view() in that the returned tensor isn't treated
// as a view for the purposes of automatic differentiation. (It's not listed in
// VIEW_FUNCTIONS in gen_autograd.py).  It's only safe to use if the `self` tensor
// is temporary. For example, the viewed tensor here (a + b) is discarded immediately
// after viewing:
//
//  res = at::_unsafe_view(a + b, size);
//
// This is a hack because in-place operations on tensors treated like views
// can be much more expensive than the same operations on non-view tensors.

Used e.g. in matmul.

1 Like
4

Hi @ptrblck,

I just created a very simple snippet like below:

p = torch.ones((3,4), requires_grad = True)

p1 = p.permute(1,0).flatten()
p2 = p.transpose(1,0).flatten()
p3 = p.T.reshape(-1)

p4 = torch.empty(12)
for i in range(4):
    for j in range(3):
        p4[i*3 + j] = p[j,i]

print(p1, p2, p3, p4)

output
----------------------
tensor([1., 1., ... 1.], grad_fn=<UnsafeViewBackward>)
tensor([1., 1., ... 1.], grad_fn=<UnsafeViewBackward>)
tensor([1., 1., ... 1.], grad_fn=<UnsafeViewBackward>)
tensor([1., 1., ... 1.], grad_fn=<CopySlices>)

seems if we change the shape of original tensor then apply flatten, the grad_fn will become
so if I am going to use p1~p3 as convolution weight and apply F.conv2d to a tensor x, which p should I use?

F.conv2d(x, p1), or F.conv2d(x, p2), or F.conv2d(x, p3) or F.conv2d(x, p4)???

5

You as a user shouldn’t see any difference, so you could use any pX tensor for your operation.
Anyway, let us know, if you see any unexpected behavior or errors.

Of course the shape doesn’t fit the expected weight shape for F.conv2d, but that is another issue.

6

ok thank you for the explaination.

7

Hi,
I receive an error while using this function for reshaping my tensor

def reshape_fortran(x, shape):
    if len(x.shape) > 0:
        x = x.permute(*reversed(range(len(x.shape))))
    return x.reshape(*reversed(shape)).permute(*reversed(range(len(shape))))

and it says:

RuntimeError: _unsafe_view does not support automatic differentiation for outputs with complex dtype.

it is weird cause, sometime I receive it and sometime it works fine!

what is this unsafe_view?

8

Could you post the type and shape of x as well as the input argument shape, so that we could debug this issue?
This error points towards the complex types so are you using this tensor type or might the error message be misleading?

9

Thanks for your response.
Yes, I am working with complex data and I use this function to implement F order reshape in tensors. (There is no built-in function for Fortran reshape of tensors)
The input is a (4,4,2) complex64 valued matrix and the shape value is (8,4).
I was able to fix the problem by using this function instead, but I have no idea why it was not working and why it is working now:

def convert_output84_torch(input):
  shape84 = (8,4)
  T1 = torch.transpose(input,0,2).permute(0,1,2).contiguous()
  T2 = T1.view(shape84[::-1])
  output = torch.transpose(T2,0,1)
  return output

I replaced reshape with view, and it works fine (for now). But as my code was working fine before, I am not sure if it is a long-term solution as I am not aware of the main source.

10

Thanks for the information.
I can reproduce this error in PyTorch 1.7.0 and get a different error message in 1.8.0.dev20201022:

def reshape_fortran(x, shape):
    if len(x.shape) > 0:
        x = x.permute(*reversed(range(len(x.shape))))
    return x.reshape(*reversed(shape)).permute(*reversed(range(len(shape))))

x = torch.randn(4, 4, 2, dtype=torch.complex64, requires_grad=True)
shape = (8, 4)
out = reshape_fortran(x, shape)
out.mean().backward()
> RuntimeError: mean does not support automatic differentiation for outputs with complex dtype.

Note that complex support is not fully implemented yet, so I would recommend to verify the results of your second approach using some reference values.

1 Like