Thank you. I have changed the stripe function and it returns a proper stripe:
def stripe(a):
i, j = a.size()
assert i >= j
k, l = a.stride()
return torch.as_strided(a, (i - j, j), (k, k+1))
a = torch.randn((182, 91)).cuda()
output = stripe(a)
# output.size()
# torch.Size([91, 91])
a = torch.randn((150, 182, 91))
output = list(map(stripe, torch.unbind(a, 0)))
output = torch.stack(output, 0)
# output.size()
# torch.Size([150, 91, 91])
Now I am facing some obscure PyTorch 0.4 error after using that stripe function when the model computes backwards loss:
Traceback (most recent call last):
File "runner.py", line 305, in <module>
main()
File "runner.py", line 249, in main
loss = model.update(batch)
File "J:\PyCharmProjects\tac-self-attention\model\rnn.py", line 67, in update
loss.backward()
File "J:\Anaconda_Python3_6\envs\cuda2\lib\site-packages\torch\tensor.py", line 93, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "J:\Anaconda_Python3_6\envs\cuda2\lib\site-packages\torch\autograd\__init__.py", line 89, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: Tensor: invalid storage offset at c:\programdata\miniconda3\conda-bld\pytorch_1524546371102\work\aten\src\thc\generic/THCTensor.c:759
(cuda2)
This doesn’t occur when using the version with .diag(). Any ideas what causes it?