Kernel died / Segmentation fault

1

After building my forward propagation function I try to test it by running this code:

A=BuildModel().cuda()
X=torch.autograd.Variable(torch.randn(1,1,96,96,16))
X.cuda()

B=A(X)

Both in this way and by removing both .cuda(), after a few minutes wait I get a “Kernel died, restarting” error in spyder and a segmentation fault error running it from terminal (in a conda environment). By following these instructions all I could get from the core dumps was:

Cannot access memory at address 0x7f2df11a60b8
Cannot access memory at address 0x7f53fa53d0b8

From GPU and CPU runs. I’m using pytorch-gpu version 0.1.12 in a conda environment, Python 3.6.8, GeForce GTX 1080

2

Hi,

You don’t need variables anymore, so you can drop the torch.autograd.Variable and just use Tensors everywhere.
Also the .cuda() call on Tensor is not inplace, you need to do X = X.cuda().

1 Like
3

Not sure if that’s related to Spyder, I had some of these when I was using Jupyter Notebook some time ago. It happened very rarely, and even resetting GPU memory didn’t help. What helped was simply quitting and restarting the Notebook.

From GPU and CPU runs. I’m using pytorch-gpu version 0.1.12 in a conda environment,

PyTorch 0.1.12 is a very, very old and if possible, I recommend updating to PyTorch 1.0, which could potentially help solving the problem.

1 Like
4

Thanks, It really looks like I just had to update pytorch and install it properly instead of using the pytorch-gpu package.