PyTorch2.0 dev20230105 crashes kernel in jupyternotebook when starting training

vision
1

I removed the last version of PyTorch2.0 from December 26th that had due to the security fix.

pip3 uninstall -y torch torchvision torchaudio torchtriton

I then reinstalled PyTorch 2.0
pip3 install numpy --pre torch --force-reinstall --index-url https://download.pytorch.org/whl/nightly/cu117

I have CUDA 11.7, 4 A6000 GPUs, Driver 525.60.13.
The training doesn’t start and the kernel dies immediately after filling the GPU memory.

If I revert back to the PyTorch version from December 26th, everything works fine.

2

The vulnerable binaries were deleted so I assume you are using your cache?
In any case, do not reinstall these binaries as you might also reinstall the triton malware binary.

I’ve just installed the latest binaries on an A6000 and it works properly:

python -c "import torch; print(torch.__version__); print(torch.randn(1).cuda())"
2.0.0.dev20230105+cu117
tensor([0.6857], device='cuda:0')