Pytorch verison: torch-1.0.1.post2-cp36-cp36m-linux_x86_64.whl
To be clear, the Pytorch is running in the docker with nvidia-docker2.
torch.backends.cudnn.benchmark=True is added to avoid slow training.
torch.get_num_threads() . I still face this problem when it returns 1 or 4.
Problem Description:
A general CNN network is training with Jupyter in GPU mode. Then I restart the Jupyter or Shutdown the Jupyter kernel so that I can change the code to try again. However, after restarting or Shutdowning the Jupyter kernel several times (I have faced 1~6 times), the code is freezing the computer when I want to re-run the code. I have experienced this problem for about 2 months.
I have tried to test on Ubuntu18.04 or Ubuntu 16.04 with Cuda 10 or Cuda 9. However, I still face this problem. However, several months ago, another CNN network is training and testing well with pytorch0.41+cuda9.
Testing
Firstly, I guess this problem is related to the docker.
Verifying that is this problem related to docker. Whenever I want to stop the training process, I will first interrupt the kernel. Then, restart the container.
Anyone who meets this problem, I remind you that every time you want to refine code, you need to restart the docker container.