Does pytorch pdist have size limit?

1

I ran into a weird error where I cannot use torch.nn.functional.pdist after a certain tensor size. In my case it was 9999 or maybe I am missing something?

import math
import torch #version 1.0
import torch.nn.functional as F

sizes = [100, 1000, 3000, 5000, 7500, 8500, 9999, 10000]
for size in sizes:
    print("[{}] Start".format(size))
    a = torch.randn(size, 20).float().cuda()
    print("Before")
    # The operation
    b = F.pdist(a, p=2)
    print("After")
    a = torch.randn(size, 20).float().cuda()
    print("---End---")

I can use the result of pdist but I cant allocate new variables after that. And it give me this error
CUDA error: an illegal memory access was encountered.

2

this seems like a bug, I opened an issue here: https://github.com/pytorch/pytorch/issues/15511

Would you mind running

wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py

and then paste the results of your environment in the issue linked above?

3

Here is the output. I’ve install pytorch using conda as stated from the official website.

Collecting environment information...
PyTorch version: 1.0.0
Is debug build: No
CUDA used to build PyTorch: 9.0.176

OS: Ubuntu 18.04.1 LTS
GCC version: (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0
CMake version: version 3.10.2

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: Could not collect
GPU models and configuration: GPU 0: GeForce GTX 1080 Ti
Nvidia driver version: 396.37
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.7.2.1
/usr/lib/x86_64-linux-gnu/libcudnn_static_v7.a

Versions of relevant libraries:
[pip] Could not collect
[conda] blas                      1.0                         mkl  
[conda] mkl                       2018.0.3                      1  
[conda] mkl_fft                   1.0.6                    py36_0    conda-forge
[conda] mkl_random                1.0.2                    py36_0    conda-forge
[conda] pytorch                   1.0.0           py3.6_cuda9.0.176_cudnn7.4.1_1    pytorch
[conda] torch                     0.4.1                     <pip>
[conda] torchvision               0.2.1                     <pip>
[conda] torchvision               0.2.1                      py_2    pytorch

Also this is the cuda and cudnn version 9.0.176 and 7401