Hi, I’m working on my project involving the great high-order differentiation operator. It works fine under CPU stride, however, when I try to work on GPU strides, I encountered the following error
RuntimeError: CUDNN_STATUS_NOT_SUPPORTED. This error may appear if you passed in a non-contiguous input.
I’m working with python3.6.1, torch 0.2.0_4, CUDA Version 8.0.44 and cudnn 6021, which I think are all updated. I track back the error, and it shows the error shows up whenever I use conv2d layer.
I tried torch.backends.cudnn.enabled=False to get rid of the error but then no acceleration happens seems.
Some other solution like torch.backends.cudnn.benchmark=True is not working in my case.
As for now, I’m not able to find a way to resolve this, would anyone help me to figure it out? thank you in advance,
I created the following demo code to show my issue, please let me know if you need anything more from me 
import torch
import torch.nn as nn
from torch.autograd import Variable, grad
import torch.utils.data as Data
import torchvision
train_data = torchvision.datasets.MNIST(
root='./mnist/',
train=True,
transform=torchvision.transforms.ToTensor(),
download=False,
)
train_loader = Data.DataLoader(
dataset=train_data, batch_size=50, shuffle=True, num_workers=2)
class CNN(nn.Module):
def __init__(self):
super(CNN, self).__init__()
self.conv1 = nn.Sequential(nn.Conv2d(1, 16, 5, 1, 2))
self.out = nn.Linear(16 * 28 * 28, 10)
def forward(self, x):
x = self.conv1(x)
x = x.view(x.size(0), -1)
output = self.out(x)
return output, x
cnn = CNN()
cnn.cuda()
loss_func = nn.CrossEntropyLoss()
for step, (data, label) in enumerate(train_loader):
input = Variable(data).cuda()
target = Variable(label).cuda()
output = cnn(input)[0]
loss = loss_func(output, target)
params = cnn.parameters()
g = grad(loss, params, create_graph=True)
g_sum = 0
for g_para in g:
g_sum += g_para.sum()
params = cnn.parameters()
hv = grad(g_sum, params, create_graph=True)
break
The error message is as following in my setting:
Traceback (most recent call last):
File "gpu_demo.py", line 67, in <module>
hv = grad(g_sum, params, create_graph=True)
File "/root/miniconda3/lib/python3.6/site-packages/torch/autograd/__init__.py", line 153, in grad
inputs, only_inputs)
RuntimeError: CUDNN_STATUS_NOT_SUPPORTED. This error may appear if you passed in a non-contiguous input.