Hi all, I have test following code using cuda tensor, but failed.
from torch.autograd import gradgradcheck, gradcheck
inputs = [autograd.Variable(torch.randn(5, 12).double(), requires_grad=True),
autograd.Variable(torch.randn(10, 12).double(), requires_grad=True),
autograd.Variable(torch.randn(10).double(), requires_grad=True)]
for i in xrange(len(inputs)):
inputs[i] = inputs[i].cuda()
test = gradcheck(lambda i, w, b : F.linear(i, w, b), inputs)
print(test)
It return False. When I remove the cuda() operation, it return True.
I’d probably do the .cuda() before the Variable wrapping:
from torch.nn import functional as F
from torch import autograd
from torch.autograd import gradcheck
import torch
inputs = [autograd.Variable(torch.randn(5, 12).cuda().double(), requires_grad=True),
autograd.Variable(torch.randn(10, 12).cuda().double(), requires_grad=True),
autograd.Variable(torch.randn(10).cuda().double(), requires_grad=True)]
for i in range(len(inputs)):
inputs[i] = inputs[i]
test = gradcheck(lambda i, w, b : F.linear(i, w, b), inputs)
print(test)
prints True for me. (Don’t know if it should also work for single, but that is another thing.)