TypeError: addmm_, after transfering the net to GPU (works fine on CPU)

1

Hi all,

I received the following error after transfering the net to the GPU. Things worked fine when the net is in CPU.
Since the (int, int) arguments are from pytorch internally. I wonder what I need to do to fix this. Should I simply go the the line and change

output.addmm_(0, 1, input, weight.t())

to

output.addmm_(0., 1., input, weight.t())

Thanks a lot for your help.

/home/nelson/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/nn/functions/linear.py in forward(self, input, weight, bias)
8 self.save_for_backward(input, weight, bias)
9 output = input.new(input.size(0), weight.size(0))
—> 10 output.addmm(0, 1, input, weight.t())
11 if bias is not None:
12 # cuBLAS doesn’t support 0 strides in sger, so we can’t use expand

TypeError: addmm_ received an invalid combination of arguments - got (int, int, torch.FloatTensor, torch.cuda.FloatTensor), but expected one of:

  • (torch.FloatTensor mat1, torch.FloatTensor mat2)
  • (torch.SparseFloatTensor mat1, torch.FloatTensor mat2)
  • (float beta, torch.FloatTensor mat1, torch.FloatTensor mat2)
  • (float alpha, torch.FloatTensor mat1, torch.FloatTensor mat2)
  • (float beta, torch.SparseFloatTensor mat1, torch.FloatTensor mat2)
  • (float alpha, torch.SparseFloatTensor mat1, torch.FloatTensor mat2)
  • (float beta, float alpha, torch.FloatTensor mat1, torch.FloatTensor mat2)
  • (float beta, float alpha, torch.SparseFloatTensor mat1, torch.FloatTensor mat2)
2

the input to your network is on the CPU, but your network is transferred to the GPU (or vice versa).

Here’s the critical part of the error that helped me figure this out:

 got (int, int, torch.FloatTensor, torch.cuda.FloatTensor),
3

I see. Got it to work. Thanks a lot for your prompt reply!

5

Hi,
what about this situation? I am ture that my input data is on GPU.
TypeError: addmm_ received an invalid combination of arguments - got (int, int, torch.cuda.LongTensor, torch.cuda.FloatTensor), but expected one of:

Thanks in advance.

6

@Ke_Bai You might want to read this post https://discuss.pytorch.org/t/problems-with-weight-array-of-floattensor-type-in-loss-function/381. I too had the issue where both the network and inputs were on GPU, causing the error. But the input type was DoubleTensor, however it should have been FloatTensor instead.

7

Sir, I’d like to ask for your help. I adopt a simple feed-forward net to classify feature maps. Following is the code:
class FmDataLoad(Dataset):
def init(self, npy_file, root_dir):
self.npy_data = np.load(npy_file)
self.root_dir = root_dir

def __len__(self):
    return self.npy_data.shape[0]

def __getitem__(self, index):
    fm_data = np.array(self.npy_data[:, :-1], dtype = np.float)
    label_data = np.array(self.npy_data[:, -1], dtype = np.float)

    return fm_data, label_data

train_data = FmDataLoad(npy_file = ‘models/train_data_conv22.npy’, root_dir = ‘models/’)
test_data = FmDataLoad(npy_file = ‘models/test_data_conv22.npy’, root_dir = ‘models/’)

class Net(nn.Module):
def init(self, input_size, hidden_size, num_classes):
super(Net, self).init()
self.fc1 = nn.Linear(input_size, hidden_size)
self.relu = nn.ReLU()
self.fc2 = nn.Linear(hidden_size, num_classes)

def forward(self, x):
    out = self.fc1(x)
    out = self.relu(out)
    out = self.fc2(out)
    return out

net = Net(input_size, hidden_size, num_classes)
net.cuda()

criterion = nn.MSELoss()
optimizer = torch.optim.Adam(net.parameters(), lr=learning_rate)

for epoch in range(num_epochs):
for i in range(len(train_data)):
for fm_data, label_data in train_data:
fm_data = torch.from_numpy(fm_data).float().cuda()
fm_data = Variable(fm_data)

        label_data = torch.from_numpy(label_data).float().cuda()
        label_data = Variable(label_data)

        optimizer.zero_grad()  # zero the gradient buffer
        outputs = net(fm_data)
        loss = criterion(outputs, label_data)
        loss.backward()
        optimizer.step()

        if (i + 1) % 100 == 0:
            print ('Epoch [%d/%d], Step [%d/%d], Loss: %.4f'
                   %(epoch+1, num_epochs, i+1, len(train_data)//batch_size, loss.data[0]))

The error is:
Traceback (most recent call last):
File “feed_forward.py”, line 64, in
loss = criterion(outputs, label_data)
File “/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py”, line 224, in call
result = self.forward(*input, **kwargs)
File “/usr/local/lib/python2.7/dist-packages/torch/nn/modules/loss.py”, line 272, in forward
return F.mse_loss(input, target, size_average=self.size_average)
File “/usr/local/lib/python2.7/dist-packages/torch/nn/functional.py”, line 819, in mse_loss
return _functions.thnn.MSELoss.apply(input, target, size_average)
File “/usr/local/lib/python2.7/dist-packages/torch/nn/_functions/thnn/auto.py”, line 47, in forward
output, *ctx.additional_args)
RuntimeError: input and target have different number of elements: input[6060 x 3] has 18180 elements, while target[6060] has 6060 elements at /pytorch/torch/lib/THCUNN/generic/MSECriterion.cu:12

could you give me an advice to solve this problem? Thank you very much!

8

For MSELoss, the output and target (label_data in your case) must be of same shape. You are likely looking for CrossEntropyLoss.

By the way, your question has nothing to do with the original post. So you should post a new thread instead replying in here. Further more, the error message is quite clear in your case, you might be able to solve such issues faster if you think about the message and consult the doc in future…

9

Thank you for your advice. I have tried to used CroosEntropy, but the error is:
TypeError: CudaClassNLLCriterion_updateOutput received an invalid combination of arguments - got (int, torch.cuda.FloatTensor, torch.cuda.FloatTensor, torch.cuda.FloatTensor, bool, NoneType, torch.cuda.FloatTensor, int), but expected (int state, torch.cuda.FloatTensor input, torch.cuda.LongTensor target, torch.cuda.FloatTensor output, bool sizeAverage, [torch.cuda.FloatTensor weights or None], torch.cuda.FloatTensor total_weight, int ignore_index)

I cannot find the answer according to the former related post, so I put the question here.
Thank you again.

12

Hi Nelson

I have the same problem.
May I know how you got it worked?

13

Hey. Have you got it worked?