Torch.cat for cuda tensors

1 
concatenated_input = Variable(torch.cat([input.data.view(-1,3*32*32), condition.data], 1))

TypeError: cat received an invalid combination of arguments - got (list, int), but expected one of:

  • (sequence[torch.cuda.FloatTensor] seq)
  • (sequence[torch.cuda.FloatTensor] seq, int dim)
    didn’t match because some of the arguments have invalid types: (list, int)

This works if they are simply FloatTensors and not cuda.FloatTensors. Is this expected? Should i use a different function to concatenate cuda tensors?

Thanks a lot!

2 Likes
2

maybe input.data and condition.data are of different types. Is one of them a torch.FloatTensor and the other of type torch.cuda.FloatTensor?

3 Likes
3

You were right = condition.data was a FloatTensor. This was a bit of a surprise because I explicitly called .cuda() on the condition variable. Is there something that I am obviously missing?

Thanks!
Pratheeksha

4

Calling .cuda() on a Variable is not in-place, so you have to do var = var.cuda().

2 Likes
5

Hi,

I’m new in PyTorch and I have the same exactly the problem using the concatenate function.

My both variables padding and source are “torch.LongTensor”

Probably I’m doing something wrong. So this is my code:

padding = torch.LongTensor(np.zeros(args.distance_context, dtype=np.int))
sequence = torch.cat([padding, source[i:seq_len+1]],dim=0)

TypeError: cat received an invalid combination of arguments - got (list, dim=int), but expected one of:

  • (sequence[torch.LongTensor] seq)
  • (sequence[torch.LongTensor] seq, int dim)
    didn’t match because some of the arguments have invalid types: (list, dim=int)

Did you solve this problem ?

Thanks you very much !

6

If I use .cuda() in each Tensor I get this message:

sequence = torch.cat([padding,sub_sequence],dim=0)
RuntimeError: inconsistent tensor sizes at /b/wheel/pytorch-src/torch/lib/THC/generic/THCTensorMath.cu:141

7

padding and sub_sequence are of what shape? they have to be of the same shape except in dim=0 (the dimension in which they are concatenated can be different shape, but other dimensions should have same shape).

1 Like
8

Hi,

I am having the same problem:

[...]
    x = torch.cat([x, fill], 1)
TypeError: cat received an invalid combination of arguments - got (list, int), but expected one of:
 * (sequence[torch.FloatTensor] seq)
 * (sequence[torch.FloatTensor] seq, int dim)
      didn't match because some of the arguments have invalid types: (list, int)

After reading this discussion and similar others, I checked type(_) and _.data.type() of x and fill:

x:
torch.FloatTensor
<class 'torch.autograd.variable.Variable'>
torch.Size([10, 1, 40])
fill:
torch.FloatTensor
<class 'torch.autograd.variable.Variable'>
torch.Size([10, 1, 40])

I am not using .cuda(), and both tensors seem to have the same size and type.

This is the code, which causes the problem:

if x.size()[1] < batch_size:
    fill = (x[:, 0, :].contiguous().view(x.size()[0], 1, x.size()[2]))
    for x in range(batch_size-x.size()[1]):
        x = torch.cat([x, fill], 1)

(I want to fill the last batch of an epoch in case the number of elements in the data set % batch_size is not 0)

Any idea what could be the problem here?

Thanks!

9

I’m facing the similiar problem using pytorch 0.3.1.

When invoking torch.cat(hidden,encoder_outputs)
hidden type : [torch.cuda.FloatTensor of size 32x90x512 (GPU 0)]
encoder_outputs type: [torch.cuda.FloatTensor of size 32x90x512 (GPU 0)]

Traceback (most recent call last):
File “/home/yb/project/crnn.pytorch-master/crnn_att_main.py”, line 365, in
cost = trainBatch(crnn, optimizer)
File “/home/yb/project/crnn.pytorch-master/crnn_att_main.py”, line 340, in trainBatch
preds = crnn(image,batch_label_tensor) #44X3X7
File “/ENTER/lib/python2.7/site-packages/torch/nn/modules/module.py”, line 357, in call
result = self.forward(*input, **kwargs)
File “/ENTER/lib/python2.7/site-packages/torch/nn/parallel/data_parallel.py”, line 71, in forward
return self.module(*inputs[0], **kwargs[0])
File “/ENTER/lib/python2.7/site-packages/torch/nn/modules/module.py”, line 357, in call
result = self.forward(*input, **kwargs)
File “/home/yb/project/crnn.pytorch-master/models/crnn_att.py”, line 201, in forward
output, hidden, encoder_outputs)
File “/ENTER/lib/python2.7/site-packages/torch/nn/modules/module.py”, line 357, in call
result = self.forward(*input, **kwargs)
File “/home/yb/project/crnn.pytorch-master/models/crnn_att.py”, line 39, in forward
rnn_input = torch.cat([embedded, context.data], 2)
TypeError: cat received an invalid combination of arguments - got (list, int), but expected one of:

  • (sequence[torch.FloatTensor] seq)
  • (sequence[torch.FloatTensor] seq, int dim)
    didn’t match because some of the arguments have invalid types: (list, int)

So the problem is that the two input tensors are both cuda.FloatTensor, but it’s not the expected type.
Could you please give some advice on how to use cat func for cuda.FloatTensor