How to turn a list of tensor to tensor?

#1

I have a list and there are many tensor in the list
I want to turn it to just tensor, and I can put it to dataloader

I use for loop and cat the tensor but it is very slow, data size is about 4,800,000

9 Likes
Create a single tensor from list of tensors
Input a list of tensors to a model without the need to manually transfer each item to cuda
#2

If they’re all the same size, then you could torch.unsqueeze them in dimension 0 and then torch.cat the results together.

9 Likes
#3 
a = []
for i in range(100000):
    a.append(torch.rand(1, 100, 100)

b = torch.Tensor(100000, 100, 100)
torch.cat(a, out=b)
11 Likes
#4

I was always doing something like:

a = [torch.FloatTensor([1]).view(1, -1), torch.FloatTensor([2]).view(1, -1)]
torch.stack(a)

Gives me:

(0 ,.,.) = 
  1

(1 ,.,.) = 
  2
[torch.FloatTensor of size 2x1x1]
14 Likes
#5

Actually

I have two list

list 1
a = [[tensor 40], [tensor 40], [tensor 40], …] (2400000 tensor in list each tensor size is 40)
b = [[tensor 40], [tensor 40], [tensor 40], …] (2400000 tensor in list each tensor size is 40)

I want to concat a and b to c
c is a tensor and size is torch.Size([4800000, 40])

I use this method to solve my problem
a = torch.stack(a)
b = torch.stack(b)
c = torch.cat((a, b))

Thank you for all your help !

8 Likes
#6

Or you can just torch.stack(a + b)

10 Likes
#7

It works.
Thank you.

#8

This is how it should be done.

#9

a = torch.stack(a) worked for me

11 Likes
#10

To me it’s sort of unintuitive, why wouldn’t using the tensor class work?

torch.tensor([ torch.tensor([i]).repeat(15) for i in range(0,5)])

the list is the same size and it’s really a matrix/tensor already…but somehow only:

torch.stack([ torch.tensor([i]).repeat(15) for i in range(0,5)])
tensor([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
        [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
        [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2],
        [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3],
        [4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4]])

worked.

Btw, is this this most efficient way to do it in a vectorized way?