# Pytorch tensor dimension wise concatenation

I need to achieve my targets by developing computationally efficient code. Here is the scenario:

``````x     # torch.Size([2, 5, 256]) (Batch, input_1st_dim, input_2nd_dim)
``````

Now I want to concatenate the `input_1st_dim` index-wise, like 1st with all the following four, then `2nd` with the next `three`, then `3rd` with the next `two`, then `4th` with the last `one`.
Finally, I want to get `x` as `[2, 15, not sure about this dimension]`

I can do it with nested loops and some extra lines of code. Here is my code:

``````   def my_fun(self, x):
iter = x.shape
counter = 0
new_x = torch.zeros((10, x.shape), dtype=torch.float32, device=torch.device('cuda'))
for i in range(0, x.shape - 1):
iter -= 1
for j in range(0, iter):
mean = (x[i, :] + x[i+j, :])/2
new_x[counter, :] = torch.unsqueeze(mean, 0)
counter += 1
final_T = torch.cat((x, new_x), dim=0)
return final_T
``````
``````ref = torch.zeros((x.shape, 15, x.shape), dtype=torch.float32, device=torch.device('cuda'))
for i in range (x.shape):
ref[i, :, :] = self.my_fun(x[i, :, :])
``````

But, is there any computationally efficient way to code it?

Could you post this slow approach so that we could use it as a reference to compare potentially faster approaches, please?

I stuck with one error:
`x.shape # torch.Size([5, 256])`
`new_x.shape # torch.Size([10, 256])`
Now I want concatenate these two tensors row wise means in result want to get (15, 256) and for that I am using `final_T = torch.cat((x, new_x), -1, device='cuda')`, but getting error:

TypeError: cat() received an invalid combination of arguments - got (tuple, int, device=str), but expected one of:

• (tuple of Tensors tensors, int dim, *, Tensor out)
• (tuple of Tensors tensors, name dim, *, Tensor out)`

This should work:

``````x = torch.randn(5, 256)
new_x = torch.randn(10, 256)
out = torch.cat((x, new_x), dim=0)
print(out.shape)
# torch.Size([15, 256])
``````
1 Like

your examples work, but in my example case its does not work. I tried `final_T = torch.cat((x, new_x), dim=0, device='cuda')`

Detailed Code is:
iter = x.shape
counter = 0
new_x = torch.zeros((10, x.shape), dtype=torch.float32)
for i in range(0, x.shape - 1):
iter -= 1
for j in range(0, iter):
mean = (x[i, :] + x[i+j, :])/2
new_x[counter, :] = torch.unsqueeze(mean, 0)
counter += 1
final_T = torch.cat((x, new_x), dim=0)

Remove the `device` in `torch.cat` argument as it’s unexpected.

Thanks for efficiently responding. But after removing `device`, I got:
`RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument tensors in method wrapper___cat)`

In that case move the `CPUTensor` to the `device` via `tensor.to(device)`, but don’t pass it as an argument to `torch.cat`.

1 Like

Thank you so much. Now its work.

I finished my version and updated the post with my code. Kindly check whether there is a more efficient way to accomplish this. Furthermore, please double-check that my logic matches the problem stated in the post. Thank you very much.

@ptrblck Waiting for your response. Thanks