Inserting and popping to a Tensor?

#1

Let’s say I have two tensors:

A = torch.empty([N, M]).to(device)
B = torch.empty([batch_size, M]).to(device)

The desired effect I want to achieve is this:

A = torch.cat((B.to(device), A[:-batch_size])).to(device)

In plain English, I just want to insert B into the front of A but keep A the same size.

However, when I do it with the following code:

A[batch_size:].data = A[:-batch_size].data.to(device)
A[batch_size:].data = B.data.to(device)

Now I get a CUDA error:

RuntimeError: CUDA error: device-side assert triggered

The assert it’s referring seems to be:

Assertion index >= -sizes[i] && index < sizes[i] && “index out of bounds” failed.

#2

I don’t think I got your question properly. But is this what you are trying?

a = torch.empty(10,2).to(device)
b = torch.empty(5,2).to(device)
batch_size = 5
a[:batch_size, :] = b

Here, the device can be any “cpu” or “cuda”. numbers 10,5,2 are random examples for demonstration (instead of using N, batch_size, M, but the order remains same).

#3

Close, but no, I’m trying to implement a kind of queue. So first in, first out. Your 4th line of code overwrites the top of a, but I want to shift a so that the top is preserved and insert b before it, if that makes sense.

#4

Oh but I see your point! Yes, I would do

A[batch_size:].data = A[:-batch_size].data.to(device)
A[:batch_size].data = B.data.to(device)

I missed that that colon was on the wrong side.

#5

yeah, is it solved then ?

#6

Never mind, still getting the same CUDA error, which I don’t get when I do:

A = torch.cat((B.to(device), A[:-batch_size])).to(device)

#7

I changed it a bit and now I’m getting:

A[batch_size:] = A[:-batch_size].to(device)

RuntimeError: unsupported operation: some elements of the input tensor and the written-to tensor refer to a single memory location. Please clone() the tensor before performing the operation.

But that’s exactly what’s shown being done here: python - Is it possible to create a FIFO queue with pyTorch? - Stack Overflow