To slice up sequence of data into shorter sequences

1

Hello!
I have sequences of data created from multiple agents playing a game for a given amount of timesteps in synchron, thus with a shape of (#agents, #timesteps, x, …), where the data denoted by ‘x, …’ can be of any shape.

What would be an easy way to slice the ‘timestep’ long sequences of data into smaller sequential parts?

A specific example (with a picture as data):
From (32, 1024, 3, 28, 28) into (128, 256, 3, 28, 28), as we create 256 long sequences from 1024 long sequences and as a result increase our “number” of data from 32 to 4*32=128.

My only guess would be to make a fancy indexing such as one would do on a list of lists of lists, but there must be a better way.

Thanks is advance.

2

I might have come up with a solution, but i am not sure whether the sequence order would be intact:

 # Long sequenced data
a = torch.randn((32, 1024, 3 , 28, 28)) 

# Get the original shape and modify it
b = [i for i in a.size()]  
b[0] = -1
b[1] = int(b[1] / 4) 

# Desired length sequences
c = a.view(b) 

>>>c.shape
torch.Size([128, 256, 3, 28, 28])
3

Hi @Dudly01,

Yes this is the way to go, why would the sequence order not be intact?
view doesn’t modify the underlying sequence, just how you “see” it.

and you can simply do this to get the shape list:

b = list(a.size())
1 Like