I have very few data and want to store it all in gpu.
Still it more than i can use in minibatch.
What is best way to select random sub batch on every step?
You could randomly permute the indices and then slice the batch:
batch = torch.arange(10).view(10, 1)
# select 5 samples randomly
idx = torch.randperm(10)[:5]
mini_batch = batch[idx]