How to merge labels and images in Pytorch

data = np.load("training_data.npy", allow_pickle=True)

X = torch.Tensor([i[0] for i in data]).view(-1, 50, 50) 
X = X / 255.0

print(X.size())
y = torch.Tensor([i[1] for i in data])

I have this and want to have:

t_Data = torch.from_numpy(data)

But it gives error as:

TypeError                                 Traceback (most recent call last)
<ipython-input-23-ecb1b9da7431> in <module>
----> 1 t_Data = torch.from_numpy(data)

TypeError: can't convert np.ndarray of type numpy.object_. The only supported types are: float64, float32, float16, int64, int32, int16, int8, uint8, and bool.

Now I have two questions:

From what I get, when downloading dataset from torch vision and loading with dataloader it gives labels inside of it. However, I try to prepare my own dataset, so how could I merge labels and images into one dataset so that when loading with shuffle=True they won’t mismatch?

I have an another solution once that sampling batches sequentially with torch.utils.data.SequentialSampler(data_source) as it doesn’t require shuffle to be true. However, this doesn’t differ the batches(they are same in iteration), too, as if shuffle=True with none sampler! I don’t know 'sequential sampler’s point? Can anyone explain?

Hy, I guess what you’re asking for is how to load your own dataset into dataloader. One way to do this is using TensorDataset.

from torch.utils.data import DataLoader , TensorDataset
dataset = TensorDataset(X , y)
trainloader = DataLoader(dataset , batch_size = 16, shuffle=True)

2 Likes

that’s what I am looking for, thxx :slight_smile: