Where can I find the simplest vision dataset?

Hi, I want to find a very simple dataset of let’s say 10 images per class and few very classes. I would need small dataset since I’m modifying pytorch source code. And adding log statements. So, when there are num_workers > 0, then it would be very difficult to trace what’s happening internally.

The simplest dataset I know is lenet / mnist. But, this itself is very big for me and would need a much smaller one.
I know that I could always prune the existing dataset manually. But, that;s the last option that I want to have.

Thanks a lot !!!

Writing a custom Dataset might be a good idea as it would give you the flexibility to manipulate it as you wish:

class MyDataset(Dataset):
    def __init__(self, nb_images=10):
        data = torch.ones(3, 32, 32).expand(nb_images, -1, -1, -1)
        self.data = data * torch.arange(nb_images)[:, None, None, None].expand_as(data)
        self.target = torch.arange(nb_images)
        
    def __getitem__(self, index):
        x = self.data[index]
        y = self.target[index]
        return x, y
    
    def __len__(self):
        return len(self.data)

dataset = MyDataset()
for x, y in dataset:
    print(x)
    print(y)

or using Subset might also work, but would “hide” the dataset behind the wrapper.

Hi @ptrblck I’m not able to follow the operations in the first and second line of the __init__ function.
In the first line, we create a tensor of 3*32*32 and expand it to 10 images. So, the shape of data at this point is 10*3*32*32

In the second line, we are constructing a new 10*1 tensor using arange and the expanding it to the same dimesnion as that of data which would be 10*3*32*32.

And this new tensor of shape 10*3*32*32 is multiplied with data of shape 10*3*32*32 ?

Yes, your explanation is right and I’ve used the code to create nb_images each containing a different value.
I’ll reduce the nb_images as well as the spatial size as it would otherwise be a long output in this post but here is the result:

nb_images = 5
data = torch.ones(3, 5, 5).expand(nb_images, -1, -1, -1)
data = data * torch.arange(nb_images)[:, None, None, None].expand_as(data)
print(data)

tensor([[[[0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.]],

         [[0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.]],

         [[0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.]]],


        [[[1., 1., 1., 1., 1.],
          [1., 1., 1., 1., 1.],
          [1., 1., 1., 1., 1.],
          [1., 1., 1., 1., 1.],
          [1., 1., 1., 1., 1.]],

         [[1., 1., 1., 1., 1.],
          [1., 1., 1., 1., 1.],
          [1., 1., 1., 1., 1.],
          [1., 1., 1., 1., 1.],
          [1., 1., 1., 1., 1.]],

         [[1., 1., 1., 1., 1.],
          [1., 1., 1., 1., 1.],
          [1., 1., 1., 1., 1.],
          [1., 1., 1., 1., 1.],
          [1., 1., 1., 1., 1.]]],


        [[[2., 2., 2., 2., 2.],
          [2., 2., 2., 2., 2.],
          [2., 2., 2., 2., 2.],
          [2., 2., 2., 2., 2.],
          [2., 2., 2., 2., 2.]],

         [[2., 2., 2., 2., 2.],
          [2., 2., 2., 2., 2.],
          [2., 2., 2., 2., 2.],
          [2., 2., 2., 2., 2.],
          [2., 2., 2., 2., 2.]],

         [[2., 2., 2., 2., 2.],
          [2., 2., 2., 2., 2.],
          [2., 2., 2., 2., 2.],
          [2., 2., 2., 2., 2.],
          [2., 2., 2., 2., 2.]]],


        [[[3., 3., 3., 3., 3.],
          [3., 3., 3., 3., 3.],
          [3., 3., 3., 3., 3.],
          [3., 3., 3., 3., 3.],
          [3., 3., 3., 3., 3.]],

         [[3., 3., 3., 3., 3.],
          [3., 3., 3., 3., 3.],
          [3., 3., 3., 3., 3.],
          [3., 3., 3., 3., 3.],
          [3., 3., 3., 3., 3.]],

         [[3., 3., 3., 3., 3.],
          [3., 3., 3., 3., 3.],
          [3., 3., 3., 3., 3.],
          [3., 3., 3., 3., 3.],
          [3., 3., 3., 3., 3.]]],


        [[[4., 4., 4., 4., 4.],
          [4., 4., 4., 4., 4.],
          [4., 4., 4., 4., 4.],
          [4., 4., 4., 4., 4.],
          [4., 4., 4., 4., 4.]],

         [[4., 4., 4., 4., 4.],
          [4., 4., 4., 4., 4.],
          [4., 4., 4., 4., 4.],
          [4., 4., 4., 4., 4.],
          [4., 4., 4., 4., 4.]],

         [[4., 4., 4., 4., 4.],
          [4., 4., 4., 4., 4.],
          [4., 4., 4., 4., 4.],
          [4., 4., 4., 4., 4.],
          [4., 4., 4., 4., 4.]]]])

You can of course use any random input data if this fits your use case better.

1 Like