How to use own data on 'Training a Classifier'

1

Dear PyTorch-Community!

I’m very new to the programming using python and from time to time I feel too dumb to understand it. I got I really basic problem right now, but there’s someone out there to help, for sure

Most of you might know the ‘training a classifier’ algorythm from pytorch (Training a Classifier — PyTorch Tutorials 1.8.1+cu102 documentation). I’d like to use it on my own dataset of cow-pictures.
Problem: How do I get my own data instead of the cifar10 dataset in there? I tried a million things but none worked…

Thanks in advance!

2

What have you tried already and where are you stuck at the moment?
Knowing about the failed attempts would be helpful so that we could point you towards potential issues.
In the meantime you could take a look at this tutorial, which explains how to write a custom Dataset and use a DataLoader.

3

Dear ptrblck,
Thank you so much for your response!

So far I annotated my pictures using CVAT and also got a csv-file with those annotations.
The tutorial is giving me this code, where I poorly tried to modify the input to be my dataset:

transform = transforms.Compose(
    [transforms.ToTensor(),
     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])

batch_size = 4

trainset = torchvision.datasets.CIFAR10(root='./data', train=True,
                                        download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=batch_size,
                                          shuffle=True, num_workers=2)

testset = torchvision.datasets.CIFAR10(root='./data', train=False,
                                       download=True, transform=transform)
testloader = torch.utils.data.DataLoader(testset, batch_size=batch_size,
                                         shuffle=False, num_workers=2)

classes = ('plane', 'car', 'bird', 'cat',
           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')

I tried to replace the “tochvision.datasets.CIFAR10(root='./data', train=True, download=True, transform=transform)” part with the root to my dataset and changed my classes to the two relevant for me: “cow”, “no_cow”

4

I wouldn’t try to reuse another Dataset if your custom data doesn’t match the layout, stored file format etc. Take a look at the creation of the FaceLandmarkDataset in the linked tutorial.
The basic idea is to load all paths, store the transformations etc. in the __init__, return the number of samples in the __len__, and load, process, as well as return each sample and target in the __getitem__.
The common approach would be to store the image paths in the __init__ with the corresponding targets and load each sample using the index in the __getitem__.

5

Alright, sounds reasonable

To be sure: If I define the dataset as described in the FaceLandmarkDataset tutorial I should be able to load it in this given code and use it?

6

Yes, you should be able to download the mentioned data and use it with the posted FaceLandmarkDataset. The tutorial is also executable on Google Colab.
For your custom Dataset you would of course need to change the paths and adapt the data structure etc.

1 Like