Datasets: change y values of existing dataset

1

Hi all,
I would love to get some assistance regarding Dataset handling.

My data:

  1. a single folder of images [~3K small images]
  2. a pandas dataframe containing a numeric value for each image [range 0 - 10000]

My Task:
to “predict” the numeric value of an unseen image - using convolution layers and fc layers.

Status:
since i am relatively new to pytorch and to this task, I am quiet struggling with creating the dataset.
i read some examples online - and in most of them the dataset [like cifar-10] is loaded already with the classes attached for each image.

Questions:

  1. I tried to create a dataset using the ImageFolder class - which in turn makes all images class value 0. is there a way to change the class value with my actual values ?
  2. is my approach wrong ? should i try to create a new dataset ? and if so, how do i attach “the y values” to the “x values” ?
  3. since my computer isnt very strong - how can i divide my data into batches efficiently in order to train my entire dataset without the computer yelling “not enough memory space” ?

thanks !

P.S - if you think there is a category for this post - let me know
P.S - if you think the title of this question should be different - let me know
P.S - i tried reading this post : https://pytorch.org/tutorials/beginner/data_loading_tutorial.html
but didnt understand how to apply it to my situation

2

You can define your custom dataset class extending the torch.utils.data.Dataset class. You have to implement the init(), getitem() and len() classes. Here is a good tutorial for writing custom datasets Pytorch Custom Dataset