Datasets: change y values of existing dataset

Hi all,
I would love to get some assistance regarding Dataset handling.

My data:

  1. a single folder of images [~3K small images]
  2. a pandas dataframe containing a numeric value for each image [range 0 - 10000]

My Task:
to “predict” the numeric value of an unseen image - using convolution layers and fc layers.

since i am relatively new to pytorch and to this task, I am quiet struggling with creating the dataset.
i read some examples online - and in most of them the dataset [like cifar-10] is loaded already with the classes attached for each image.


  1. I tried to create a dataset using the ImageFolder class - which in turn makes all images class value 0. is there a way to change the class value with my actual values ?
  2. is my approach wrong ? should i try to create a new dataset ? and if so, how do i attach “the y values” to the “x values” ?
  3. since my computer isnt very strong - how can i divide my data into batches efficiently in order to train my entire dataset without the computer yelling “not enough memory space” ?

thanks !

P.S - if you think there is a category for this post - let me know
P.S - if you think the title of this question should be different - let me know
P.S - i tried reading this post :
but didnt understand how to apply it to my situation

You can define your custom dataset class extending the class. You have to implement the init(), getitem() and len() classes. Here is a good tutorial for writing custom datasets Pytorch Custom Dataset