# Splitting ImageFolder into data and labels? **URL:** \ **Category:** Uncategorized\ **Created:** [May 18, 2020, 1:40pm UTC](https://discuss.pytorch.org/t/splitting-imagefolder-into-data-and-labels/81825 "2020-05-18T13:40:05Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Michel\_Sfeir](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/michel_sfeir/32/24419_2.png) [@Michel\_Sfeir](https://discuss.pytorch.org/u/Michel_Sfeir)\ **Post date:** [May 18, 2020, 1:40pm UTC](https://discuss.pytorch.org/t/splitting-imagefolder-into-data-and-labels/81825/1 "2020-05-18T13:40:05Z")
Hello all, I’m new here and sorry if this question has been asked before, I have tried searching for a long time now and cannot seem to find the solution. When I load a standard dataset from pytorch such as CIFAR, I can access the images and labels separately with dataset.data and dataset.targets. When I create a dataset with ImageFolder however, I only get back one list that does not contain .data or .label fields. I know that I can access the images and labels separately by doing dataset[i][0] or [1] to get respectively the ith image or label. But this list structure is bothersome because a) I cannot take the mean or std of my training set easily and b) I cannot pass the images and labels separately to stratifiedshufflesplit to create a validation set. (I know that I can use random\_split but that does not guarantee equal distribution of classes in the validation set). The way that I was using was to do: images = [] labels = [] for i in range(len(my\_data)): images.append(my\_data[i][0]) labels.append(my\_data[i][1]) Which would give me separate lists of just the data or labels that I could then turn into a tensor with tensor.stack() for the data and a numpy array for the labels, both much nicer to work with. This method works fine but uses up a lot of memory and is very slow. When trying to run it on Kaggle, 90% of the time my notebook restarts due to trying to use too much memory. Can someone help me divide up my imagefolder into data and labels in a neater/more efficient way? Or am I not supposed to try and do that? Is there something conceptually wrong with what I am attempting here? I also considered editing ImageFolder itself to have .data and .targets fields but I don’t know how I would go about doing that. ---
**Author:** ![Isaac\_Kargar](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/isaac_kargar/32/9424_2.png) [@Isaac\_Kargar](https://discuss.pytorch.org/u/Isaac_Kargar)\ **Post date:** [May 18, 2020, 4:50pm UTC](https://discuss.pytorch.org/t/splitting-imagefolder-into-data-and-labels/81825/2 "2020-05-18T16:50:47Z")
Could you please explain about your folder structure? data, labels, … ---
**Author:** ![Michel\_Sfeir](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/michel_sfeir/32/24419_2.png) [@Michel\_Sfeir](https://discuss.pytorch.org/u/Michel_Sfeir)\ **Post date:** [May 18, 2020, 4:58pm UTC](https://discuss.pytorch.org/t/splitting-imagefolder-into-data-and-labels/81825/3 "2020-05-18T16:58:08Z")
It goes like this: /root/train/[class names]/[images] /root/test/[only one class]/[testing images] Since my test data is not labeled, I’m only reading the train folder into ImageFolder and trying to split that into a training and validation set ---
**Author:** ![Isaac\_Kargar](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/isaac_kargar/32/9424_2.png) [@Isaac\_Kargar](https://discuss.pytorch.org/u/Isaac_Kargar)\ **Post date:** [May 18, 2020, 5:09pm UTC](https://discuss.pytorch.org/t/splitting-imagefolder-into-data-and-labels/81825/4 "2020-05-18T17:09:56Z")
I usually use a custom dataset class for this type of problems and in the ` __init__ ` method read the name of images and do sorting and other stuff if needed and then in the ` __getitem__ ` method, load the image and do the processing and return the image and its label. You can have a lot of freedom and can control everything in the class. You can check [this](https://pytorch.org/tutorials/beginner/data_loading_tutorial.html) tutorial for more details.