# Questions about ImageFolder

**URL:** <https://discuss.pytorch.org/t/questions-about-imagefolder/774>\
**Category:** Uncategorized\
**Created:** [February 28, 2017, 3:03am UTC](https://discuss.pytorch.org/t/questions-about-imagefolder/774 "2017-02-28T03:03:55Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![yichuan9527](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/y/7c8e57/32.png) [@yichuan9527](https://discuss.pytorch.org/u/yichuan9527)\
**Post date:** [February 28, 2017, 3:03am UTC](https://discuss.pytorch.org/t/questions-about-imagefolder/774/1 "2017-02-28T03:03:55Z")

</div>

Which data format of the picture that ImageFolder can read? (pickle, JPG or PNG)

---

<div class="post-metadata">

**Author:** ![ncullen93](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ncullen93/32/68760_2.png) [@ncullen93](https://discuss.pytorch.org/u/ncullen93)\
**Post date:** [February 28, 2017, 3:07am UTC](https://discuss.pytorch.org/t/questions-about-imagefolder/774/2 "2017-02-28T03:07:51Z")

</div>

From the code (`torchvision.datasets.folder.py`), supported formats:

```auto
IMG_EXTENSIONS = [
    '.jpg', '.JPG', '.jpeg', '.JPEG',
    '.png', '.PNG', '.ppm', '.PPM', '.bmp', '.BMP',
]

```

---

<div class="post-metadata">

**Author:** ![yichuan9527](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/y/7c8e57/32.png) [@yichuan9527](https://discuss.pytorch.org/u/yichuan9527)\
**Post date:** [February 28, 2017, 3:09am UTC](https://discuss.pytorch.org/t/questions-about-imagefolder/774/3 "2017-02-28T03:09:17Z")

</div>

Ok, I found the solution。  
 ![](https://discuss.pytorch.org/uploads/default/original/1X/bb2e9ad910ef74c69c9b32d53b5985e4a2665a7e.png)  
but, how to read labels of pictures

---

<div class="post-metadata">

**Author:** ![yichuan9527](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/y/7c8e57/32.png) [@yichuan9527](https://discuss.pytorch.org/u/yichuan9527)\
**Post date:** [February 28, 2017, 3:10am UTC](https://discuss.pytorch.org/t/questions-about-imagefolder/774/4 "2017-02-28T03:10:10Z")

</div>

Thank you very much!😁😁  
But how can I read lables?

---

<div class="post-metadata">

**Author:** ![xiahouzuoxin](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/x/f14d63/32.png) [@xiahouzuoxin](https://discuss.pytorch.org/u/xiahouzuoxin)\
**Post date:** [February 28, 2017, 3:34am UTC](https://discuss.pytorch.org/t/questions-about-imagefolder/774/5 "2017-02-28T03:34:47Z")

</div>

The directory name is the class name I think, also from [torchvision.datasets.folder.py](http://torchvision.datasets.folder.py)

```
def find_classes(dir):
classes = [d for d in os.listdir(dir) if os.path.isdir(os.path.join(dir, d))]
classes.sort()
class_to_idx = {classes[i]: i for i in range(len(classes))}
return classes, class_to_idx
```

---

<div class="post-metadata">

**Author:** ![ncullen93](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ncullen93/32/68760_2.png) [@ncullen93](https://discuss.pytorch.org/u/ncullen93)\
**Post date:** [February 28, 2017, 3:38am UTC](https://discuss.pytorch.org/t/questions-about-imagefolder/774/6 "2017-02-28T03:38:46Z")

</div>

The labels are the sub-folders from the main directory. Say you have mnist images separated by digit like this:

```auto
main_dir/
     0/
         img1_digit0.jpg
         img2_digit0.jpg
     1/
         img3_digit1.jpg
         ....
     ....
     9/
        ...

```

Then you create an `ImageFolder` object.

```python
from torchvision.datasets import ImageFolder
from torchvision.transforms import ToTensor
data = ImageFolder(root='main_dir', transform=ToTensor())

```

Note that you have the `ToTensor()` transform to convert from jpg to torch tensor.

Now the unique set of class labels is found easily, but this isn’t the class label for each individual image.

```python
print(data.classes) # ['0','1',..'9']

```

And the images can be accessed by an integer index:

```python
x,y = data[0] # x is the first image as PIL, y is that images class label

```

You can easily iterate through these images:

```python
for i in range(len(data)):
    x,y = data[i]

```

But for sampling like an iterator, I think you should use `DataLoader` class:

```python
from torch.utils.data import DataLoader
loader = DataLoader(data)
for x, y in loader:
    print(x) # image
    print(y) # image label

```

Now, just FYI - I believe this will sample all the `0` images first, then all the `1` images, etc. To shuffle the images instead, you can either set `shuffle=True` in the `DataLoader`:

```python
loader = DataLoader(data, shuffle=True)

```

Or, you can create an instance of the `RandomSampler` class found in `torch.utils.data.sampler.py` and pass that into the `DataLoader` as the `sampler` argument:

```python
from torch.utils.data import RandomSampler
sampler = RandomSampler(data)
loader = DataLoader(data, sampler=sampler)

```

Also, note that if you want to read in data in a folder of a different type (e.g. numpy arrays), you can make your own function and pass it into the `ImageFolder` class in the `loader` argument. This function should take in a file path and return the loaded data.

```python
import numpy as np
def my_numpy_loader(filename):
    return np.load(filename)

data = ImageFolder(root='path/to/numpy/maindir/', loader=my_numpy_loader, transform=ToTensor())

```

The `ToTensor()` transform will do its job to convert the numpy array to a torch tensor. However, unfortunately it seems that all the other transforms (e.g. cropping, flipping, etc) only work on PIL images, so you’ll have to build your own transforms for that.

EDIT: As @apaszke notes, you can use the `ToPILImage()` transform to convert your array to PIL to use the transforms… I’d personally prefer transforms to work directly on torch tensors, but that’s just my opinion.

Example:

```python
import numpy as np
def my_numpy_loader(filename):
    return np.load(filename)
from torchvision.transforms import Compose, ToPILImage, ToTensor, ...
my_transform = Compose([ToPILImage(), ... typical transforms .., ToTensor()])
data = ImageFolder(root='path/to/numpy/maindir/', 
                loader=my_numpy_loader, transform=my_transform)

```

Here, you convert numpy -\> pil image -\> do some transforms -\> convert back to torch tensor…

---

<div class="post-metadata">

**Author:** ![Veril](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/v/77aa72/32.png) [@Veril](https://discuss.pytorch.org/u/Veril)\
**Post date:** [February 28, 2017, 4:08am UTC](https://discuss.pytorch.org/t/questions-about-imagefolder/774/7 "2017-02-28T04:08:37Z")

</div>

Speaking of formats, would it be possible to add secondary LSUN datasets to torchvision? ex. bird.  
Was somewhat surprised they weren’t supported.

[http://www.yf.io/p/lsun](http://www.yf.io/p/lsun)  
"20 object categories"  
[http://tigress-web.princeton.edu/~fy/lsun/public/release/](http://tigress-web.princeton.edu/~fy/lsun/public/release/)

---

<div class="post-metadata">

**Author:** ![ncullen93](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ncullen93/32/68760_2.png) [@ncullen93](https://discuss.pytorch.org/u/ncullen93)\
**Post date:** [February 28, 2017, 4:21am UTC](https://discuss.pytorch.org/t/questions-about-imagefolder/774/8 "2017-02-28T04:21:41Z")

</div>

submit an issue on [https://github.com/pytorch/vision](https://github.com/pytorch/vision)

---

<div class="post-metadata">

**Author:** ![yichuan9527](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/y/7c8e57/32.png) [@yichuan9527](https://discuss.pytorch.org/u/yichuan9527)\
**Post date:** [February 28, 2017, 7:44am UTC](https://discuss.pytorch.org/t/questions-about-imagefolder/774/9 "2017-02-28T07:44:53Z")

</div>

Dose pytorch have any functions to read the images from pickle file?

---

<div class="post-metadata">

**Author:** ![apaszke](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/apaszke/32/21_2.png) [@apaszke](https://discuss.pytorch.org/u/apaszke)\
**Post date:** [February 28, 2017, 12:57pm UTC](https://discuss.pytorch.org/t/questions-about-imagefolder/774/10 "2017-02-28T12:57:25Z")

</div>

@yichuan9527 why not use `pickle.load`? If you want to use it with ImageFolder just provide `pickle.load` as load function.

@Veril yes, if you think there’s a dataset that we don’t have but it would be useful, please open an issue in the vision repo.

@ncullen93 all the transforms work only on PIL images, but you can use `ToPILImage()` to convert a tensor back to a PIL image.

---

<div class="post-metadata">

**Author:** ![FuriouslyCurious](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/furiouslycurious/32/294_2.png) [@FuriouslyCurious](https://discuss.pytorch.org/u/FuriouslyCurious)\
**Post date:** [March 18, 2017, 10:20pm UTC](https://discuss.pytorch.org/t/questions-about-imagefolder/774/11 "2017-03-18T22:20:54Z")

</div>

I noticed that ImageFolder can process following structure:

```
root/dog/xxx.png
root/dog/xxy.png

root/cat/123.png
root/cat/asd932_.png

```

My question is, can it process a deeper /nested folder structure? e.g.:

```
root/mammal/dog/xxx.png
root/mammal/cat/xxy.png

root/avian/pigeon/123.png
root/avian/crow/nsdf3.png

```

If yes, would the classes be “mammal, avian”, or would they be “dog, cat, pigeon, crow”?

Thanks!

---

<div class="post-metadata">

**Author:** ![chenyuntc](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/chenyuntc/32/299_2.png) [@chenyuntc](https://discuss.pytorch.org/u/chenyuntc)\
**Post date:** [March 19, 2017, 7:50am UTC](https://discuss.pytorch.org/t/questions-about-imagefolder/774/12 "2017-03-19T07:50:09Z")

</div>

it will return an empty dataset. because there is no pictures in folder `mamma` and `avian`.

You can just create **softlink** in linux, to make them in the same folder.

Also the source code of `Imagefolder` is really simple to read and modify, you can inherit it and make some modify. have a try.  
like

```auto
class TrainImageFolder(ImageFolder):

    def __getitem__ (self, index):
            filename = self.imgs[index]# 'avian/crow/nsdf3.png'
            if filename.split('/')[0] =='avian': label = 0
            ### or 
            # if filename.split('/')[1] =='dog': label = 0
            return super(TrainImageFolder, self). __getitem__ (index)[0], label

```

---

<div class="post-metadata">

**Author:** ![Stefan\_Radonjic](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/stefan_radonjic/32/69632_2.png) [@Stefan\_Radonjic](https://discuss.pytorch.org/u/Stefan_Radonjic)\
**Post date:** [February 9, 2018, 7:06pm UTC](https://discuss.pytorch.org/t/questions-about-imagefolder/774/13 "2018-02-09T19:06:14Z")

</div>

Hello everyone! Does anyone know how can i merge two ImageFolder objects into one? If you can look at this issue i would really appreciate it:

> <https://datascience.stackexchange.com/questions/27626/perform-different-several-different-torchvision-transforms-on-imagefolder-object>

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [February 9, 2018, 8:17pm UTC](https://discuss.pytorch.org/t/questions-about-imagefolder/774/14 "2018-02-09T20:17:06Z")

</div>

You can use `ConcatDataset` [documentation](http://pytorch.org/docs/master/data.html#torch.utils.data.ConcatDataset) to merge different `Dataset` s into one.  
However, in your other question you would like to save the randomly transformed images, right?  
You could just iterate your Dataset and save the images using PIL. It’s not really common though, since you augment the dataset on the fly without saving it usually.

---

<div class="post-metadata">

**Author:** ![Stefan\_Radonjic](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/stefan_radonjic/32/69632_2.png) [@Stefan\_Radonjic](https://discuss.pytorch.org/u/Stefan_Radonjic)\
**Post date:** [February 9, 2018, 10:34pm UTC](https://discuss.pytorch.org/t/questions-about-imagefolder/774/15 "2018-02-09T22:34:59Z")

</div>

Thank you for your reply ! I just tried implementing the code i saw in the documentation you presented but i ran into an error. Here is a screenshot of my code :

 ![problem](https://discuss.pytorch.org/uploads/default/original/2X/8/895ad3d6cffc4ea09de2daa37b0414ef959351fd.png)

Line 42: im trying to merge two datasets which are loaded from the same path but performed different transformations on them - but when i print datasets len it saids 1218 (which is equal to a single dataset not two)  
Line 89 (I think) - Error for some reason

I have a small dataset of only 1200 images. My idea was to to load dataset, perform lets say Random Crop on it, then load dataset again but this time perform ColorJitter and then merge those two datasets into one which will result in dataset of 2400 pictures. I wanted to create a dataset of atleast 7200 pictures ( which is 6 transformations each time i load dataset) . Do you by any chance have a better idea?

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [February 10, 2018, 8:28pm UTC](https://discuss.pytorch.org/t/questions-about-imagefolder/774/16 "2018-02-10T20:28:19Z")

</div>

It’s strange that `ConcatDataset` does not throw an error, since its ` __init__ ` function only takes one argument and it looks like you’ve provided two (both datasets).  
Try to create the `ConcatDataset` like this:

```auto
class MyData(Dataset):
    def __init__ (self):
        self.data = torch.randn(100)
        
    def __getitem__ (self, index):
        return self.data[index]
    
    def __len__ (self):
        return len(self.data)
    
data1 = MyData()
data_loader1 = DataLoader(data1, batch_size=1)

for batch_idx, data in enumerate(data_loader1):
    print(batch_idx) # prints to idx 99
    
data2 = MyData()
concat_data = ConcatDataset((data1, data2))
concat_loader = DataLoader(concat_data, batch_size=1)

for batch_idx, data in enumerate(concat_loader):
    print(batch_idx) # prints to idx 199
    
print(len(data_loader1)) # 100
print(len(concat_loader)) # 200

```

I cannot see line 89. Would you please post the code snippet?

Generally, you could apply both transformations to your dataset.  
Since transformations like `RandomCrop` are applied randomly, one cannot say it’s doubling your data size.  
In each iteration your model will see a random transformation of your input.

---

<div class="post-metadata">

**Author:** ![Sumesh\_Uploader](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sumesh_uploader/32/10106_2.png) [@Sumesh\_Uploader](https://discuss.pytorch.org/u/Sumesh_Uploader)\
**Post date:** [October 3, 2018, 9:21pm UTC](https://discuss.pytorch.org/t/questions-about-imagefolder/774/18 "2018-10-03T21:21:37Z")

</div>

I was able to concat the data sets but unable to separate the respective image tensors while enumerating. When I just print the data it shows both tensor values in concatenated way. If I try something like the following:  
**for batch\_idx,(inp1,inp2) in enumerate(concat\_loader):**  
it gives “ValueError: not enough values to unpack (expected 2, got 1)”  
If i use zip the enumerator for loop goes into infinite loop.  
How to solve this?

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [October 4, 2018, 5:20am UTC](https://discuss.pytorch.org/t/questions-about-imagefolder/774/19 "2018-10-04T05:20:05Z")

</div>

`ConcatDateset` will concatenate both datasets on top of each other, so that the number of samples will grow. It won’t return both samples for each index.  
If you would like to do that, you could try to adapt this code:

```python
class MyDataset(Dataset):
    def __init__ (self, dataset1, dataset2):
        self.dataset1 = dataset1
        self.dataset2 = dataset2
        
    def __getitem__ (self, index):
        x1 = self.dataset1[index]
        x2 = self.dataset2[index]
        
        return x1, x2
    
    def __len__ (self):
        return len(self.dataset1)

dataset1 = TensorDataset(torch.randn(100, 1))
dataset2 = TensorDataset(torch.randn(100, 1))
dataset = MyDataset(dataset1, dataset2)

```

---

<div class="post-metadata">

**Author:** ![nimning](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/n/bcef8e/32.png) [@nimning](https://discuss.pytorch.org/u/nimning)\
**Post date:** [November 1, 2018, 7:20am UTC](https://discuss.pytorch.org/t/questions-about-imagefolder/774/20 "2018-11-01T07:20:20Z")

</div>

Hi, Nick

Is there way I can edit the IMG\_EXTENSIONS so that the ImageFolder would be able to read more image with .gif and .oct-stream extension?

Best

---

<div class="post-metadata">

**Author:** ![Alan\_Wu](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alan_wu/32/13846_2.png) [@Alan\_Wu](https://discuss.pytorch.org/u/Alan_Wu)\
**Post date:** [July 8, 2019, 2:39am UTC](https://discuss.pytorch.org/t/questions-about-imagefolder/774/21 "2019-07-08T02:39:12Z")

</div>

I just have a little question. Say the image folder is like this:  
root/dog/xxx.png  
root/dog/xxy.png

root/cat/123.png  
root/cat/asd932\_.png

Then I suppose it’s that the class dog is encoded into [1, 0] as for the first category, And the cat goes to [0, 1], Am I right?

[Next page](https://discuss.pytorch.org/t/questions-about-imagefolder/774.md?page=2)