torch.utils.data.DataLoader get input class and image name?

1

hi ,is there a way to get the class and the original name of the tranfrom image,when using the model(x) code
to forward torch.utils.data.DataLoader as an input

normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406],
                                     std=[0.229, 0.224, 0.225])

val_loader = torch.utils.data.DataLoader(
        datasets.ImageFolder(valdir, transforms.Compose([
            transforms.Scale(256),
            transforms.CenterCrop(224),
            transforms.ToTensor(),
            normalize,
        ])),
      batch_size=1, shuffle=False,
        num_workers=4, pin_memory=True)

a torch.utils.data.DataLoader is created here

and i want to get the input result:

for i, (input, target) in enumerate(val_loader):
target_var = torch.autograd.Variable(target)
target = target.cuda(async=True)
x = torch.autograd.Variable(input, volatile=True)
target_var = torch.autograd.Variable(target, volatile=True)

#want to print the input varialbe’s imagename and classes here,how?

f2=model(x)

i want to print the input varialbe’s imagename and classes before code model(x)
many thanks!

1 Like
2

the source code of ImageFolder is simple and easy to understand, I would strongly like to advice you to have a look at it .
here is what you want:

class MyImageFolder(ImageFolder):
    def __getitem__(self, index):
        return super(MyImageFolder, self).__getitem__(index), self.imgs[index]#return image path

val_dataloader=t.utils.data.DataLoader(val_dataset)
for ii,data in enumerate(val_dataloader):
        (input,label),(path,_) = data
         .....
2 Likes
3

hi ,chen,thank u so much for your reply,i try ur code ,but it seems not work,my code is this:

valdir=r'test1/'
class MyImageFolder(datasets.ImageFolder):
    def __getitem__(self, index):
        return super(MyImageFolder, self).__getitem__(index), self.imgs[index]#return image path
        #return super(datasets.ImageFolder, self).__getitem__(index), self.imgs[index]#return image path
gg=MyImageFolder(valdir, transforms.Compose([
            transforms.Scale(256),
            transforms.CenterCrop(224),
            transforms.ToTensor(),
            normalize,
        ]))

normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406],
                                     std=[0.229, 0.224, 0.225])

val_loader = torch.utils.data.DataLoader(
        MyImageFolder(valdir, transforms.Compose([
            transforms.Scale(256),
            transforms.CenterCrop(224),
            transforms.ToTensor(),
            normalize,
        ])),
      batch_size=1, shuffle=False,
        num_workers=4, pin_memory=True)
    
for ii,py in enumerate(val_loader):    
    print(py)
    break
#for ii, (py,ik) enumerate(val_loader):

an error is something like this ,what is the reason?

        RuntimeError                              Traceback (most recent call last)
        <ipython-input-32-fb3a2b3714cd> in <module>()
             24         num_workers=4, pin_memory=True)
             25 
        ---> 26 for ii,py in enumerate(val_loader):
             27     print(py)
             28     break

    /usr/local/lib/python2.7/dist-packages/torch/utils/data/dataloader.pyc in __next__(self)
        172                 self.reorder_dict[idx] = batch
        173                 continue
    --> 174             return self._process_next_batch(batch)
        175 
        176     next = __next__  # Python 2 compatibility

/usr/local/lib/python2.7/dist-packages/torch/utils/data/dataloader.pyc in _process_next_batch(self, batch)
    196         self._put_indices()

…

return [pin_memory_batch(sample) for sample in batch]
  File "/usr/local/lib/python2.7/dist-packages/torch/utils/data/dataloader.py", line 91, in pin_memory_batch
return [pin_memory_batch(sample) for sample in batch]
  File "/usr/local/lib/python2.7/dist-packages/torch/utils/data/dataloader.py", line 90, in pin_memory_batch
elif isinstance(batch, collections.Iterable):
  File "/usr/lib/python2.7/abc.py", line 132, in __instancecheck__
if subclass is not None and subclass in cls._abc_cache:
  File "/usr/lib/python2.7/_weakrefset.py", line 75, in __contains__
return wr in self.data
RuntimeError: maximum recursion depth exceeded in cmp
4

seems something to do with the pin_memory ,what do this argument mean ?

val_loader = torch.utils.data.DataLoader(
        MyImageFolder(valdir, transforms.Compose([
            transforms.Scale(256),
            transforms.CenterCrop(224),
            transforms.ToTensor(),
            normalize,
        ])),
      batch_size=1, shuffle=False,
        num_workers=4, pin_memory=False)
        #num_workers=4, pin_memory=True)

this works

5

what should i have to do if i want to make pin_memory=True,i am new bb to python,the source code seem to be so complex

6

pin_memory Copies the tensor to pinned memory, if it’s not already pinned.This will make tensor transfers faster to GPU.
The reason it failed may because the paths I return is string not tensor. But this has been fixed. So you may update PyTorch and try again.