Getting image name from tensor data after batch data loading

vision
1

To debug my code I need to get name of images in my batch with its labels and prediction. As images in batch are in form of tensor, So I can not acess image name.

In my dataloader below part of code is used-

def getitem(self, i):
data, label = self.data[i], self.label[i] #here data is actual file name of image
image = self.transform(Image.open(data).convert(‘RGB’)) # here its converted into tensor
return image, label #return image tensor and labels
Below part of code I am using to load data in batch-

for i, batch in tqdm(enumerate(self.test_loader, 1)):
if torch.cuda.is_available():
data, _ = [_.cuda() for _ in batch] data loaded in form of tensor
logits = self.model(data)
loss = F.cross_entropy(logits, label)

So what needs to be changed so that I can see the data not only in form of tensor but also as name of data(file name)?

2

You could return the name (data in your example) directly in the Dataset.__getitem__ method:

def __getitem___(self, i):
    data, label = self.data[i], self.label[i] #here data is actual file name of image
    image = self.transform(Image.open(data).convert(‘RGB’)) # here its converted into tensor
    return image, label, data
3

I already tried to do the same suggested by you but then I got below error-

data, _ = [_.cuda() for _ in batch]
AttributeError: 'list' object has no attribute 'cuda'

Above error disapperas when I don’t return 3 values from dataloader
test_loader is used as below-

testset = Dataset('test', args)
test_sampler = CategoriesSampler(testset.label, 1, # args.num_eval_episodes, 
args.eval_way, args.eval_shot + args.eval_query)
test_loader = DataLoader(dataset=testset, batch_sampler=test_sampler, num_workers=args.num_workers)

Do I have to adapt test_loader so that batch takes image name?

4

You would have to split the returned batch to the three objects (tensor, tensor, list), as you are currently treating them all as tensors:

data, _ = [_.cuda() for _ in batch]
5

you mean like -

for i, batch in tqdm(enumerate(self.test_loader, 1)):
if torch.cuda.is_available():
    file_name=[]
    data, _ , file_name= [_.cuda() for _ in batch]
6

No, this should work:

data, _, file_name = batch
data = data.cuda()

assuming batch contains 3 objects.

7

ValueError Traceback (most recent call last)
/tmp/ipykernel_689838/1233800762.py in
12 tl=test_loader
13 model_path= ‘best_model.pt’
—> 14 test_model(tl,model_path, save_mod=False)

/tmp/ipykernel_689838/2148491603.py in test_model(test_loader, model_path, save_mod)
270
271
→ 272 ids=get_ids(test_loader)
273 img_id=ids
274 im=transforms.ToPILImage()(t).convert(“RGB”)

/tmp/ipykernel_689838/2148491603.py in get_ids(test_loader)
164 torch.device(“cuda” if torch.cuda.is_available() else “cpu”)
165 ids=[]
→ 166 data,_,ids= batch
167 data=data.to(device)
168 ids=ids.to(device)

ValueError: not enough values to unpack (expected 3, got 2)

I got this Error when I tried . Can you help

This was my code
def getitem_(self, i):
data, label = self.data[i], self.label[i] #here data is actual file name of image
#image=self.transforms.ToPILImage()(data).convert(“RGB”)
image = self.transform(Image.open(data).convert(“RGB”)) # here its converted into tensor
return image, label, data
def get_ids(test_loader):
all_ids = torch.tensor([]).to(device)
for i, batch in enumerate(test_loader):
torch.device(“cuda” if torch.cuda.is_available() else “cpu”)

    ids=[]
    data,_,ids= batch
    data=data.to(device)
    ids=ids.to(device)
    ids = ids.float()
    all_ids = torch.cat((all_ids, ids)) 
return all_ids
8

Your __getitem__ method seems to return a single tensor (all_ids) while you are trying to unpack it into 3 return values. Could you explain your use case a bit more?

9

This is the first part
getitem(self, i):
data, label = self.data[i], self.label[i]
#image=self.transforms.ToPILImage()(data).convert(“RGB”)
image = self.transform(Image.open(data).convert(“RGB”))
return image, label, data

it first returns image, label and data
So i want to extract the filenames(ids) from the inputs (data) and that I tried
for i, batch in enumerate(test_loader):
torch.device(“cuda” if torch.cuda.is_available() else " cpu")
ids=[ ]
data, _, ids= batch
data = data.to (device)
ids= ids.to(device) # i want to get the ids here
return ids

10

Ah, I missed the return statement as your code isn’t formatted properly. You can add code snippets by wrapping them into three backticks ``` for easier debugging.
Your code should work, so feel free to post a minimal, executable code snippet reproducing the error.

1 Like