[DataLoader Problem] dataloader with data in different dimension

When working on object detection problem, there will be different number of object in one picture. For example, when writing my customize dataset, the ground-truth for object location([x, y, w, h]) will have different number in first dimension
First image has 10 object in image, location ground-truth=[10,4];
Second image has 34 object in image, location ground-truth=[34,4]).

This will cause the error in dataloader where torch.stack will encounter dimension mismatch problem.
Is there any solution for this?

You can either pad so that they all have the same dimension, or disable pytorch auto stacking: collate_fn = lambda i: i

Do u mean that I have to write my own collate_fn (disable auto stacking)?

use the one that I put there (or you can write your own too)

Thanks for ur help, lambda i:i really work!!!
Say the batch size = 8, by using this method I have to iterate these 8 data seperately and add the loss???
And also Is it be able to use lambda i:i with dataparallel?

I’m not sure how your loss function work (maybe you can split 4 objects into 4 items or something), I just manually combine the data into a batch when I do it (basically your own collate_fn function)

maybe somebody can answer that, I’ve never done dataparallel so I don’t know

How about the first question??
It makes no difference between the aforementioned(batch_size=8 and iterate separately and add loss) and batch_size=1
correct me if Ive got something wrong here, thanks :slight_smile: )