Cropping batches at the same position

mhusseinsh · September 22, 2018, 7:21pm

here it is

Using 221 55 10 10
Using 221 55 10 10
Using 243 87 10 10
Using 243 87 10 10
Using 89 94 10 10
Using 89 94 10 10
Using 145 83 10 10
Using 145 83 10 10
Using 195 113 10 10
Using 195 113 10 10
Using 78 148 10 10
Using 78 148 10 10
Using 171 70 10 10
Using 171 70 10 10
Using 125 57 10 10
Using 125 57 10 10
Using 111 225 10 10
Using 79 10 10 10
Using 206 55 10 10
Using 12 192 10 10
Using 158 132 10 10
Using 143 62 10 10
Using 23 71 10 10
Using 163 115 10 10
Using 111 225 10 10
Using 79 10 10 10
Using 12 192 10 10
Using 206 55 10 10
Using 158 132 10 10
Using 143 62 10 10
Using 163 115 10 10
Using 23 71 10 10
Using 174 39 10 10

mhusseinsh · September 22, 2018, 7:22pm

ok now it worked
but there are two things weird
why it worked with a small size and not 256

also, if you check the print outs, you can see that they are consistent and then something weird happens

mhusseinsh · September 22, 2018, 7:24pm

I tried with 255
look what happened

Using 1 0 255 255
Using 1 0 255 255
Using 1 0 255 255
Using 1 0 255 255
Using 0 0 255 255
Using 0 0 255 255
Using 0 1 255 255
Using 0 1 255 255
Using 1 1 255 255
Using 1 1 255 255
Using 1 1 255 255
Using 1 1 255 255
Using 0 1 255 255
Using 0 1 255 255
Using 0 0 255 255
Using 0 0 255 255
Using 0 0 255 255
Using 0 0 255 255
Using 0 1 255 255
Using 0 1 255 255
Using 1 0 255 255
Using 1 1 255 255
Using 1 1 255 255
Using 1 1 255 255
Using 0 0 255 255
Using 0 0 255 255
Using 0 1 255 255
Using 1 1 255 255
Using 0 1 255 255
Using 1 0 255 255
Using 1 1 255 255
Using 1 1 255 255
Using 0 0 255 255
Using 1 0 255 255
Using 0 0 255 255
Using 0 0 255 255
Using 0 0 255 255
Using 0 0 255 255
Using 0 1 255 255
Using 1 1 255 255
Using 0 0 255 255
Using 0 1 255 255
Using 0 0 255 255
Using 0 0 255 255
Using 0 0 255 255
Using 1 1 255 255
Using 1 0 255 255
Using 0 0 255 255
Using 0 0 255 255
Using 1 0 255 255

mhusseinsh · September 22, 2018, 7:25pm

as far as I understand and notice, I believe that the resizing happens before the cropping

ptrblck · September 22, 2018, 7:26pm

Exactly! Was going to post the same.
You are resizing the images to [256, 256] so a crop won’t do anything afterwards.
Change your resize shape or crop size.

mhusseinsh · September 22, 2018, 7:27pm

yes exactly, as i assumed
I removed the resize and it worked


Using 28 48 256 256
Using 28 48 256 256
Using 0 127 256 256
Using 0 127 256 256
Using 115 330 256 256
Using 115 330 256 256
Using 224 279 256 256
Using 224 279 256 256
Using 41 62 256 256
Using 41 62 256 256
Using 18 126 256 256
Using 18 126 256 256
Using 180 271 256 256
Using 180 271 256 256
Using 217 372 256 256
Using 217 372 256 256
Using 12 120 256 256
Using 15 58 256 256
Using 36 4 256 256
Using 40 44 256 256
Using 23 25 256 256
Using 7 143 256 256
Using 4 85 256 256
Using 18 57 256 256
Using 36 4 256 256
Using 40 44 256 256
Using 15 58 256 256
Using 18 57 256 256
Using 7 143 256 256
Using 12 120 256 256
Using 4 85 256 256
Using 23 25 256 256
Using 42 11 256 256
Using 134 16 256 256
Using 7 283 256 256
Using 41 335 256 256
Using 220 381 256 256
Using 129 228 256 256
Using 37 206 256 256
Using 60 177 256 256
Using 340 358 256 256
Using 7 283 256 256
Using 220 381 256 256
Using 41 335 256 256
Using 134 16 256 256
Using 129 228 256 256
Using 60 177 256 256
Using 340 358 256 256
Using 37 206 256 256

mhusseinsh · September 22, 2018, 7:28pm

but again here lies the problem, that the indices become inconsistent at some point

ptrblck · September 22, 2018, 7:30pm

This might be due to using multiple workers.

mhusseinsh · September 22, 2018, 7:31pm

and does this have a solution by any means ?

ptrblck · September 22, 2018, 7:34pm

Let me think about a solution and make sure I understand the issue properly.
You want a batch of tensors where two neighboring samples were cropped using the same indices.
Also your dataset might be shuffled, so that the order of the images does not matter.
Multiple workers should also be used.
Is that correct?

mhusseinsh · September 22, 2018, 7:37pm

concerning the shuffling where I already asked here, I solved it by doing like this


def shuffle_pairs(images):
	pairs = [[images[z+y] for y in range(2)] for z in range(len(images)-1)]
	random.shuffle(pairs)

	shuffled = []
	for x in pairs:
	  for y in x:
	    shuffled.append(y)

	return shuffled
class ImageFolder(data.Dataset):

    def __init__(self, root, transform=None, return_paths=False,
                 loader=default_loader):
        #imgs = sorted(make_dataset(root))
        # shuffle implicit pairs
        if "test" in root: 
            imgs = sorted(make_dataset(root))
        else:
            imgs = shuffle_pairs(sorted(make_dataset(root)))
        if len(imgs) == 0:
            raise(RuntimeError("Found 0 images in: " + root + "\n"
                               "Supported image extensions are: " +
                               ",".join(IMG_EXTENSIONS)))

        self.root = root
        self.imgs = imgs
        self.transform = transform
        self.return_paths = return_paths
        self.loader = loader

    def __getitem__(self, index):
        path = self.imgs[index]
        img = self.loader(path)
        if self.transform is not None:
            img.save('x_a1.png')
            img = self.transform(img)
            vutils.save_image(img, 'x_a2.png', nrow=1)
            #exit()
        if self.return_paths:
            return img, path
        else:
            return img

    def __len__(self):
        return len(self.imgs)

so i didn’t use any sampler or something related to a dataloader … I just shuffled the list after reading it from the folder
so this works fine for me now

what i want, is that for every 2 samples, they have the same cropping
so sample 1 and 2, will be cropped from the same position
sample 3 and 4
sample 5 and 6

and so on

ptrblck · September 22, 2018, 7:58pm

Was this solution working properly?
I have the feeling that the mixup of the crop sizes might be due to some racing conditions of the workers, which would also influence the order of your samples.

mhusseinsh · September 22, 2018, 8:04pm

without trying the cropping, i think i have no problem with this solution, it is working okay

ptrblck · September 23, 2018, 12:46pm

Could you check if with the indices please?
I.e. add your load logic to this crop approach and print or store the indices you are expecting.

mhusseinsh · September 24, 2018, 6:18am

I don’t see a way of getting the indices here before and after shuffling

import torch
import torchvision.transforms.functional as TF
from torchvision import transforms
import random

class MyRandomCrop(transforms.RandomCrop):
    def __init__(self, size, padding=0, pad_if_needed=False):
        super(MyRandomCrop, self).__init__(size, padding, pad_if_needed)
        self.counter = 0
        self.crop_indices = []
        
    def __call__(self, img):
        if self.padding > 0:
            img = TF.pad(img, self.padding)

        # pad the width if needed
        if self.pad_if_needed and img.size[0] < self.size[1]:
            img = TF.pad(img, (int((1 + self.size[1] - img.size[0]) / 2), 0))
        # pad the height if needed
        if self.pad_if_needed and img.size[1] < self.size[0]:
            img = TF.pad(img, (0, int((1 + self.size[0] - img.size[1]) / 2)))
        
        resample = self.counter % 2 == 0
        self.counter += 1
        #print(self.counter, resample)
        if resample:
            self.crop_indices = self.get_params(img, self.size)
        i, j, h, w = self.crop_indices
        print('Using {} {} {} {}'.format(i, j, h, w))

        return TF.crop(img, i, j, h, w)

def shuffle_pairs(images):
    pairs = [[images[z+y] for y in range(2)] for z in range(len(images)-1)]
    random.shuffle(pairs)

    shuffled = []
    for x in pairs:
      for y in x:
        shuffled.append(y)

    return shuffled

images = []
crop = MyRandomCrop((256, 256))
for i in range(10):
    img = transforms.ToPILImage()(torch.randn(3, 600, 600))
    images.append(img)


shuffled_images = shuffle_pairs(images)

for i in (shuffled_images):
    print(crop(i).size)

ptrblck · September 24, 2018, 11:32am

You could first create the indices, shuffle them, and then slice your images.
Passing both the indices and images to your Dataset you could return a tuple of image and index.

images = []
for i in range(10):
    img = transforms.ToPILImage()(torch.randn(3, 600, 600))
    images.append(img)
    
pairs_idx = [[z+y for y in range(2)] for z in range(len(images)-1)]
random.shuffle(pairs_idx)
pairs = [[images[a], images[b]] for a, b in pairs_idx]

Ripley · November 2, 2020, 5:32am

HI, Just nice I found this code useful on my case study that I am doing now. Just wondering will this code do the randomcrop on every second images? What if I want to randomcrop based on probability? How can I do that? I am also working on doing data augmentation on tensors rather than images itself. Thank you for your help!

ptrblck · November 2, 2020, 6:53am

The code is quite old by now but it seems that resample will be active for each second call.

You can sample the crop position, height and width with any method you like and don’t need to use self.get_params if that doesn’t fit your use case.

Ripley · November 2, 2020, 8:40am

I see! Thanks for your advice! It seem to work okay! btw I am still unfamiliar with pytorch forum but uh how do I open a topic when I want to ask smth? haha

ptrblck · November 2, 2020, 8:59am

You shouldn’t tag specific users, so just create a topic and let all users take a look at your question.