Why isn't RandomCrop inserting the padding in pytorch?

vision
1

I am getting odd images despite printing the transform the data is using:

-- splits[i]='train'
taskset=<learn2learn.data.task_dataset.TaskDataset object at 0x7fbc38345880>
taskset.dataset.dataset.datasets[0].dataset.transform=Compose(
    ToPILImage()
    RandomCrop(size=(84, 84), padding=8)
    ColorJitter(brightness=[0.6, 1.4], contrast=[0.6, 1.4], saturation=[0.6, 1.4], hue=None)
    RandomHorizontalFlip(p=0.5)
    ToTensor()
    Normalize(mean=[0.47214064400000005, 0.45330829125490196, 0.4099612805098039], std=[0.2771838538039216, 0.26775040952941176, 0.28449041290196075])
)

but the padding is missing:
enter image description here

but when I use this instead:

    train_data_transform = Compose([
        RandomResizedCrop((size - padding*2, size - padding*2), scale=scale, ratio=ratio),
        Pad(padding=padding),
        ColorJitter(brightness=0.4, contrast=0.4, saturation=0.4),
        RandomHorizontalFlip(),
        ToTensor(),
        Normalize(mean=mean, std=std),
    ])

it seems to work:
enter image description here
why don’t both have the 8 and 8 padding on both sides I expect?

cross:

2

I think this is a bug in pytorch: Padding for RandomCrop does not match description in documentation · Issue #89253 · pytorch/pytorch · GitHub

3

Answered on GitHib: it’s expected to see the padding in on corner/edge since the padding is applied before the tensor/image is cropped as seen here.

4

Docs are super confusing

5

I don’t think the docs are confusing and explain which operations are applied as described here:

6

as a side note @ptrblck do you think doing this transform makes sense?

    train_data_transform = Compose([
        RandomResizedCrop((size, size), scale=scale, ratio=ratio),
        RandomCrop(size=size, padding=padding),
        ColorJitter(brightness=0.4, contrast=0.4, saturation=0.4),
        RandomHorizontalFlip(),
        ToTensor(),
        Normalize(mean=mean, std=std),
    ])
7

It’s hard to tell if this transformation would work for your use case.
The RandomCrop transformation will basically add the padding border to the already cropped image (coming from RandomResizedCrop) and will then return a new crop in the same shape as the input from the padded image.
Without knowing about your use case and dataset, I would not expect to see a huge difference between this approach and a transformation which drops the RandomCrop transformation, but in any case you should try both and compare their performance.

8

Alternatively you can check Kornia that includes more functionality with examples, differentiability, serialise the sampled parameters and compute the inverse of the operation.

https://kornia.readthedocs.io/en/latest/augmentation.module.html#kornia.augmentation.RandomCrop