How to make targets go through the same augmentations as the images?

vision
1

Hi all,

I’ve been trying the instance segmentation tutorial and I have a question about augmentations. Basically, I want to perform augmentations to the images, but the targets (boxes, masks) also need to suffer the same augments. If I do them individually everything comes out wrong, because of the randomness of these operations.

The tutorial has some helper functions that, from what I understand, are not part of the main torchvision module. They do what I want, but only some very specific augmentations are provided. Horizontal flip is provided but not vertical, for example.

Is there any way or any library that does this? I’ve looked at albumentations but it doesn’t support masks for instance segmentation (it expects a single segmentation mask).

I’d want something like in the tutorial, that does everything in a single line, like:

        target = {}
        target["boxes"] = boxes
        target["labels"] = labels
        target["masks"] = masks
        target["image_id"] = image_id
        target["area"] = area
        target["iscrowd"] = iscrowd

        if self.transforms is not None:
            img, target = self.transforms(img, target)
2

One possible approach would be to use the functional transforms API and sample the “random” parameters once and apply them to the input image as well as all targets.
This post gives you an examples and you could also use the reference implementation from here to add more transformations.