How to reverse spatial transformer operation

I am trying to use a spacial transformer network in a segmentation setting.
Therefore it would be very important to ‘undo’ the transformation, basically applying the inverse transformation.

If this is the transformation:

grid = F.affine_grid(theta, x.size())
y = F.grid_sample(x, grid)

what is the inverse to get back to x?
gridinverse = ?
x1 = F.grid_sample(y, gridinverse)

Any help would be greatly appreciated.
Thank you very much!
Veit

1 Like

I am thinking about how to achieve this operation too! Did you find some solutions?