Hi, all!
I met a problem when using torch.nn.functioal.affine_grid(theta,size).
I’d like to know how to convert an affine transformation matrix described in scipy/skimage.transform/opencv into a right argument theta in torch.nn.functioal.affine_grid(theta,size)?

Now suppose we want to apply an affine transormation on an image with shape=(H,W,3), where

What’s the right theta which should be used in torch.nn.functioal.affine_grid(theta,size) ?

I think it’s incompatibal in F.affine_grid theta.
The shape of affine_grid shape is (N, 3, 4)
likes: R | T
[
[a, b, c, t0],
[e, f, g, t1],
[h, i, j, t2]
]
I think the anterior three rows of the matrix is theta, but it’s wrong.
the R is likely equal to theta[:3, :3],(I am not sure)
but the T (t0, t1, t2) must be incompatible.

How can I get a right theta?
By the way, how can I know what does the torch.nn.functioal.affine_grid really do ?

It seems that the current PyTorch API doesn’t support 3D affine transformation. Up to now, affine_grid() and grid_sample() can only support 2D affine transformation (especially, 2D perspective transformation is not supported yet).

As the advices from @HectorAnadon, to implement complicated geometric transformations, you can try Kornia.

@oldriver-wang Oh, you are right. affine_grid() begin to support 3D transformation. It’s good news and glad to see that! However, it doesn’t support perspective transformation.