Hi, all!
I met a problem when using torch.nn.functioal.affine_grid(theta,size).
I’d like to know how to convert an affine transformation matrix described in scipy/skimage.transform/opencv into a right argument theta in torch.nn.functioal.affine_grid(theta,size)?
Now suppose we want to apply an affine transormation on an image
with shape=(H,W,3), where
What’s the right theta which should be used in torch.nn.functioal.affine_grid(theta,size) ?
Hi, I met same problem, Have you solved it?
Hi, i have solved it.Code is below.
import cv2
import torch.nn.functional as F
import skimage.transform as trans
import numpy as np
def convert_image_np(inp):
“”“Convert a Tensor to numpy image.”""
inp = inp.numpy().transpose((1, 2, 0))
inp = (inp*255).astype(np.uint8)
return inp
def param2theta(param, w, h):
param = np.linalg.inv(param)
theta = np.zeros([2,3])
theta[0,0] = param[0,0]
theta[0,1] = param[0,1]*h/w
theta[0,2] = param[0,2]*2/w + param[0,0] + param[0,1] - 1
theta[1,0] = param[1,0]*w/h
theta[1,1] = param[1,1]
theta[1,2] = param[1,2]*2/h + param[1,0] + param[1,1] - 1
return theta
tr = trans.estimate_transform(‘affine’, src=src, dst=dst)
M = tr.params[0:2,:]
img = cv2.warpAffine(image,M,(w,h))
theta = param2theta(tr.params, w, h)
image = Variable(image).unsqueeze(0).cuda()
theta = Variable(theta).unsqueeze(0).cuda()
grid = F.affine_grid(theta,image.size())
img_ = F.grid_sample(image,grid)
img_ = convert_image_np(img_.data.cpu().squeeze(0))
The visualizations of img and img_ should be same. Hope it can help you!
Thanks for your sharing.
theta[0,0] = param[0,0]
theta[0,1] = param[0,1]*h/w
theta[0,2] = param[0,2]*2/w + theta[0,0] + theta[0,1] - 1
theta[1,0] = param[1,0]*w/h
theta[1,1] = param[1,1]
theta[1,2] = param[1,2]*2/h + theta[1,0] + theta[1,1] - 1
I have solved this issue by the above code. Maybe your code only works under some non-trival cases.
I think it works under general case, not some non-trival cases.
My recomendation is using Kornia https://torchgeometry.readthedocs.io/en/latest/geometry.transform.html?highlight=warp_affine#kornia.geometry.transform.warp_affine
Example: https://torchgeometry.readthedocs.io/en/latest/warp_affine.html
I have a similar problem.
I already use ITK-snap to process an affine registration : 3D volume image A to B.
Then I get a matrix:
0.97464 0.0364447 0.00781475 1.95549
0.00375859 0.922498 0.0297265 -8.87417
-0.00727863 -0.0145295 0.961844 -1.10425
0 0 0 1
I think it’s incompatibal in F.affine_grid theta.
The shape of affine_grid shape is (N, 3, 4)
likes: R | T
[
[a, b, c, t0],
[e, f, g, t1],
[h, i, j, t2]
]
I think the anterior three rows of the matrix is theta, but it’s wrong.
the R is likely equal to theta[:3, :3],(I am not sure)
but the T (t0, t1, t2) must be incompatible.
How can I get a right theta?
By the way, how can I know what does the torch.nn.functioal.affine_grid really do ?
It seems that the current PyTorch API doesn’t support 3D affine transformation. Up to now, affine_grid() and grid_sample() can only support 2D affine transformation (especially, 2D perspective transformation is not supported yet).
As the advices from @HectorAnadon, to implement complicated geometric transformations, you can try Kornia.
Hi @ljqcava
Thanks for your advice!
It seems that the 3d affine_grid is available in torch 1.5.1 but not in 1.2.0.
@oldriver-wang Oh, you are right. affine_grid() begin to support 3D transformation. It’s good news and glad to see that! However, it doesn’t support perspective transformation.