More efficient face alignment using affine/similarity transform in pytorch

[rafale77]

I have been searching for a solution to do this more efficiently entirely with torch tensors but have not found one so I am posting here to see if some expertise could help.
I am running inference on a facial detection model which then needs an alignment to then be an input for recognition.
The flow goes from: cv2 image → torch → detection model ->torch landmarks → numpy(cpu) ->estimate transform (cv2 or skimage) (cpu) → execute transform on cv2 image (cpu) → torch for next model

All torch tensors are actions are done on GPU. Now I would like to avoid doing this back and forth between numpy and torch.

In order to do this I would need to get torch equivalent functions to the two cv2 functions below:

        tfm, _ = cv2.estimateAffinePartial2D(src_pts.cpu().numpy(), ref_pts)
        face_img = cv2.warpAffine(image, tfm, out_size)

Are there any such functions? I have found ways to convert the transform matrix to F.affine_grid which would still require to generate the transformation matrix in numpy:

Affine transformation matrix paramters conversion

Hi all, I want to rotate an image about a specific point. First I create the Transformation matrices for moving the center point to the origin, rotating and then moving back to the first point, then apply the transform using affine_grid and grid_sample functions. But the resulting image is not what it should be. Once I tested these parameters by applying them on the image using scipy and it works. import torch.nn.functional as F mat_move = torch.eye(3) mat_move[0,2] = -x_center*2/my_image.shap…

but would like to avoid this altogether by generating the similarity transformation directly from torch tensors and apply it to a tensor.