Custom loss function using Scipy operations

autograd
1 

from scipy.spatial.transform import Rotation as R

class MNCriterion(nn.Module):

    def __init__(self, t_loss_fn=nn.L1Loss(), q_loss_fn=nn.L1Loss(), sax=0.0, saq=0.0, srx=0, srq=0.0, learn_beta=False, learn_gamma=False):
        super(MapNetCriterion, self).__init__()
        self.t_loss_fn = t_loss_fn
        self.q_loss_fn = q_loss_fn
        self.sax = nn.Parameter(torch.Tensor([sax]), requires_grad=learn_beta)
        self.saq = nn.Parameter(torch.Tensor([saq]), requires_grad=learn_beta)
        self.srx = nn.Parameter(torch.Tensor([srx]), requires_grad=learn_gamma)
        self.srq = nn.Parameter(torch.Tensor([srq]), requires_grad=learn_gamma)
        self.register_buffer("conj", torch.tensor([1,-1,-1,-1]))

    def pose_loss(self, pose_a, pose_b):
        quat_a = pose_a[3:]
        quat_b = pose_b[3:]
        r = R.from_quat(quat_a) 
        q_mat = r.as_matrix().T
        q_conj = torch.mul(quat_a, self.conj)
        sub = pose_b[:3] - pose_a[:3]
        delta_position = torch.matmul(q_mat, sub.T)
        delta_quat = self.hamilton_product(q_conj, quat_b)
        return np.concatenate((delta_position, delta_quat)).unsqueeze(0)

    def forward(self, pred, targ):
        p_loss =  self.pose_loss(pred, targ)
        ......

I get an error

r = R.from_quat(quat_a))
TypeError: can’t convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.

Since Scipy expects the numpy array, how do I deal with this?

1 Like
2

Have you tried the suggestion from the error, ie using Tensor.cpu()? In your case, assuming quat_a is a Tensor, you could just add quat_a.cpu() before the line which you get your error on.

3

I tried that.
That would be a problem if I convert them to cpu tensors and then to numpy arrays, since gradients willl not be computed respect to them and I get an error.

4

Going from tensors to numpy arrays and viceversa is not recommended, especially when you wan to compute gradients over this tensors.
Have you tried to work only on tensors?
Maybe you can code that Rotation using pytorch functions, or maybe there are some libraries written in pytorch you could use.
For instance, I found this method from PyTorch3D library: pytorch3d.transforms

2 Likes
5

Implementing the Rotation using pytorch helps.

Thanks a lot :slight_smile: