I am creating my own custom loss in pytorch, this loss is based on angular distance using the cosine similarity. The difference is that in my case I added a weight factor in order to magnify the error in some cases. The thing is, the network I am trying to train is not working as expected. For this reason I would like to know if the implementation that I made in pytorch is correct. I am using pytorch 0.4.1, I read some topics on this forum related on the custom losses with old versions of pytorch. Far as I saw one of the common error is the unpack of the Variables during the calculation , but in pytorch 0.4.1 this does not matter, right? I think my implementation is correct but maybe I miss some details of how this implementation needs to be done in pytorch. I would like to be sure that the error is not something related to my understanding of the implementation of losses in pytorch. Thanks in advance.
def weighted_angular_MSE(input,target,weight) input = torch.mul(F.normalize(input,p=2,dim=1),0.99999) target = torch.mul(F.normalize(target,p=2,dim=1),0.99999) error =torch.div(torch.acos(torch.sum(torch.mul(input,target),dim=1,keepdim=True)),math.pi) return torch.div(torch.sum(torch.mul(torch.pow(error,2)[0,:],weight)), torch.nonzero(weight.data).size(0)) class WeightedAngularMSE(nn.Module): def __init__(self): super(WeightedAngularMSE,self).__init__() def forward(self, input, target, weight): return weighted_angular_MSE(input,target,weight)