Zero division error when I build my own loss function

I want to build my own loss function, which is combine with cross-entropy and modified L2_norm loss.

criterion = nn.CrossEntropyLoss()
loss = criterion(outputs, labels)+verification_loss(fc, labels, parameters)

The verification_loss function is defined as followed:

from torch.autograd import Variable
parameters = torch.rand(1)
parameters = Variable(parameters, requires_grad=True)

def verification_loss(features, labels, parameters):`
    features = features
    labels = labels
    parameters = parameters
    loss = 0.0
    for i in range(0,
        for j in range(i,
            fi = features[i,:]
            fj = features[j,:]
            if[i] ==[j]:
                L2_norm = 0.5*torch.norm(fi-fj)**2
                loss += L2_norm
                L2_norm = torch.norm(fi-fj)
                diff = parameters - L2_norm
                if diff > 0:
                    loss += 0.5*(diff**2)
    return loss

The parameters is trainable parameters in this loss function, features is extracted from the internal layers, labels is the labels of the training samples.

   ZeroDivisionError                         Traceback (most recent call last)
<ipython-input-156-55ecba691a76> in <module>()
     15         outputs,fc = net(inputs)
     16         loss = criterion(outputs, labels)+verification_loss(fc, labels)
---> 17         loss.backward()
     18         optimizer.step()

//anaconda/lib/python2.7/site-packages/torch/autograd/variable.pyc in backward(self, gradient, retain_variables)
    144                     'or with gradient w.r.t. the variable')
    145             gradient =
--> 146         self._execution_engine.run_backward((self,), (gradient,), retain_variables)
    148     def register_hook(self, hook):

//anaconda/lib/python2.7/site-packages/torch/autograd/_functions/reduce.pyc in backward(self, grad_output)
    198             input, = self.saved_tensors
    199             if self.norm_type == 2:
--> 200                 return input.mul(grad_output[0] / self.norm)
    201             else:
    202                 pow = input.abs().pow(self.norm_type - 2)

ZeroDivisionError: float division by zero

how should I solve this problem, help~~

1 Like


The square root implicit in the 2-norm isn’t differentiable at 0, that may upset the backprop when the argument is numerically a zero norm vector. It may be a good idea to sum the squared differences yourself instead of usin norm (…)**2, that is mathematically equivalent and certain to be differentiable.

Best regards



Many thanks. It works but the parameters can’t be updated during backward processing. Maybe there must be something wrong. Actually, I want to write a contrasive loss function. The parameter is margin. Do you have any idea about that?