I want to build my own loss function, which is combine with cross-entropy and modified L2_norm loss.
criterion = nn.CrossEntropyLoss()
loss = criterion(outputs, labels)+verification_loss(fc, labels, parameters)
loss.backward()
The verification_loss function is defined as followed:
from torch.autograd import Variable
parameters = torch.rand(1)
parameters = Variable(parameters, requires_grad=True)
def verification_loss(features, labels, parameters):`
features = features
labels = labels
parameters = parameters
loss = 0.0
for i in range(0, labels.data.size(0)-1):
for j in range(i, labels.data.size(0)):
fi = features[i,:]
fj = features[j,:]
if labels.data[i] == labels.data[j]:
L2_norm = 0.5*torch.norm(fi-fj)**2
loss += L2_norm
else:
L2_norm = torch.norm(fi-fj)
diff = parameters - L2_norm
if diff > 0:
loss += 0.5*(diff**2)
return loss
The parameters is trainable parameters in this loss function, features is extracted from the internal layers, labels is the labels of the training samples.
ZeroDivisionError Traceback (most recent call last)
<ipython-input-156-55ecba691a76> in <module>()
15 outputs,fc = net(inputs)
16 loss = criterion(outputs, labels)+verification_loss(fc, labels)
---> 17 loss.backward()
18 optimizer.step()
19
//anaconda/lib/python2.7/site-packages/torch/autograd/variable.pyc in backward(self, gradient, retain_variables)
144 'or with gradient w.r.t. the variable')
145 gradient = self.data.new().resize_as_(self.data).fill_(1)
--> 146 self._execution_engine.run_backward((self,), (gradient,), retain_variables)
147
148 def register_hook(self, hook):
//anaconda/lib/python2.7/site-packages/torch/autograd/_functions/reduce.pyc in backward(self, grad_output)
198 input, = self.saved_tensors
199 if self.norm_type == 2:
--> 200 return input.mul(grad_output[0] / self.norm)
201 else:
202 pow = input.abs().pow(self.norm_type - 2)
ZeroDivisionError: float division by zero
how should I solve this problem, help~~