RuntimeError: Expected object of scalar type Long but got scalar type Float when using CrossEntropyLoss

I have a NN that ends with the following linear layers

dense = nn.Linear(input_size, 1)

if I use CrossEntropyLoss as loss function (as I’m y is supposed to be the class number) I get the following error

RuntimeError                              Traceback (most recent call last)
<ipython-input-39-72a754e03ca3> in <module>()
      1 lr = 2e-2
      2 learner = SimpleLearner([train_dl, test_dl], model, loss_func)
----> 3 history = learner.fit(10)

<ipython-input-37-121ec7440a76> in fit(self, epochs, lr)
     26             losses = []
     27             for x,y in self.data[0]:
---> 28                 losses.append(self.update(x, y , lr))
     29             history['losses'].append(np.mean(losses))
     30         return history

<ipython-input-37-121ec7440a76> in update(self, x, y, lr)
     10         for p in model.parameters(): w2 += (p**2).sum()
     11         # add to regular loss
---> 12         loss = loss_func(y_hat, y) + w2 * self.wd
     13         loss.backward()
     14         with torch.no_grad():

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    477             result = self._slow_forward(*input, **kwargs)
    478         else:
--> 479             result = self.forward(*input, **kwargs)
    480         for hook in self._forward_hooks.values():
    481             hook_result = hook(self, input, result)

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/loss.py in forward(self, input, target)
    865     def forward(self, input, target):
    866         return F.cross_entropy(input, target, weight=self.weight,
--> 867                                ignore_index=self.ignore_index, reduction=self.reduction)
    868 
    869 

/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction)
   1778     if size_average is not None or reduce is not None:
   1779         reduction = _Reduction.legacy_get_string(size_average, reduce)
-> 1780     return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
   1781 
   1782 

/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in nll_loss(input, target, weight, size_average, ignore_index, reduce, reduction)
   1623                          .format(input.size(0), target.size(0)))
   1624     if dim == 2:
-> 1625         ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
   1626     elif dim == 4:
   1627         ret = torch._C._nn.nll_loss2d(input, target, weight, _Reduction.get_enum(reduction), ignore_index)

RuntimeError: Expected object of scalar type Long but got scalar type Float for argument #2 'target'

What should be the loss (similarly for the accuracy) function that I should be using? if CrossEntropyLoss is the good one that should I do with the output of the linear layer?

The output layer should have the number of classes as out_features.
Currently your output layer only returns one neuron, which corresponds to class0.
For a binary use case, this should work:

batch_size = 5
nb_classes = 2
in_features = 10

model = nn.Linear(in_features, nb_classes)
criterion = nn.CrossEntropyLoss()

x = torch.randn(batch_size, in_features)
target = torch.empty(batch_size, dtype=torch.long).random_(nb_classes)

output = model(x)
loss = criterion(output, target)
loss.backward()

However, this doesn’t seem to be the error you are seeing here.
As you can see in my example, target should be of type torch.long. Try to fix the shapes and call target = target.long() to transform the data type.

Alternatively, you could return just one neuron and use nn.BCEWithLogitsLoss as your criterion.
This would also work with a float target.

9 Likes

I figured the problem, I was creating the target tensor and passing float as dtype :disappointed_relieved:
the following fixed the issue.

y_tensor = torch.tensor(y_train, dtype=torch.long, device=device)
5 Likes

target should be of type torch.long .

Why does the type of target have to be torch.long ?
Thank you!

The target should be a LongTensor using nn.CrossEntropyLoss (or nn.NLLLoss), since it is used to index the output logit (or log probability) for the current target class as shown in this formula (note the indexing in x[class]).

2 Likes

I’ve specified the dtype to be torch.long, and still the error pops out, but not in nn.CrossEntropyLoss in the prediction line, here is the error:

RuntimeError Traceback (most recent call last)
in
----> 1 train(model,train_loader,valid_loader)

in train(model, train_dataloader, valid_dataloader, epochs, lr)
18 #print(len(target[0][‘label’]))
19 optimizer.zero_grad()
—> 20 output = model(data,target)
21 print(‘check’)
22 loss = criterion(output,target)

/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py in call(self, *input, **kwargs)
491 result = self._slow_forward(*input, **kwargs)
492 else:
–> 493 result = self.forward(*input, **kwargs)
494 for hook in self._forward_hooks.values():
495 hook_result = hook(self, input, result)

/opt/conda/lib/python3.6/site-packages/torchvision/models/detection/generalized_rcnn.py in forward(self, images, targets)
49 if isinstance(features, torch.Tensor):
50 features = OrderedDict([(0, features)])
—> 51 proposals, proposal_losses = self.rpn(images, features, targets)
52 detections, detector_losses = self.roi_heads(features, proposals, images.image_sizes, targets)
53 detections = self.transform.postprocess(detections, images.image_sizes, original_image_sizes)

/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py in call(self, *input, **kwargs)
491 result = self._slow_forward(*input, **kwargs)
492 else:
–> 493 result = self.forward(*input, **kwargs)
494 for hook in self._forward_hooks.values():
495 hook_result = hook(self, input, result)

/opt/conda/lib/python3.6/site-packages/torchvision/models/detection/rpn.py in forward(self, images, features, targets)
413 losses = {}
414 if self.training:
–> 415 labels, matched_gt_boxes = self.assign_targets_to_anchors(anchors, targets)
416 regression_targets = self.box_coder.encode(matched_gt_boxes, anchors)
417 loss_objectness, loss_rpn_box_reg = self.compute_loss(

/opt/conda/lib/python3.6/site-packages/torchvision/models/detection/rpn.py in assign_targets_to_anchors(self, anchors, targets)
272 for anchors_per_image, targets_per_image in zip(anchors, targets):
273 gt_boxes = targets_per_image[“boxes”]
–> 274 match_quality_matrix = self.box_similarity(gt_boxes, anchors_per_image)
275 matched_idxs = self.proposal_matcher(match_quality_matrix)
276 # get the targets corresponding GT for each proposal

/opt/conda/lib/python3.6/site-packages/torchvision/ops/boxes.py in box_iou(boxes1, boxes2)
130 area2 = box_area(boxes2)
131
–> 132 lt = torch.max(boxes1[:, None, :2], boxes2[:, :2]) # [N,M,2]
133 rb = torch.min(boxes1[:, None, 2:], boxes2[:, 2:]) # [N,M,2]
134

RuntimeError: Expected object of scalar type Long but got scalar type Float for argument #2 ‘other’

I’m not familiar with your code, but it seems boxes1 is passed as a LongTensor, while boxes2 is a FloatTensor. Could this be the case?
If so, you could call float() in the first or long() on the second argument, depending on your expected result.

thanks for your response, I was wondering why there is boxes 1 and boxes 2, my input is about an image with a lot of labels in it and each label have 4 values for it box, so actually each label have one box only, so what does boxes 2 is for, if you have any idea in general??

It is helpful for me