C++ loss functions

1

Hi,
I am porting the code from “Deep Learning with PyTorch” from python to C++ and learning the C++ frontend API at the same time.

I am facing a difficulty when porting this snippet:

loss = nn.CrossEntropyLoss()
input = Variable(torch.randn(3, 5), requires_grad=True) 
target = Variable(torch.LongTensor(3).random_(5)) 
output = loss(input, target)
output.backward()

I have written this:

  auto input2 = torch::randn({3, 5}, torch::requires_grad(true).dtype(torch::kLong));
  auto target2 = torch::tensor({3},torch::kLong).random_(5);
  auto output2 = torch::binary_cross_entropy(input2, target2);
  output2.backward();

It compiles, but I have a runtime exception which says:

terminate called after throwing an instance of 'at::Error'
  what():  normal_ is not implemented for type CPULongType (normal_ at /pytorch/build/aten/src/ATen/TypeDefault.cpp:1652)
frame #0: at::native::randn(at::ArrayRef<long>, at::Generator*, at::TensorOptions const&) + 0x44 (0x7f1ffc68a704 in /home/inglada/local/libtorch/lib/libcaffe2.so)
frame #1: at::native::randn(at::ArrayRef<long>, at::TensorOptions const&) + 0xe (0x7f1ffc68a7be in /home/inglada/local/libtorch/lib/libcaffe2.so)
frame #2: ./Chapter03/chapter03() [0x428c8a]
frame #3: torch::randn(at::ArrayRef<long>, at::TensorOptions const&) + 0x177 (0x42ade2 in ./Chapter03/chapter03)
frame #4: main + 0x20a (0x427494 in ./Chapter03/chapter03)
frame #5: __libc_start_main + 0xf1 (0x7f1ffb7342e1 in /lib/x86_64-linux-gnu/libc.so.6)
frame #6: _start + 0x2a (0x42687a in ./Chapter03/chapter03)

I understand that I can not create a normal random tensor with long ints, but if I do

auto input2 = torch::randn({3, 5}, torch::requires_grad(true));

then the exception appears when computing the cross entropy:

terminate called after throwing an instance of 'at::Error'
  what():  Expected object of scalar type Float but got scalar type Long for argument #2 'target' (checked_tensor_unwrap at /pytorch/aten/src/ATen/Utils.h:74)

What can I do to get the same behaviour as in python?

Thank you.

2

nn.BCELoss expects FloatTensors for both the input and output.
As you are using nn.CrossentropyLoss I think you should use torch::log_softmax and torch::nll_loss as your criterion.
I can’t find the CrossEntropyLoss on the C++ side.

3

Hi,
Thanks for your answer. Does that mean that I can’t port to C++ exactly this python code?

loss = nn.CrossEntropyLoss()
input = Variable(torch.randn(3, 5), requires_grad=True) 
target = Variable(torch.LongTensor(3).random_(5)) 
output = loss(input, target)
output.backward()
4

I’m not sure. @goldsborough might have a better answer here.
Let’s wait for his wisdom. :wink:

5

We don’t have a cross_entropy_loss per se, but as @ptrblck mentioned, nn.CrossEntropyLoss()(input, target) is the same as cross_entropy(input, target), which per https://github.com/pytorch/pytorch/blob/master/torch/nn/functional.py#L1671 is the same as nll_loss(log_softmax(input, 1), target), so your code in C++ would be something like

auto input = torch::randn({3, 5}, torch::requires_grad(true));
auto target = torch::empty(3, torch::kLong).random_(5);
auto output = torch::nll_loss(torch::log_softmax(input, /*dim=*/1), target)
output.backward()

Can you try that?

6

Hi,
It works now.
Thanks!