How to print CrossEntropyLoss of data

I want to test nn.CrossEntropyLoss() is same as tf.nn.softmax_cross_entropy_with_logits in tensorflow.
so I have tested on tensorflow and pytorch. I got value with tensorflow, but I don`t know how to get value of pytorch.

Tensorflow test :

    sess = tf.Session()
    y_true = tf.convert_to_tensor(np.array([[0.0, 1.0, 0.0], [0.0, 0.0, 1.0]]))
    y_hat = tf.convert_to_tensor(np.array([[0.5, 1.5, 0.1], [2.2, 1.3, 1.7]]))

    # print(y_true)
    tf_loss = tf.nn.softmax_cross_entropy_with_logits(labels=y_hat, logits=y_true)
    print(sess.run(tf_loss))
   -> [1.7580339  6.36751251]

Pytorch test :

    loss = nn.CrossEntropyLoss()
    input = torch.Tensor(np.array([[0.0, 1.0, 0.0], [0.0, 0.0, 1.0]]))
    target = torch.Tensor(np.array([[0.0, 1.0, 0.0], [0.0, 0.0, 1.0]]), dtype=torch.long)
    print(type(input), type(target))
    output = loss(input, target)
    output.backward()
    print(output)
It occurs error :weary:
>        target = torch.Tensor(np.array([[0.0, 1.0, 0.0], [0.0, 0.0, 1.0]]), dtype=torch.long)
> TypeError: new() received an invalid combination of arguments - got (numpy.ndarray, dtype=torch.dtype), but expected one of:
>  * (torch.device device)
>  * (torch.Storage storage)
>  * (Tensor other)
>  * (tuple of ints size, torch.device device)
>       didn't match because some of the keywords were incorrect: dtype
>  * (object data, torch.device device)
>       didn't match because some of the keywords were incorrect: dtype

Anyone can help me?

I’ve fixed your PyTorch code as there were some minor issues:

loss = nn.CrossEntropyLoss(reduction='none')
input = torch.tensor([[0.5, 1.5, 0.1], [2.2, 1.3, 1.7]], requires_grad=True)
target = torch.tensor([1, 2], dtype=torch.long)
print(type(input), type(target))
output = loss(input, target)
output.backward(torch.ones_like(target, dtype=torch.float))
print(output)

The first one was that nn.CrossEntropyLoss does not use one-hot encoded targets, but targets storing the class index. To get the loss for both elements, we also have to pass reduction='none' as otherwise the default will be elementwise_mean.

If you want to cast a numpy array in to a tensor, you should use torch.from_numpy instead if directly creating it.

In your tensorflow code it looks like you’ve mixed up labels and logits.

I exchanged y_true and y_hat in tensorflow code.
and I got same results on both tensorflow and pytorch codes.
Thanks !!!

I was trying to reproduce this test for an input with size torch.Size([100, 9, 9, 4]) and target with size torch.Size([100, 9, 9]), but got error.

RuntimeError: Assertion `input0 == target0 && input2 == target1 && input3 == target2’ failed.

I was wondering what is the best way to support batch size, etc. for computing the loss. In my attempt, I used input.view(-1,4) with target.view(-1,4).argmax(). Is there a better way to do it?

Could you post a code snippet to reproduce this issue?

    output = torch.rand(2,4,4,2,requires_grad=True)

    target = torch.tensor([[[[0, 1],
              [1, 0],
              [1, 0],
              [0, 1]],

             [[1, 0],
              [0, 1],
              [0, 1],
              [1, 0]],

             [[0, 1],
              [0, 1],
              [1, 0],
              [1, 0]],

             [[0, 1],
              [1, 0],
              [0, 1],
              [0, 1]]],

            [[[0, 1],
              [0, 1],
              [1, 0],
              [0, 1]],

             [[0, 1],
              [0, 1],
              [0, 1],
              [1, 0]],

             [[1, 0],
              [1, 0],
              [1, 0],
              [0, 1]],

             [[1, 0],
              [1, 0],
              [0, 1],
              [0, 1]]]])

    output = output.view(-1,4)
    target = target.view(-1,4).argmax(1)
    loss = torch.nn.CrossEntropyLoss(reduction='none')
    my_loss = loss(output, target).view(-1,4,4)

I didn’t know of a more graceful way to produce random label. Would have been more appealing to eye if I did.

Your code snippet doesn’t produce this error using 1.4.0.dev20191109.
Which PyTorch version are you using?