Why the torch.nn.functional.log_softmax returns the same data?

I trained a model with two class,and it works well in my dataload, but when I want to test it with an arbitrary data, the pred is wrong, and I find that the two score of log_softmax are always the same. I’m confused with this data.

my net is here :

class Mnet(nn.Module):
 def __init__(self, k = 2):
        super(Mnet, self).__init__()
        self.feat = Nnet()
        self.fc1 = nn.Linear(1024, 512)
        self.fc2 = nn.Linear(512, 256)
        self.fc3 = nn.Linear(256, k)
 def forward(self, x):
        x, trans = self.feat(x)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        fc3 = x
        print('f3: ',fc3)
        print('F_log: ',F.log_softmax(x, dim=0))
        return F.log_softmax(x, dim=0), trans,fc3

and there is my data with my test loader:

f3:  tensor([[-0.2002,  0.2312],
        [-0.5871,  2.8717],
        [-0.5332,  2.4883],
        [-0.1908,  0.1144],
        [-0.4099,  0.8726],
        [-0.4612,  1.9445],
        [-0.2196,  0.1526],
        [-0.5849,  2.8590],
        [-0.5060,  2.2855]], device='cuda:0', grad_fn=<ThAddmmBackward>)
F_log:  tensor([[-1.9994, -4.0161],
        [-2.3863, -1.3755],
        [-2.3324, -1.7589],
        [-1.9900, -4.1329],
        [-2.2091, -3.3746],
        [-2.2603, -2.3027],
        [-2.0188, -4.0947],
        [-2.3840, -1.3883],
        [-2.3052, -1.9617]], device='cuda:0', grad_fn=<LogSoftmaxBackward>)

the batch_size is 10
but when I test one data, thefc3 and F_log are:

f3:  tensor([[-0.0625, -0.1528]], device='cuda:0', grad_fn=<ThAddmmBackward>)
F_log:  tensor([[-0.6931, -0.6931]], device='cuda:0', grad_fn=<LogSoftmaxBackward>)

why the output of F.log_softmax is the same?

but if I use the fc3 to compute log_softmax, the scores are different:

newpred = F.log_softmax(fc3)

the output:

newpred: tensor([[-1.4822, -0.2576]], device=‘cuda:0’, grad_fn=<LogSoftmaxBackward>)

just without the dim=0, softmax can have different scores although the score is wrong, what results in it? thanks for your help!

Usually your data will have the batch dimension in dim0, so using F.log_softmax in dim0, will normalize your log probabilities in this batch dimension, which is most likely wrong.
Your logits are probably in dim1, so F.log_softmax(tensor, dim=1) should give you the right results.

3 Likes

It works! thank you so much!