Softmax activation

1

hi im new to pytorch there is something that keeps my mind busy
i see these code for classifying mnist

class Classifier(nn.Module):

def __init__(self, input_featurs, h1, h2,  output_featurs):
    super().__init__()
    self.linear= nn.Linear(input_featurs, h1)
    self.linear1 = nn.Linear(h1, h2)
    self.linear2 = nn.Linear(h2, output_featurs)
      
def forward(self, x):
    x =F.relu(self.linear(x))
    x =F.relu(self.linear1(x))  
    x =self.linear2(x)
    return x

why we dont use softmax activation function for last layer
thank you

2

It could be that nn.CrossEntropyLoss was used as a loss function, which does combine F.log_softmax and nn.NLLLoss (https://pytorch.org/docs/stable/nn.html#crossentropyloss). So therefor you don’t need a activation function at the end!

Softmax could also be applied later on in the training loop, right before the loss calculation!

1 Like