# VGG output layer - no softmax? **URL:** \ **Category:** vision\ **Created:** [October 30, 2017, 8:38pm UTC](https://discuss.pytorch.org/t/vgg-output-layer-no-softmax/9273 "2017-10-30T20:38:21Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![gaurub](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/gaurub/32/2069_2.png) [@gaurub](https://discuss.pytorch.org/u/gaurub)\ **Post date:** [October 30, 2017, 8:38pm UTC](https://discuss.pytorch.org/t/vgg-output-layer-no-softmax/9273/1 "2017-10-30T20:38:21Z")
This may seem like an extremely stupid question, but I was curious about something: 1. In other implementations of VGG, the last layer is always put through softmax; however, in the torchvision implementation [here](https://github.com/pytorch/vision/blob/master/torchvision/models/vgg.py), the last layer is the following: nn.Linear(4096, num\_classes), There is no softmax layer after this and the VGG documentation [https://gist.githubusercontent.com/ksimonyan/211839e770f7b538e2d8/raw/0067c9b32f60362c74f4c445a080beed06b07eb3/VGG\_ILSVRC\_16\_layers\_deploy.prototxt](http://here) also states that the last layer “prob” is a softmax layer. I was wondering how this was possible and why this works. I realize this might be a more general math question, but I couldn’t think of anywhere else to post it, so I posted here. Thanks! ---
**Author:** ![SimonW](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/simonw/32/1702_2.png) [@SimonW](https://discuss.pytorch.org/u/SimonW)\ **Post date:** [October 30, 2017, 8:45pm UTC](https://discuss.pytorch.org/t/vgg-output-layer-no-softmax/9273/2 "2017-10-30T20:45:58Z")
I didn’t double check, but it feels to me that it is trained with softmax. It might is just removed in the model zoo because the 4096 vector is more interesting in most use cases of a pretrained model. I still can be totally wrong though. 🙂 ---
**Author:** ![gaurub](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/gaurub/32/2069_2.png) [@gaurub](https://discuss.pytorch.org/u/gaurub)\ **Post date:** [October 30, 2017, 8:50pm UTC](https://discuss.pytorch.org/t/vgg-output-layer-no-softmax/9273/3 "2017-10-30T20:50:42Z")
Hi, Thanks for your reply. I figured that it’s trained with softmax and that it’s probably mostly used for transfer learning, so the last layer(s) will be thrown away anyway. However, what confused me is it’s total absence. If what you say is true and it’s been left off intentionally with the knowledge that it will mostly be used for transfer learning, then that does, at the very least, tell me that my understanding is correct and that I can stop thinking I’m an idiot :). ---
**Author:** ![SimonW](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/simonw/32/1702_2.png) [@SimonW](https://discuss.pytorch.org/u/SimonW)\ **Post date:** [October 30, 2017, 10:01pm UTC](https://discuss.pytorch.org/t/vgg-output-layer-no-softmax/9273/4 "2017-10-30T22:01:36Z")
Upon digging into the details, it appears that the models are trained with `CrossEntropyLoss` which has `SoftMax` built in. Imagenet training script: Criterion defined at: [https://github.com/pytorch/examples/blob/master/imagenet/main.py#L91](https://github.com/pytorch/examples/blob/master/imagenet/main.py#L91) Loss defined at: [https://github.com/pytorch/examples/blob/master/imagenet/main.py#L195](https://github.com/pytorch/examples/blob/master/imagenet/main.py#L195) `CrossEntropyLoss` doc: [http://pytorch.org/docs/master/nn.html#torch.nn.CrossEntropyLoss](http://pytorch.org/docs/master/nn.html#torch.nn.CrossEntropyLoss) ---
**Author:** ![gaurub](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/gaurub/32/2069_2.png) [@gaurub](https://discuss.pytorch.org/u/gaurub)\ **Post date:** [October 30, 2017, 10:39pm UTC](https://discuss.pytorch.org/t/vgg-output-layer-no-softmax/9273/5 "2017-10-30T22:39:04Z")
Hi, Again, thank you very much for your response. I feel there’s still a gap in my understanding, however. I understand that they are trained with CrossEntropyLoss which includes Softmax, but I’m trying to understand the output from the network when a test image is passed through it. In that particular case, wouldn’t the output be different from VGG output, which has the softmax layer as part of the architecture? Yes, we will throw this away in the transfer learning use case, but as a hypothetical, say I wanted to just use VGG as a classifier; in that case, I wouldn’t get pseudo-probabilities that sum to 1, correct? If that is the case, is the main reason the softmax layer was left off merely because no one would be using it? And once more, thank you for the help in understanding this. Thanks! ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [October 30, 2017, 11:48pm UTC](https://discuss.pytorch.org/t/vgg-output-layer-no-softmax/9273/6 "2017-10-30T23:48:46Z")
In case you want to use the VGG network to classify new samples, you can just call `argmax` on the logits to get the most likely class. The softmax won’t change the classification. However, if you need the probabilities, you can always call `softmax` on the net’s output. ---
**Author:** ![gaurub](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/gaurub/32/2069_2.png) [@gaurub](https://discuss.pytorch.org/u/gaurub)\ **Post date:** [October 31, 2017, 1:34am UTC](https://discuss.pytorch.org/t/vgg-output-layer-no-softmax/9273/7 "2017-10-31T01:34:42Z")
Hi @ptrblck, Thank you very much! That clears up everything for me. ---
**Author:** ![SpandanMadan](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/spandanmadan/32/1371_2.png) [@SpandanMadan](https://discuss.pytorch.org/u/SpandanMadan)\ **Post date:** [October 31, 2017, 1:38am UTC](https://discuss.pytorch.org/t/vgg-output-layer-no-softmax/9273/8 "2017-10-31T01:38:16Z")
The reason why this is done is because you only need the softmax layer at the time of inferencing. While training, to calculate the loss you don’t need to softmax and just calculate loss without it. This way the number of computations get reduced! ---
**Author:** ![Santhosh\_Kumar](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/santhosh_kumar/32/23512_2.png) [@Santhosh\_Kumar](https://discuss.pytorch.org/u/Santhosh_Kumar)\ **Post date:** [April 29, 2020, 3:05pm UTC](https://discuss.pytorch.org/t/vgg-output-layer-no-softmax/9273/9 "2020-04-29T15:05:31Z")
I try to apply softmax on output it returns probability for the single record only instead of returning the probability for whole test data, it returns probability correctly for the single record torch.softmax(output[0], dim=0). but this one not working torch.softmax(output, dim=1) its returning some other value instead of probability @ptrblck ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [April 30, 2020, 4:08am UTC](https://discuss.pytorch.org/t/vgg-output-layer-no-softmax/9273/10 "2020-04-30T04:08:54Z")
If you output is returned as `[batch_size, nb_classes]` (which would be the default for a classification use case), then `softmax(output, dim=1)` is the right approach, since the sum in dim1 will be 1. Each row (which corresponds to a sample in the batch) will contain the probabilities for each class. ---
**Author:** ![Santhosh\_Kumar](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/santhosh_kumar/32/23512_2.png) [@Santhosh\_Kumar](https://discuss.pytorch.org/u/Santhosh_Kumar)\ **Post date:** [April 30, 2020, 5:24am UTC](https://discuss.pytorch.org/t/vgg-output-layer-no-softmax/9273/11 "2020-04-30T05:24:15Z")
Yeah understood, Thank u so much👍 @ptrblck