What should be the output of the Sigmoid layer

PyTorch Live
1

I have used the Sigmoid layer as the output layer for the discriminator of a GAN model. The discriminator is supposed to classify fake and real images. The output of the Sigmoid layer produces a tensor of shape

torch.Size([1, 1, 16, 16])

Here is an example of the output:

tensor([[[[0.9162, 0.9692, 0.9791, 0.9853, 0.9897, 0.9900, 0.9909, 0.9912,
           0.9914, 0.9888, 0.9846, 0.9822, 0.9871, 0.9891, 0.9783, 0.8980],
          [0.9801, 0.9961, 0.9977, 0.9984, 0.9989, 0.9991, 0.9992, 0.9992,
           0.9992, 0.9991, 0.9987, 0.9986, 0.9989, 0.9990, 0.9976, 0.9671],
          [0.9874, 0.9984, 0.9991, 0.9993, 0.9994, 0.9994, 0.9996, 0.9996,
           0.9995, 0.9994, 0.9994, 0.9994, 0.9994, 0.9995, 0.9988, 0.9797],
          [0.9878, 0.9987, 0.9993, 0.9994, 0.9993, 0.9994, 0.9995, 0.9995,
           0.9994, 0.9994, 0.9993, 0.9992, 0.9992, 0.9993, 0.9988, 0.9821],
          [0.9880, 0.9987, 0.9994, 0.9994, 0.9994, 0.9994, 0.9994, 0.9994,
           0.9994, 0.9993, 0.9990, 0.9988, 0.9988, 0.9991, 0.9986, 0.9830],
          [0.9885, 0.9988, 0.9994, 0.9994, 0.9994, 0.9994, 0.9993, 0.9993,
           0.9993, 0.9993, 0.9991, 0.9988, 0.9986, 0.9989, 0.9986, 0.9832],
          [0.9893, 0.9989, 0.9994, 0.9994, 0.9993, 0.9992, 0.9991, 0.9992,
           0.9993, 0.9992, 0.9993, 0.9990, 0.9990, 0.9991, 0.9986, 0.9835],
          [0.9892, 0.9990, 0.9994, 0.9993, 0.9991, 0.9990, 0.9990, 0.9991,
           0.9991, 0.9991, 0.9993, 0.9993, 0.9992, 0.9993, 0.9987, 0.9823],
          [0.9893, 0.9991, 0.9995, 0.9992, 0.9991, 0.9991, 0.9992, 0.9992,
           0.9989, 0.9989, 0.9991, 0.9992, 0.9993, 0.9992, 0.9982, 0.9788],
          [0.9902, 0.9993, 0.9996, 0.9994, 0.9990, 0.9991, 0.9994, 0.9995,
           0.9992, 0.9991, 0.9991, 0.9992, 0.9991, 0.9990, 0.9977, 0.9770],
          [0.9906, 0.9993, 0.9997, 0.9995, 0.9993, 0.9992, 0.9995, 0.9996,
           0.9996, 0.9994, 0.9994, 0.9994, 0.9994, 0.9993, 0.9982, 0.9789],
          [0.9903, 0.9993, 0.9997, 0.9996, 0.9994, 0.9993, 0.9995, 0.9997,
           0.9997, 0.9996, 0.9995, 0.9996, 0.9996, 0.9994, 0.9985, 0.9816],
          [0.9882, 0.9989, 0.9995, 0.9995, 0.9994, 0.9991, 0.9992, 0.9995,
           0.9996, 0.9996, 0.9996, 0.9996, 0.9996, 0.9996, 0.9990, 0.9846],
          [0.9877, 0.9986, 0.9992, 0.9992, 0.9991, 0.9989, 0.9990, 0.9992,
           0.9995, 0.9996, 0.9996, 0.9996, 0.9996, 0.9996, 0.9992, 0.9866],
          [0.9826, 0.9974, 0.9983, 0.9976, 0.9976, 0.9971, 0.9974, 0.9980,
           0.9989, 0.9992, 0.9992, 0.9991, 0.9992, 0.9991, 0.9982, 0.9778],
          [0.9283, 0.9793, 0.9815, 0.9768, 0.9749, 0.9744, 0.9762, 0.9795,
           0.9845, 0.9877, 0.9886, 0.9877, 0.9883, 0.9881, 0.9800, 0.9083]]]]

Shouldn’t the output be a single probability value ranging between 0 and 1 instead of a tensor?

2

nn.Sigmoid applies the sigmoid function to all inputs separately.

If you want to get one value out of your model, you could either down sample it once more, or use:

x=x.flatten(start_dim=2).mean(dim=2)
3

@J_Johnson I am a bit confused about the dim part. What is the significance of start_dim and dim here and why do you set them to 2 specifically?
Will it create any discrepancies with the actual values?

4

start_dim just tells the function what to flatten and what to leave in separate dimensions. For example, in this case you might have a size [16, 3, 32, 32] —> [16, 3, 32*32].

https://pytorch.org/docs/stable/generated/torch.flatten.html

dim in the .mean() function tells it what dimension to take the mean of. For example, we want the mean of each channel, vs. the mean of the entire batch.

https://pytorch.org/docs/stable/generated/torch.mean.html

Both functions together are flattening the output channels and getting the mean of each channel.