Why the output of sigmoid always converge to 0.5?

I have a network likes

input (64x16x16)---> Global Average Pooling (1)--> Conv (1x1)--> ReLU--> Sigmoid()-->Output

During training, I saw that the output converge to 0.5, it means the input of Sigmoid function become zero. What is happening in my network? How should I solve it? Thanks all

1 Like

There really isn’t enough info in your post to give an extensive answer. But generally, have you tried removing the sigmoid function and have you made sure that your loss function accepts inputs between 0 and 1? Many of the loss functions require log probabilities. Hopefully this helps but if not, you will probably have better luck if you post an example input and your code.

Same problem as you… How did you solve it??

Thanks in advance!