I want to make a simple regression network to estimate certain value.
The value should be in the range [0, 2pi], since it is a angle.
I tried to use sigmoid() at the end of the network, and multiply 2pi on it.
But backprop works awful at its head an tail, and gradients are ruined.
Is there any proper way to restrict the prediction value of network, without harm its optimization?
In my opinion the output range has to be specified in your training data and after that you can then check the accuracy of the target with respect to the given features.
Also your cost function plays a role in this too
Eg: their are cost functions that a suitable for regression models like MSEloss, RMSEloss, and basically other loss functions you’d normally use to estimate error back in high school when dealing with continuous data.
I also suggest that in this scenario instead of using sigmoid function, use softmax in your model output layer. If your training data target is set with the values ranging from (0 to 2) then I suggest that you also normalize them to be in the range of (0 to 1) too, so that after your model is being trained and makes a prediction you can then apply a reverse of the training target normalization on that prediction, after that u can approximate the values to the nearest whatever u want and see how well it trained.
Hope this helped?
Perhaps keep the sigmoid output constraining to (0,1) but scale your target by 1/2.pi