HI, I am working with 3D point cloud data, and I want to predict point wise value. Or point-wise regression task.
I normalized the output of all the data to be within range [0,1]. Now how can I restrict my last layer to only output values within this range. In general, I heard it is not a good idea to use any activation function (e.g., Sigmoid) in the last layer during regression task. I am wondering is there any better idea?
Is it a good idea to unnormalize within the model in my case? Currently I am unnormalizing after prediction using model.
I would not recommend restricting the output of your model – just let your
model learn that predictions outside of the range [0, 1] are poor predictions.
Let’s say your (normalized) ground-truth target is 0.98 and your model
predicts 1.02. Your regression loss (typically something like MSELoss)
will penalize your prediction by a little bit. But that’s appropriate, because
the prediction is only a little be off. If the prediction were 2.34, it would be
significantly off and would be penalized appropriately.
Would anything break if your model made a prediction greater than one?
What would be worse: a prediction of 0.85 when the ground truth is 0.71
or a prediction of 1.02 when the ground truth is 0.98?
I wouldn’t. I would expect that you could do so in a logically-consistent
way, but I don’t see what benefit you would get.
There is often some benefit to normalizing your data (especially your
input data). But if the range of your ground-truth target data isn’t
outlandish, things might work fine leaving it (for simplicity’s sake)
unnormalized.