how do i force my MLP model to predict values between 0 and 1(I want to do this because my target value is a continuous value between 0 and 1)

NB: This is a regression problem and my loss is MSE

how do i force my MLP model to predict values between 0 and 1(I want to do this because my target value is a continuous value between 0 and 1)

NB: This is a regression problem and my loss is MSE

Hi Ays and Alpha!

An alternative approach that may or may not train more effectively with a

mean-squared-error loss would be to use the output of your model (that I

assume ranges from `-inf`

to `inf`

) unchanged â€“ that is, not pass is through

a `Sigmoid`

â€“ but pass your target value through `torch.logit()`

, which will

transform your targetâ€™s range from `[0.0, 1.0]`

to `[-inf, inf]`

.

`logit()`

is the inverse function of `sigmoid()`

. So, in a sense, these two

schemes are comparing the same quantities, but doing so with a transformed

loss function.

(Note that if your target value represents a probability, then treating this

as a regression problem with a mean-squared-error loss, while logically

consistent, will probably not train as well as treating it as a classification

problem with binary cross entropy as the loss.)

Best.

K. Frank

1 Like

Thanks @KFrank and @AlphaBetaGamma96 for the reply. Isnâ€™t it just for classification task that I pass it through a sigmoid (or it doesnt matter and I can use it in this case too)? Also my target ranges between 0 and 1 (because Iâ€™m trying to predict a dice score which ranges between 0 and 1), and there is no class so itâ€™s a continues value.

Probably a naive opinion: I wouldnâ€™t try to enforce anything.

You clearly say itâ€™s a regression task. This also means, I assume, that the ground truth values in your training dataset are between 0 and 1. So if the model is trained well, and unseen data samples stems from a similar distribution as the training data, the prediction will be within 0 and 1.

In contrast, if the prediction is indeed outside this range, this might mean that the unseen sample â€ślooksâ€ť very different from the training data. And in this case forcing it to be within 0 and 1 doesnâ€™t feel right. Of course, in practice, you can always map negative values to 0 and larger values to 1. But enforcing the boundaries during training seems not a good idea.

After all, this is the common situation that a trained regressor is only valid within the range of the training data. For example, if you train a regressor predicting the resale price of houses based on #rooms, #floors, age, area_size, etc., you can predict the prices for hypothetical houses with 0 rooms or a million rooms. The regressor will give you some prediction, but itâ€™s arguably not very meaningful as this unseen sample is so different compared to the training data. In short, while you can extrapolate with the regressor, the more you extrapolate the less meaningful the prediction might become.

Despite this, Iâ€™ve never seen approaches to enforce any lower and upper bound during training. That being said, you could try tree-based models (Decision Trees, Random Forests, Gradient-Boosted Trees, etc.) as those models cannot extrapolate (compared to, say, Logistic Regression). So with tree-based models, your prediction will always be between the lowest and highest value in your training data.

1 Like

That makes sense. Thanks!