Is that possible to predict both classification and regression in single model?
Suppose that we have target shape like [b1, b2, c1, c2, c3]
which bs are for regression target and cs are for classification target and they are from one-hot encoding.
Hi Mahdi!
Yes, and it can be a reasonable thing to do.
I recommend against using one-hot encoding. There’s no reason to do
so and it introduces at least a little inefficiency.
(I assume that you mean that you have three classes and that exactly
one of c1
, c2
, and c3
is one, with the other two being zero.)
Instead, let your classification target
be a single integer class label that
ranges from 0 to 2.
The simplest approach will be to have your final layer be a Linear
with
out_features = 5
. The first two outputs will be understood as the
continuous-valued predictions for b1
and b2
. The remaining three will
be understood as raw-score logits for the three classes. Then, e.g.:
loss_regression = torch.nn.MSELoss() (output[:, 0:2], reg_target)
loss_classification = torch.nn.CrossEntropyLoss() (output[:, 2:5], class_target)
loss_total = reg_weight * loss_regression + loss_classification
In this example, reg_target
will be a floating-point tensor of shape
[nBatch, 2]
and class_target
will be a long
tensor of shape [nBatch]
.
reg_weight
is a hyperparameter (not trained) that specifies the relative
weight between your two losses.
(You could also split your model into regression and classification branches
prior to the very last layer. Whether doing so would offer any advantage
would depend on the details of your use case.)
Best.
K. Frank
And another question: May we expect that both problems go with the same speed? I mean, is that possible that the regression part still is under-fited but the classification part goes over-fitted? Or they will have the same situation?
Thanks