Is there a way to combine classification and regression in single model?

Is that possible to predict both classification and regression in single model?
Suppose that we have target shape like [b1, b2, c1, c2, c3] which bs are for regression target and cs are for classification target and they are from one-hot encoding.

Hi Mahdi!

Yes, and it can be a reasonable thing to do.

I recommend against using one-hot encoding. There’s no reason to do
so and it introduces at least a little inefficiency.

(I assume that you mean that you have three classes and that exactly
one of c1, c2, and c3 is one, with the other two being zero.)

Instead, let your classification target be a single integer class label that
ranges from 0 to 2.

The simplest approach will be to have your final layer be a Linear with
out_features = 5. The first two outputs will be understood as the
continuous-valued predictions for b1 and b2. The remaining three will
be understood as raw-score logits for the three classes. Then, e.g.:

loss_regression = torch.nn.MSELoss() (output[:, 0:2], reg_target)
loss_classification = torch.nn.CrossEntropyLoss() (output[:, 2:5], class_target)
loss_total = reg_weight * loss_regression + loss_classification

In this example, reg_target will be a floating-point tensor of shape
[nBatch, 2] and class_target will be a long tensor of shape [nBatch].
reg_weight is a hyperparameter (not trained) that specifies the relative
weight between your two losses.

(You could also split your model into regression and classification branches
prior to the very last layer. Whether doing so would offer any advantage
would depend on the details of your use case.)

Best.

K. Frank

1 Like

And another question: May we expect that both problems go with the same speed? I mean, is that possible that the regression part still is under-fited but the classification part goes over-fitted? Or they will have the same situation?
Thanks