Your idea of clustering the regression targets to a few clusters and assigning weights to these seem reasonable.
You could either do it manually (as it seems to be the case now) or use something like k-means.
Once you have the clusters, you could count the samples similar to a classification task and calculate the weights based on the number of samples for each cluster and create a mapping between the cluster and weight.
After creating the weights, you could write a function, which accepts the current output batch with the regression prediction, as well as your cluster centers (k-means dict), and returns a batch of weights, which can then be multiplied to create the final loss.
1 Like