I implemented multi-class Focal Loss in pytorch. Bellow is the code. log_pred_prob_onehot is batched log_softmax in one_hot format, target is batched target in number(e.g. 0, 1, 2, 3).

However, when I tested it, it worked poorly. I read the Focal Loss paper a couple of times. It seems straight. Maybe I didn’ t understand it very well. I’d appreciate if anybody can correct me! Or if there is a workable implementation, please let me know! Thanks in advance!

@BowieHsu I used it in my own project, which has a multi-class, unbalanced data set. So far, it is not as good as it is mentioned in the paper. It is still very hard to train the data.

I also tried to use it in my own project, I found I had to reduce the lr by a factor of 10, leading to a better first iteration, but then due to the reduced lr, the precision over the epochs is barely improving. Maybe increasing lr after the first epoch could improve.

i haven’t read the paper in deatils. But I thought the the term (1-p)^gamma and p^gamma are for weighing only. They should not be back propagated during gradient descent. Maybe you need to detach() your variables?

Hi Ben.
Have you confirmed that training with gamma=0 is same to with cross entropy loss?
I tried that in my implementation of focal loss. The result became very different

And I ask someone to answer my forum question. I can’t identify the problem.

I’d like to add something to this, since it was leading me to an error.
The implementation suggested by @Ben (marvis/pytorch-yolo2/blob/master/FocalLoss.py) beware specify alpha as a (C,1)-shaped tensor rather than a (C,).
Otherwise, the implementation will be still working, but the loss will be computed as a dot product between the batch alpha values and the batch class probabilities, which makes conceptually no sense.

Can anyone give me comment on my implementation?
I was using it to solve classification problem with imbalance class but when I checked the TensorBoard logging, the loss is always the same
I changed to nll_loss and the loss is changing now, but I am still looking forward to having others comments