How to handle loss function for imbalanced dataset in cnn?

You don’t necessarily need to normalize your weights and could e.g. just use the inverse of the class counts.
@K_Frank gives some good examples and intuitions in this post.