You could use a WeightedRandomSampler and try to adapt this example for your use case.
Based on the figure it seems you are working on a regression task, so you would need to create bins first before calculating the “class weights” (which would be bin weights in your case).