How to make a loss avoid some of the regions for back propagation

I have inputs with different sizes. In order to be able to use them in a batch, I padded them to have the same size. now at the end of each iteration, I have loss(output, label), where both output and label are in the shape of B,C,W,H but there are just certain part o each image that I care about and don’t want to penalize the model for making mistake on the padded parts.
Is there a way to modify the loss (in my case MSE) to not care about the regions that are padded?

does it make sense if I just multiply the output with a mask at the end that has zero for the regions that I don’t care before computing the loss?

Yes, using a mask to ignore the padded parts seems right. At the moment, you can’t specify for MSELoss that some parts of the input should be ignored so zeroing those parts out is the only solution.