Automatic Mixed Precision Sum of different losses

You could use a single GradScaler and scale the final loss as described here.