# Why are the scales of max\_grad\_norm different between loss\_reduction='mean' and 'sum'

**URL:** <https://discuss.pytorch.org/t/why-are-the-scales-of-max-grad-norm-different-between-loss-reduction-mean-and-sum/224712>\
**Category:** Opacus\
**Created:** [March 18, 2026, 1:07pm UTC](https://discuss.pytorch.org/t/why-are-the-scales-of-max-grad-norm-different-between-loss-reduction-mean-and-sum/224712 "2026-03-18T13:07:39Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Amatubamem](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/amatubamem/32/79411_2.png) [@Amatubamem](https://discuss.pytorch.org/u/Amatubamem)\
**Post date:** [March 18, 2026, 1:07pm UTC](https://discuss.pytorch.org/t/why-are-the-scales-of-max-grad-norm-different-between-loss-reduction-mean-and-sum/224712/1 "2026-03-18T13:07:39Z")

</div>

I have a question regarding the effective scale of the gradient clipping threshold when using the default **`loss_reduction='mean'`**.

According to the Opacus tutorial (“[Building an Image Classifier with Differential Privacy](https://opacus.ai/tutorials/building_image_classifier)”), **`max_grad_norm`** is defined as:

“Max Grad Norm: The maximum L2 norm of per-sample gradients **before they are aggregated by the averaging step.** ”

However, my understanding of the current implementation is that when a PyTorch loss function uses **`reduction='mean'`** and **`make_private`** is called with **`loss_reduction='mean'`** , the per-sample gradients captured by Opacus are already scaled down by 1/_B_ (where _B_ is the batch size) at the loss computation stage.

Opacus then appears to perform clipping directly on these 1/_B_-scaled gradients using the provided **`max_grad_norm`** (_C_). As a result, the effective clipping threshold relative to the raw, unscaled gradients becomes _B_ times larger (_B_×_C_) than the theoretical definition.

I would like to ask for your insights on this behavior:

- Is this the intended design? Should users manually scale down **`max_grad_norm`** by 1/_B_ when using **`loss_reduction='mean'`** to align with the standard DP-SGD theory?

- Or is Opacus supposed to automatically handle this scale correction internally so that **`max_grad_norm`** consistently applies to the unscaled gradients?

Thank you!
