How to use SyncBn in pytorch 1.0?

when will pytorch to support Synchronized Batch Normalization?and how to use it in pytorch 1.0?

nn.SynchBatchNorm was added into the PyTorch Master branch 9 days ago via this PR. Alternatively, you could use the NVIDIA/apex implementation from here, if you don’t want to build PyTorch from source.

thanks for your reply!! can i update my pytorch version through

conda update pytorch torchvision

to update my pytorch to get syncbn?

You could try to install the nightly builds and check if the module is already there.
Alternatively you could build from source.

thanks for for your reply!!

hi, @ptrblck

What’s the major difference between nn.SynchBacthNorm and NVIDIA/apex? :thinking:

Or which one tends to achieve a better performance? :sweat_smile:

We recommend to use nn.SyncBatchNorm, as you shouldn’t see a large difference performance-wise.

1 Like

How to freeze the parameters of SyncBatchNorm?