How to use SyncBn in pytorch 1.0?

vision
1

when will pytorch to support Synchronized Batch Normalization?and how to use it in pytorch 1.0?

2

nn.SynchBatchNorm was added into the PyTorch Master branch 9 days ago via this PR. Alternatively, you could use the NVIDIA/apex implementation from here, if you don’t want to build PyTorch from source.

3

thanks for your reply!! can i update my pytorch version through

conda update pytorch torchvision

to update my pytorch to get syncbn?

4

You could try to install the nightly builds and check if the module is already there.
Alternatively you could build from source.

5

thanks for for your reply!!

6

hi, @ptrblck

What’s the major difference between nn.SynchBacthNorm and NVIDIA/apex? :thinking:

Or which one tends to achieve a better performance? :sweat_smile:

7

We recommend to use nn.SyncBatchNorm, as you shouldn’t see a large difference performance-wise.

1 Like
8

How to freeze the parameters of SyncBatchNorm?