InPlace BatchNorm

This batchnorm claims it is 60% faster than pytorch’s. Why is this not the default in Pytorch?

Hi,

The main reason is that this is not a batchnorm. This is a “combo” layer with batchnorm + activation (+other stuff I don’t know). This approach allows you to save memory (and a bit of runtime).

This batchnorm claims it is 60% faster than pytorch’s.

Where does it claims this? This sounds quite unlikely. It’s not memory usage?