Is there a layer normalization for Conv2D

is there a layer normalization for 2d feature maps which has the same function with tf.contrib.layers.layer_norm in tensorflow?

1 Like

from what i see, no one has implemented it yet.

LayerNorm

if batch size is always 1,can i do it by reshape a (1,C,H,W) tensor to (1,CHW),and then tranpose it to (CHW,1) and apply batchnorm ? then reshape it back to (1,c,h,w)

Use the GroupNorm as followed:

nn.GroupNorm(1, out_channels)

It is equivalent with LayerNorm. It is useful if you only now the number of channels of your input and you want to define your layers as such

nn.Sequential(nn.Conv2d(in_channels, out_channels, kernel_size, stride), nn.GroupNorm(1, out_channels), nn.ReLU())
5 Likes