Is F.layernorm equal to normalize tensor?

vision
1

a = torch.rand(32, 100)
layer_norm = nn.LayerNorm(100, elementwise_affine=False)
b = layer_norm(a)
c = (a - a.mean(1).unsqueeze(1).repeat(1,100)) / a.std(1).unsqueeze(1).repeat(1,100)
In my understanding, b should be equal to c in every element, however, i obtained slightly different results, can anyone help me?

2

As explained in the docs, the stddev is calculated via the biased estimator.
This should work:

c = (a - a.mean(1).unsqueeze(1).repeat(1,100)) / torch.sqrt(a.var(1, unbiased=False).unsqueeze(1).repeat(1,100) + 1e-5)