It seems that in OpenAI implementation, the L2-normalization of the weight computation is being differentiated through: https://github.com/openai/weightnorm/blob/master/tensorflow/nn.py#L200
Is it being differentiated through in PyTorch? https://github.com/pytorch/pytorch/blob/852f8526c52190125446adc9a6ecbcc28fb66182/aten/src/ATen/native/WeightNorm.cpp#L58
It seems so, but just to be sure.