eg if I have embedding = nn.Embedding(V, E), is the shortest method something like:
scale = my_hand_written_formula_here
embedding.weight.data.uniform(-scale, scale)
(and ditto for bias too, eg in a Linear).
?
I think you are missing the underscore to call uniform inplace. Besides that, that would be one way.
I would recommend to avoid using .data and use something like this instead:
with torch.no_grad():
embedding.weight.uniform_(-scale, scale)
Ok. How to get scale? Do I need to hunt down an appropriate formula? (I know a couple, just wondering if there is an easy way that avoids having to do this?).
If none of the implemented init functions provides what you need, you would have to implement it somehow manually.
However, there are already a lot of init functions like xavier_uniform etc.
Also, maybe it’ll help to have a look at the code to get some idea how to calculate the scale etc.
Ok. missing sqrt(3) / sqrt(num_inputs) * factor initialization, eg see https://www.tensorflow.org/api_docs/python/tf/uniform_unit_scaling_initializer
(But does otherwise seem quite comprehensive 
And maybe I just overlooked this one in the list somewhere, eg not written quite in the same formula, but is equivalent).