Custom weight intialization

I am sorry if this is a very simple question. How do I initialize the weights of my network such that the weights are produced by the softmax function on the learnable log-probabilities?

You could create the desired initial weights using any method and copy them into the model parameter via:

my_weight = ... # your method to create the initial weights
with torch.no_grad():
1 Like