I’m trying to initialize weight of a conv layer like below,
def forward(self, x):
But I got the following error message:
ValueError: can’t optimize a non-leaf Tensor
What is the correct way to initialize weight mannually?
I should instead do:
weird flex but okay…
.data i discouraged, you should do:
what does torch.no_grad() do here?
I know normally we would put evaluation calls inside it, since gradients are not needed. But why do we need it when initializing weights?
Since everything that happens inside this block is not recorded. The inplace operation that change the value of your weights is not recorded either. And so it is a good way to initialize the value.
I guess my question then is what does ‘being recorded’ or ‘not being recorded’ do?
That is not such an easy question.
In some sense, ignore these operations when computing gradients.