Is it possible to learn and tune the neural network in pytorch with updating network parameters and weigths?
I mean Can I do the proccess of the learning the neural network with updating only the weights and parameters by using " state_dict.items() " ? Or the only way to learn the network is using the loss.backward() and optimzer.step() ?
thanks
It’s not exactly clear what you mean by learn and tune the networks.
If you want to train the network from scratch then you would initialise the weights randomly and then use loss.backward() which computes the gradients for tensors with requires_grad=True and then when you do an optimizer.step() you perform an update for the weights of the network.
If you want to tune the network, essentially you do the same thing as above except that you freeze the weights for certain layers by setting requires_grad=False for the weights of those layers. The transfer learning tutorial explains both fine-tuning as well as training from scratch.
1 Like