Question About Adding Parameters/Biases


If I want to add more neurons to a layer inside of a model during training, do I only need to update the model.parameters() for each layer(weights and biases) after backpropagation, or is there more to it than that? If so, what would I need to do in addition to resizing the parameter tensors?

Since you are changing the parameters, you would need to pass these new parameters to the optimizer (or create a new one). The running stats in this optimizer would be lost in this case (if it’s using internal estimates).

After reading several posts, I’ve come to this same conclusion. Which leads to my next question, here: