Copied Neural Nets Not Training Properly

autograd
1

I want to experiment with using genetic algorithms to evolve neural net architectures, and this requires creating duplicate copies of a neural net, mutating them in different ways, and training them separately to compare.

Calling deepcopy doesn’t work, but I built my own copy function which appears to copy everything over correctly (essentially by serializing the net and then building a new net from that data). However, when I then attempt to train the duplicate neural net, the loss stays the same at each iteration. Training the first net works as it should on the other hand.

I’m wondering if it has something to do with the optimizer looking at the computational graph for the first net instead of its copy, but I’m not sure - I don’t know a lot about how torch works under the hood.

Thanks in advance! I’m new around here.

2

If you are creating a new model, you would also have to create a new optimizer and pass the newly created parameters to it.
Using the old optimizer might work, if you just change the parameters somehow inplace, but we would need to see some code, which you are using, to debug it properly. :wink:

3

Thank you! It ended up being something simple I was overlooking, and I felt silly in hindsight.