Actually @ptrblck, I had already fixed that part…my apologies for not updating that part of the code.
Regardless, it seems the training is still not working. What I did to debug it is set trainable to a net with standard initialization and then non-trainable to zero. Then I just computed in the placeholder the net f(W_nontrain+W_train) = f(0+W_train) = f(W_train) which should just train as standard SGD but I don’t get any change in the loss which must mean something is wrong. Did you manage to get it to work?
Thanks so much for giving this a check. I appreciate it 
I’ve debugged your code a bit and it seems that some names are a bit confusing.
In combine_nets it seems are mixing the train/no_train naming scheme:
params_train = net_no_train.named_parameters()
dict_params_no_train = dict(net_train.named_parameters())
However, the other issue is, that your dict_params_place_holder will be empty once we set the new attributes.
Most likely my suggestion to use setattr(getattr(...)) was wrong.
The modules are not registered anymore and thus the function doesn’t do anything at all after the first run.
1 Like
Oh dam thats embarrassing I forgot that I already fixed that issue too (of course after so many days). I feel really bad about that. Definitively requires an apologies…
hmmm…does that mean that what I am trying to do is not possible? Or is there a way to make it work?
Brando, Brando… now you owe me a beer… or two. 
Could you post the most recent code then?