The code is flowing:
if epoch > self.epoch_merge:
if epoch == self.epoch_merge:
for i,param in enumerate(zip(self.decoder1.parameters(),self.decoder2.parameters(),self.decoder_merge.parameters())):
param[2].data = param[0].data + param[1].data
out_merge = self.decoder_merge(feature_merge_list)
My goal is to merge two all parameters of decoders into a new decoder(named decoder_merge) with same network structure,then the new decoder will be trained continuedly.However, When the epoch equals to epoch_merge,I got a very low accuracy than previous one. After this,the model accuracy was increased normally.
Therefore,I want to know what cause the cliff descent? Is it right for my merge measure?Thanks in advanced.