One complex weight share problem

There is a two-way GAN in paper https://www.cmlab.csie.ntu.edu.tw/project/Deep-Photo-Enhancer/CVPR-2018-DPE.pdf, there are four generators, their network structure is the same, Gx and G`x share all parameters except the BN layer, Gy and G`y share all parameters except the BN layer, but Gx and Gy does not share parameters.

How to implement this kind of network with pytorch?

A crude version would be to copy the parameters from one network to the sibling after each weight update. Not very quick but maybe it’s good enough for your needs