Probably you can try to deepcopy the branches and then resetting the parameters of the branches to make them independent?
Current code has all the branches sharing the same weights.
Try resetting the parameters as shown here: How to re-set alll parameters in a network - #12 by Brando_Miranda