How to combine gradients

I’m trying to combine the gradients of different instances of the same model but not really sure how to do it. I’m pretty new to PyTorch so I thought iterating through lists of gradients would work but printing out the gradients they seem to be in a format I’m not familiar with. Any help or advice would be greatly appreciated, as I’m not sure where to go from here. Thank you!


Does something like this works?

for p1, p2, p3 in zip(model1.parameters(), model2.parameters(), model3.parameters()):
    combined = your_fn(p1.grad, p2.grad, p3.grad)