How to combine gradients

vision
1

I’m trying to combine the gradients of different instances of the same model but not really sure how to do it. I’m pretty new to PyTorch so I thought iterating through lists of gradients would work but printing out the gradients they seem to be in a format I’m not familiar with. Any help or advice would be greatly appreciated, as I’m not sure where to go from here. Thank you!

2

Hi,

Does something like this works?

for p1, p2, p3 in zip(model1.parameters(), model2.parameters(), model3.parameters()):
    combined = your_fn(p1.grad, p2.grad, p3.grad)