Gradient calculation when module used multiple times (e.g. siamese network)

1

When a module is used multiple times, for example in a siamese network, are the gradients averaged?

[...]
output1 = net(image1)
output2 = net(image2)
loss = ...
loss.backward()
[...]
2

the gradients are summed (i.e. accumulated).

3 Likes