Executing a backward() call multiple times on a device, have a lot of device to host copies after each backward call

1

I’m trying to understand the usage of backward multiple times of a single operator. After each of the backward call I observe a DeviceToHost copy. Is this expected ?