I have a **shared tensor** used in different epochs, so I need to **shared_tensor = tensor(require_grad).detach()** , or I have to set loss.background( **retain_graph=True** ) because of the dependency of the shared_tensor. When I set retain_graph=True, the graph will grow during training.

The problem is that the one part in loss is based on the shared _tensor. For example, Loss = loss_a + loss_b, **loss_b is from the shared_tensor,** but since shared_tensor requires no grad (from tensor.detach() ), loss_b cannot be used to update the network in back propagation, right?

How to fix it? What’s the correct way to save the shared_tensor and calculate related loss?

code:

yi=modelA(xi)

loss_1 = L1(yi,0)

yi’ = yi (by detach() or clone())

loss_2 = L1(torch.op([y1’, y2’…]))

loss = loss_1 + loss_2

loss.backward()