I know if I run the code below, the memory would increase
loss = 0 for i in range(n) loss_part = criteria() loss += loss_part loss.backward()
But why does it increase fast and never release the memory it allocated ?
The case is, I have different dimension of data in a batch ( sequence in different length ).
So I wanna to calculate every
criteria( seq_i ) and then sum them.
One approach is I concatenate every seq_i, and only use
criteria( seq_sum, target) once. Unfortunately it cannot fixed the memory increasing problem. Thanks to all previously.
Bs, I also used list to store some data (
New It seems the increase is not caused by accumulative loss. Need help to close this.