While training a model I had assumed that I could use .detach().numpy() as a functionally equivalent call for .item() (for a single-value tensor), but found its behavior to be inconsistent.
Specifically, I was using it to store the mean loss during training iterations and found that when certain loss functions were used (BCELoss and MSELoss) it would result in constantly increasing memory usage. However, if other loss functions were used (such as a custom Dice loss function) there were no memory issues.
Changing the optimizer had no effect on the memory usage, nor did re-creating the loss function class each epoch (in case something was being stored there).
Naturally, the solution was to just use .item(), but it seems strange to me that there was an issue at all.
Am I wrong in expecting .detach().numpy() to work without issue for a single-value tensor?