I illustrate my issue with the following simple example:
x = torch.randn(3,4)
x.requires_grad = True
print(‘before del x’,torch.cuda.memory_allocated(device=0)
print(‘after del x’,torch.cuda.memory_allocated(device=0))
by running the above code ‘del x’ does not delete the tensor x and allocated memory does not change. However if i change ‘x.requires_grad= False’ and ‘q.requires_grad = True’, I can run the above code and ‘del x’ removes the tensor x.
The question is how can I delete tensor x by setting ‘x.requires_grad = True’ ?