Memory usage during evaluation

1

Hi PyTorch Experts,
I have two questions regarding memory usages in pytorch model training and evaluation. It would be very helpful to know about it.

  1. Does .unsqueeze operation increases memory?
    For example, c = a.unsqueeze(dim=1) + b.unsqueeze(dim=0) increases memory consumption during evaluation. Can with torch.no_grad(): be helpful to reduce memory footprint here?

  2. Does a:torch.Tensor = torch.sigmoid(c) creates new memory for variable a, every time this line is called?

2

  1. No, unsqueeze manipulates the metadata (shape etc.) of the tensor and does not allocate new memory since the number of elements is equal.

  2. Yes, a is a new tensor which will use memory. PyTorch uses a caching allocator, which will try to reuse GPU memory if possible to avoid expensive cudaMalloc calls.

1 Like
3

Thanks so much @ptrblck for prompt response!