For example, if I have bunch of feature maps, let’s say we have 100 feature map in the shape [128, 256, 256].
then I want to get the pixel-wise max feature map of this 100 feature.
the easiest way is to use torch.cat() to concatenate the total 100 feature map then use torch.max() to get the result. But my memory is not big enough to store the tensor from concatenation.
So, I think maybe I can use for-loop to get the max tensor. code is like this:
for i in range(100): if i == 0: out = feature[i] else: out = torch.max(out, feature[i])
but I find that memory is still growing. I believe it is because this torch.max(out, feature[i]) also need to create a continuous space to hold the tensor. But is there a way to free this memory after the calculation? Because ideally we don’t need this memory to do auto-grad, we only need to remember each pixels comes from then we can get gradient.