I noticed that when torch.save() is used on a narrowed version of a tensor, it saves the whole tensor, but it returns the narrowed tensor. To me, this looks like a bug because it saves something more than what it is intended, and we are not able to recover this redundant info. I wanted to discuss this issue before reporting it as a bug. For that, I wrote a simple example, and you can check the output to see what I mean:
import torch
import os
x= torch.randint(0, 100, size=(10000,8), dtype=torch.long)
y1 = x.narrow(dim=0, start=0, length=10)
torch.save({"y": y1},"y1.pth")
y2 = x.narrow(dim=0, start=0, length=10).detach().clone()
torch.save({"y": y2},"y2.pth")
loaded_y1 = torch.load("y1.pth")["y"]
loaded_y2 = torch.load("y2.pth")["y"]
if torch.equal(loaded_y1, loaded_y2):
print("loaded_y1==loaded_y2")
if torch.equal(loaded_y1, y1):
print("loaded_y1==y1")
print("size y1.pth (in Bytes)=", os.stat("y1.pth").st_size)
print("size y2.pth (in Bytes)=", os.stat("y2.pth").st_size)
This is the output of the above script:
loaded_y1==loaded_y2
loaded_y1==y1
size y1.pth (in Bytes)= 640754
size y2.pth (in Bytes)= 1394
As you can see, both “y1.pth” and “y2.pth” are equal, but “y1.pth” is much larger because it saves the mother tensor, while we are not able to recover it.