Hi, this is my first post as often I encouter that all my PyTorch questions have been answered here at some point. Please accept my apologies if it happens this question has been already addressed.

In particular, I found a problem regarding converting variables from numpy to torch; I found that using torch.from_numpy() on big variables results in what appears to be a memory leakage. Here’s a very simple code in which I create a dictionary where each key will hold a 66x2 array. If I create the array beforehand and convert it through torch.from_numpy (Option 1), I find a MemoryError message when trying to dump the variable onto the file. If instead I convert each 66x2 array to torch “on-the-fly”, I find no errors when dumping the files. Does anyone have a clue about why this might be happening?

Many thanks

import pickle, torch, numpy as np

######### ----------- Option 1 ------------ #################

keys = [’{:06d}’.format(k) for k in range(10000)]

db = {}

points = np.random.randn(len(keys),66,2).astype(‘float’)

points = torch.from_numpy(points).float()

for i,k in enumerate(keys):

db[k] = points[i,:,:]

pickle.dump(db,open(‘fromnumpy.pkl’,‘wb’))

MemoryError

######## ----------- Option 2 --------------##################

keys = [’{:06d}’.format(k) for k in range(10000)]

db = {}

points = np.random.randn(len(keys),66,2).astype(‘float’)

for i,k in enumerate(keys):

db[k] = torch.from_numpy(points[i,:,:]).float()

pickle.dump(db,open(‘fromnumpy.pkl’,‘wb’))