Using "" and ""

Hello everyone.

I know this has been asked several times but I still have some doubts.

I am training a NN to fit a 3D function. Then, I have a file with four columns, three of them are the input values and the forth is the value of the function at the point. Say I train the NN with 1000 - 2000 points, to give you an idea.

The way I am loading data is simply:
data = np.loadtxt(‘path_to_file’)
x = torch.from_numpy(data[:,0:3])
y = torch.from_numpy(data[:,3])

and with that I train the NN.

Checking this forum, I have seen there are options like “” or “” that seem to be build for this purpose.

Could anyone explain me:
Are they build to load datasets as the ones I am using?
What would be the advantage of using them in my case?

Thank you very much!!

They are used when 1) data is too large to load ll at once into memory, and/or 2) needs preprocessing that is most efficiently done in a separate process in order to not block training.