I currently have two feature files, each with a size of 3GB, and an 8GB machine. Now, I want to load both files into memory one after the other and merge them into a single variable. I would like to know how to do this without exceeding the available memory. I tried using torch.cat
, but it seems that the cat
function might allocate additional memory for merging, which leads to memory overflow.