Any best practice regarding storage of preprocessed training data?

Hi!

Is there any recommended way for storing preprocessed tensors? I had found some tutorial where they used torch.save to store the tensors(training data not the model) and I did the same but the resultant files are 3x times bigger in size compared to the corresponding tfrecord (22MB->7MB). So, I would like to know if there is any other recommended way of storing preprocessed tensors.

I’m not sure but how about np.save or np.savez doc?

You know, we can easily change torch.tensor <-> numpy.ndarray.