TFRecords for Pytorch

Hi, is there a direct wat to get TFRecords dataset as Pytorch Dataset?

Now i am using Tensorflow to get the dataset to numpy and to Torch Tensor.

But , this is slow.

there isn’t a direct path. via numpy is the way to go

1 Like

If you are using the dataset often, I would suggest extracting it once and saving it in another format, this way you can have fast data reads when training.

I personnaly used LMDB for a larger-than-memory sized dataset, and data loading is fast enough to saturate the GPU(s) at all times (as advised by @smth in this post BTW).