Is it acceptable for you to implement a C++ parser that plugs to python dataloader? If so, look at How to use dataset larger than memory?. The ChunkDataset API might help you and there is python bindings on the way
Is it acceptable for you to implement a C++ parser that plugs to python dataloader? If so, look at How to use dataset larger than memory?. The ChunkDataset API might help you and there is python bindings on the way