I couldn’t find any efficient or conventional way to use torchtext.data.Dataset with large dataset. We can use the large file automatically in case of the original Dataset from pytorch. But in the case of torchtext , it’s not that simple.
Is there any way we can use Generator with torchtext?
Or is there any other way?