I am using python 3.11
I am trying to speed up csv loading.
Up to now I do the following operations
- load csv via csv_read (pandas)
- copy from dataframe to torch.tensor
- manage data.
Of course via gpu processing is considerable faster, but loading and copy is the most time-wasting operation.
I found there is cudf, a cuda package to load csv directly to GPU, but it seems to be unsupported for my python version.
Did some of you have some good news for me?