Zipped csv file cannot be parsed using torchdata's CSVParser datapipe

data
1

Hi,

I’m trying to parse a single csv file that is zipped and stored in aws s3, but getting the following error:

"Exception when executing new-line character seen in unquoted field - do you need to open the file in 
universal-newline mode?\nThis exception is thrown by __iter__ of CSVParserIterDataPipe(fmtparams={'delimiter': ','}, source_datapipe=ZipArchiveLoaderIterDataPipe)"

Here’s my code:

dp = IterableWrapper([source]).list_files_by_fsspec()
dp = dp.open_files_by_fsspec(mode="rb")
dp = dp.load_from_zip()
dp = dp.parse_csv(delimiter=",")

df = pd.DataFrame()
for _, row in dp:
    print(row)
    df.append(row)

It looks like this error is due to how zip file was created (I downloaded the zip file from Kaggle), but I’m not sure what the solution is. Trying to change the mode to “rU” throws Unimplemented error.

Any help would be greatly appreciated - thank you!