Using a Google Cloud Storage bucket for dataset

[ejguan (Erjia)]

@sephib

here is a simpler version

datapipe = FSSpecFileLister(root=image_bucket, masks=['*.png'])
file_dp = datapipe.open_file_by_fsspec(mode='rb')
list(file_dp)

after this if you run

datapipe = FSSpecFileLister(root=image_bucket, masks=['*.png'])
file_dp = datapipe.open_file_by_fsspec(mode='rb')
list(file_dp)

again - you get a FileNotFoundError .

This is interesting. I am not sure if this is something with the caching within fsspec. Could you please open an issue on Github?