S3FileLister fails to connect to s3

data
1

I am having issues connecting to S3 with the S3FileLister.
my code:

from torchdata.datapipes.iter import IterableWrapper
s3_prefix = ['s3://mybucket/myfolder/']
dp = IterableWrapper(s3_prefix).list_files_by_s3()
list(dp)

the error:

ValueError                                Traceback (most recent call last)
Cell In[6], line 32
     30 s3_prefix = ['s3://vehicle-data-db/' + key_prefix for key_prefix in [f'volvo/april_{i}/' for i in range(1, 3)]]
     31 dp = IterableWrapper(s3_prefix).list_files_by_s3()
---> 32 list(dp)

File ~\anaconda3\envs\ca\lib\site-packages\torch\utils\data\datapipes\_hook_iterator.py:173, in hook_iterator.<locals>.wrap_generator(*args, **kwargs)
    171         response = gen.send(None)
    172 else:
--> 173     response = gen.send(None)
    175 while True:
    176     datapipe._number_of_samples_yielded += 1

File ~\anaconda3\envs\ca\lib\site-packages\torchdata\datapipes\iter\load\s3io.py:73, in S3FileListerIterDataPipe.__iter__(self)
     71 for prefix in self.source_datapipe:
     72     while True:
---> 73         urls = self.handler.list_files(prefix)
     74         for url in urls:
     75             if match_masks(url, self.masks):

ValueError: Encountered network error when sending http request
This exception is thrown by __iter__ of S3FileListerIterDataPipe(length=-1, masks='', source_datapipe=IterableWrapperIterDataPipe)

when trying to access this bucket with boto3 it works as expected.
my credentials file is located at C:\Users\myuser\.aws
what modifications do i need to perform to make it work?

2

i didnt specified the region, fixed code that worked for me:

from torchdata.datapipes.iter import IterableWrapper
s3_prefix = ['s3://mybucket/myfolder/']
dp = IterableWrapper(s3_prefix).list_files_by_s3(region='myregion')
list(dp)