Are you aware of any pytorch extensions made available in the internet (preferably as a pypi package) that provide connection to a database?
I would like to be able to provide a database credentials and a query and create a dataset/dataloader from that.
It’s rather an example not an extension, but here you can find a simple
Dataset implementation reading data from redis.
Hi @ptrblck, Do you see any bottlenecks in using databases like Redis or MongoDB for training? Any insights into how databases compare with a file system?
I don’t have any performance comparisons between databases and a filesystem loading, sorry.