I am running the below DCGAN code for LSUN dataset with lmdb. I noticed that as the number of iterations increases the data loading becomes very slow (like after 4000 iterations with 4 or 16 workers the new batch takes a lot of time), is this a known issue or I am missing something ?
maybe this is a disk issue? if you are on a hard-drive, random access will slow down after you exceed cache.
I’ve been using lmdb on top of SSD, and my iterations are pretty fast.