Hi, I currently use slurm and I’ve noticed when I run a program from the terminal, the dataloader when initiated the first time will load everything within 10 minutes. However, when I run the program with the same configuration, code, and resources using “sbatch” mode in a headless fashion, then it will take multiple hours for the dataloader to initiate.
Any thoughts on what might be the problem?