Hey, I wanted to share something I’ve encountered:
Issue 1:
When running my test from the PyCharm terminal with the DataLoader configured as follows, everything works fine:
--num-workers 10 --batch-size 20
However, if I run the same test from my regular terminal, I get this error:
Bus error. It is possible that dataloader's workers are out of shared memory. Please try to raise your shared memory limit.
Why the training is working with Pycharm terminal and not in my regular terminal?
Issue 2:
In a similar scenario, the test works fine in the PyCharm terminal but fails in my regular terminal with this error:
Too many open files
What I’ve discovered so far:
It seems PyCharm’s terminal applies some pre-configurations. For example:
In the PyCharm terminal:
$ ulimit -n
1048576
In my regular terminal:
$ ulimit -n
1024
Is there a recommended list of configurations to set up before starting training? It seems like there should be! 