Its mainly caused by IO problem. It seems that you used a super computer, but the swap area is too small. When loading data from disk, it would comsumes too much memory for data buffering.
Here is some suggestions:
- mapping the input feature to output label one by one and contiguously write to disk.
- clean the swap area.
- use other machine if possible.