I am trying to run the following script to retrain the ITrackerModel: https://github.com/CSAILVision/GazeCapture/tree/master/pytorch
Leaving the settings the way they are in the repo I am running into problems when I run main.py using nohup python main.py --data_path /cluster/data/preprocessed/ --reset &. I cannot see the output of the individual workers in nohup.log and at first, it looks like it is stuck. I just stops showing me any output after main.py Line 158 (for ... in enumerate(train_loader)).
But when I just run the script with python main.py --data_path /cluster/data/preprocessed/ --reset i can see the output being printed to the console.
This, for example, applies to the print() defined here: main.py L192-L197 which lies inside the for ... in enumerate(train_loader) where train_loader is a DataLoader instance. Hence my assumption, that this is happening because of the workers output not showing.
I am fairly new to pytorch (and to python as well). Any help is appreciated.
P.S.: screen and tmux are great for this kind of stuff but unfortunately I am not able to use them in this setup here.