It there anyway to let program select free GPU automatically?

[yusanshi]

I think you can try this:

How can find available gpu?

You can use this to figure out the GPU id with the most free memory: nvidia-smi --query-gpu=memory.free --format=csv,nounits,noheader | nl -v 0 | sort -nrk 2 | cut -f 1 | head -n 1 | xargs So instead of: python3 train.py You can use: CUDA_VISIBLE_DEVICES=$(nvidia-smi --query-gpu=memory.free --format=csv,nounits,noheader | nl -v 0 | sort -nrk 2 | cut -f 1 | head -n 1 | xargs) python3 train.py You can also add it into your .bashrc then you can directly run python3 train.py: alias python3='C…