I am unable to find any documentation on how to set multiple GPUs for inference.
In python the following can be done:
device = torch.device(“cuda” if torch.cuda.is_available() else “cpu”)
model = CreateModel()
However for C++ I can’t find the equivalent or any documentation.
torch::nn does not contain either DataParallel or DistributedDataParallel
Any help is appreciated, thanks!
libtorch uses the
parallel::data_parallel API as seen e.g. here. I don’t know if there are other wrappers, but based on the examples I would guess they should work in a similar way as the
DataParallel wrapper in Python.
Thank you for the snippet!!
I forgot to include
Hi, Sorry to disturb you , I tried the wrapper as you suggest,but the function “parallel::data_parallel” calls another function named “_parallel_run”, which can not be parsed by the compiler of vs2017. It seems like there is no implementation of this function.
Is there any suggestions?
Unfortunately, I don’t know what the current status and support of distributed runs on Windows is so can’t help much Let’s wait for some Windows experts in case they are familiar with it.