Hi,
I trained a pytorch model using CUDA using pytorch python, and try to inference using C++.
First, I exported the model using torch script (via tracing), with eval mode (tracing is done below the line model.eval()). Also I changed the device to cpu before tracing. The machine for training is Ubuntu 18.04, with GTX3070.
My inference tests are done in my laptop (Windows10) and linux clusters (RedHat linux), but the inference results are different from each other. Both machines don’t have GPUs, so the only way for inference is with CPUs.
Even if the differences are very small (i.e. 0.9824 vs 0.9827 and so on…) I don’t want any discrepancy between them. How can I avoid this?
The libtorch version in those machines are the same (1.9.0)