# PyTorch inference prioritize using /usr/local/cuda in PATH or cudatoolkit?

**URL:** <https://discuss.pytorch.org/t/pytorch-inference-prioritize-using-usr-local-cuda-in-path-or-cudatoolkit/190290>\
**Category:** deployment\
**Created:** [October 19, 2023, 12:21pm UTC](https://discuss.pytorch.org/t/pytorch-inference-prioritize-using-usr-local-cuda-in-path-or-cudatoolkit/190290 "2023-10-19T12:21:32Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![runa1996821](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/runa1996821/32/64738_2.png) [@runa1996821](https://discuss.pytorch.org/u/runa1996821)\
**Post date:** [October 19, 2023, 12:21pm UTC](https://discuss.pytorch.org/t/pytorch-inference-prioritize-using-usr-local-cuda-in-path-or-cudatoolkit/190290/1 "2023-10-19T12:21:32Z")

</div>

I wanted to try to align the precision of Pytorch inference between the two versions, but I found that adding CUDA paths to PATH and LD\_LIBRARY\_PATH affected the results of inference. So I did some comparative experiments, and these are my CUDA environment：

NVIDAI Driver: NVIDIA-SMI 460.73.01 Driver Version: 460.73.01 CUDA Version: 11.2  
conda env a : torch1.9.0+cu111  
conda env b : torch1.13.0+cu117  
“SET\_CUDA\_ENV” means execute following command:  
`export PATH=/usr/local/cuda-11.2`  
`export LD_LIBRARY_PATH=/usr/local/cuda-11.2/lib64:$LD_LIBRARY_PATH`  
“NO\_CUDA\_ENV” means do nothing about PATH and LD\_LIBRARY\_PATH.

experiment1:  
[torch 1.9.0+cu111 & SET\_CUDA\_ENV] vs [torch1.9.0+cu111 & NO\_CUDA\_ENV] I got same result  
experiment2:  
[torch 1.13.0+cu117 & SET\_CUDA\_ENV] vs [torch1.13.0+cu117 & NO\_CUDA\_ENV] I got different results on the first torch.nn.linear layer.  
experiment3:  
[torch 1.9.0+cu111 & SET\_CUDA\_ENV] vs [torch1.13.0+cu117 & SET\_CUDA\_ENV] I got different results on the first torch.nn.linear layer.  
experiment4:  
[torch 1.19.0+cu111 & NO\_CUDA\_ENV] vs [torch1.13.0+cu117 & NO\_CUDA\_ENV] I got different results on the first torch.nn.linear layer.

Based on the above experiments, I can’t know which cuda libraries pytorch uses for inference.If pytorch had prioritized /usr/local/cuda11.2 in PATH and LD\_LIBRARY\_PATH, experiment 3 should have yielded the same result. If cudatoolkit is preferred, then experiment 2 should got the same result. These experiments show that there may be other factors that influence the model’s inference.  
I’ve seen these discussions [How to check which cuda version my pytorch is using](https://discuss.pytorch.org/t/how-to-check-which-cuda-version-my-pytorch-is-using/116622), [How does PyTorch pick cuda versions?](https://discuss.pytorch.org/t/how-does-pytorch-pick-cuda-versions/137941)

Hope to get some explanation, thanks!

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [October 19, 2023, 5:03pm UTC](https://discuss.pytorch.org/t/pytorch-inference-prioritize-using-usr-local-cuda-in-path-or-cudatoolkit/190290/2 "2023-10-19T17:03:27Z")

</div>

The PyTorch binaries ship with their own CUDA dependencies and should not use any system-wide libs. If you want to use your system-wide CUDA toolkit you could build PyTorch from source.