Ok, so the problem was that torch+cu118 was built upon CuDNN 8.7 while the NVIDIA CUDA 11.8 image offers CuDNN 8.9
Removing /lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8 and using torch’s internal so file will instead break cv2 this time.
>>> ./classify
[ WARN:0@0.598] global init.hpp:32 checkVersions cuDNN reports version 8.7 which is not compatible with the version 8.9 with which OpenCV was built
Could not load library libcudnn_cnn_infer.so.8. Error: /lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8: undefined symbol: _ZN11nvrtcHelper4loadEb, version libcudnn_ops_infer.so.8
Aborted (core dumped)
Is there any possible way to let cv2 reference cudnn 8.9 while pytorch reference its own cudnn?
And this issue is narrowed down to PyTorch 2.2 conflict with locally installed cuDNN (version: · Issue #119989 · pytorch/pytorch · GitHub
However, will the change be applied to older torch versions like 2.1 and 2.2?