Segmentation Fault while importing torchvision 0.24.0

jdumke · September 27, 2025, 10:11pm

Hello all,

I’m experimenting with the Vulkan backend, hence my GPU is currently not supported by ROCm. I built a custom Version of torch 2.9.0-rc2 and vision 0.24.0 with following build configuration:

#!/usr/bin/env bash
export CMAKE_BUILD_TYPE=DEBUG
export BUILD_LIBTORCH_CPU_WITH_DEBUG=1
export BUILD_FUNCTORCH=1
export USE_VULKAN=1
export USE_VULKAN_FP16_INFERENCE=1
export USE_VULKAN_RELAXED_PRECISION=1
export USE_VULKAN_SHADERC_RUNTIME=1
export USE_OPENMP=1
export USE_CUDA=0
export USE_ROCM=0
export USE_SYSTEM_CPUINFO=1
export USE_SYSTEM_FP16=1
export USE_SYSTEM_PYBIND11=1
export USE_SYSTEM_XNNPACK=1
export USE_SYSTEM_FXDIV=1
export USE_SYSTEM_SLEEF=1
export USE_SYSTEM_PTHREADPOOL=1
export MAX_JOBS=8
export CMAKE_GENERATOR="Ninja"
# for vision
export WITH_CUDA=0
export WITH_MPS=0
export WITH_PNG=1
export WITH_JPEG=1
export WITH_WEBP=1
export WITH_AVIF=1

The plain torch module is working fine, unlike the syntax inconsistency when using the vulkan backend and lacking methods (like device_count()). But when I’m simply importing torchvision a segmentation fault occurs:

nv) jds@padua:~/test/vtt$ python minimal_bug.py 
[padua:248026:0:248026] Caught signal 11 (Segmentation fault: address not mapped to object at address (nil))
==== backtrace (tid: 248026) ====
 0  /lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2bc) [0x7f50eba3064c]
 1  /lib/x86_64-linux-gnu/libucs.so.0(+0x3182f) [0x7f50eba3082f]
 2  /lib/x86_64-linux-gnu/libucs.so.0(+0x319fa) [0x7f50eba309fa]
 3  /lib/x86_64-linux-gnu/libc.so.6(+0x3fdf0) [0x7f50ec677df0]
 4  /lib/x86_64-linux-gnu/libc.so.6(+0x1643c9) [0x7f50ec79c3c9]
 5  /home/jds/test/vtt/env/lib/python3.13/site-packages/torch/lib/libtorch_python.so(_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_M_constructIPcEEvT_S7_St20forward_iterator_tag+0xca) [0x7f50e27b0e5e]
 6  /home/jds/test/vtt/env/lib/python3.13/site-packages/torch/lib/libtorch_cpu.so(_ZN3c104impl13OperatorEntry14registerKernelERKNS_10DispatcherESt8optionalINS_11DispatchKeyEENS_14KernelFunctionES5_INS0_12CppSignatureEESt10unique_ptrINS_14FunctionSchemaESt14default_deleteISC_EENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0x378) [0x7f50ca3997a2]
 7  /home/jds/test/vtt/env/lib/python3.13/site-packages/torch/lib/libtorch_cpu.so(_ZN3c1010Dispatcher12registerImplENS_12OperatorNameESt8optionalINS_11DispatchKeyEENS_14KernelFunctionES2_INS_4impl12CppSignatureEESt10unique_ptrINS_14FunctionSchemaESt14default_deleteISA_EENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0x141) [0x7f50ca385203]
 8  /home/jds/test/vtt/env/lib/python3.13/site-packages/torch/lib/libtorch_cpu.so(_ZNR5torch7Library5_implEPKcONS_11CppFunctionENS_17_RegisterOrVerifyE+0x413) [0x7f50ca429c7b]
 9  /home/jds/test/vtt/env/lib/python3.13/site-packages/torchvision/_C.so(+0x248de) [0x7f503e6288de]
10  /lib64/ld-linux-x86-64.so.2(+0x4fae) [0x7f50ec9acfae]
11  /lib64/ld-linux-x86-64.so.2(+0x507c) [0x7f50ec9ad07c]
12  /lib64/ld-linux-x86-64.so.2(_dl_catch_exception+0x106) [0x7f50ec9aa426]
13  /lib64/ld-linux-x86-64.so.2(+0xba28) [0x7f50ec9b3a28]
14  /lib64/ld-linux-x86-64.so.2(_dl_catch_exception+0x79) [0x7f50ec9aa399]
15  /lib64/ld-linux-x86-64.so.2(+0xbda8) [0x7f50ec9b3da8]
16  /lib/x86_64-linux-gnu/libc.so.6(+0x8f258) [0x7f50ec6c7258]
17  /lib64/ld-linux-x86-64.so.2(_dl_catch_exception+0x79) [0x7f50ec9aa399]
18  /lib64/ld-linux-x86-64.so.2(+0x24bf) [0x7f50ec9aa4bf]
19  /lib/x86_64-linux-gnu/libc.so.6(+0x8ed67) [0x7f50ec6c6d67]
20  /lib/x86_64-linux-gnu/libc.so.6(dlopen+0x69) [0x7f50ec6c7309]
21  /usr/lib/python3.13/lib-dynload/_ctypes.cpython-313-x86_64-linux-gnu.so(+0x17dfd) [0x7f50ec58ddfd]
22  python() [0x588a20]
23  python(_PyObject_MakeTpCall+0x27b) [0x54726b]
24  python(_PyEval_EvalFrameDefault+0xfce) [0x56270e]
25  python() [0x5a090c]
26  python(_PyObject_MakeTpCall+0x1d5) [0x5471c5]
27  python(_PyEval_EvalFrameDefault+0x2661) [0x563da1]
28  python(PyEval_EvalCode+0xcc) [0x55d48c]
29  python() [0x5da802]
30  python() [0x57e1cd]
31  python(_PyEval_EvalFrameDefault+0x4008) [0x565748]
32  python() [0x587b36]
33  python(PyObject_CallMethodObjArgs+0xe2) [0x5cc632]
34  python(PyImport_ImportModuleLevelObject+0x23b) [0x5cb42b]
35  python(_PyEval_EvalFrameDefault+0x577e) [0x566ebe]
36  python(PyEval_EvalCode+0xcc) [0x55d48c]
37  python() [0x5da802]
38  python() [0x57e1cd]
39  python(_PyEval_EvalFrameDefault+0x4008) [0x565748]
40  python() [0x587b36]
41  python(PyObject_CallMethodObjArgs+0xe2) [0x5cc632]
42  python(PyImport_ImportModuleLevelObject+0x23b) [0x5cb42b]
43  python(_PyEval_EvalFrameDefault+0x577e) [0x566ebe]
44  python(PyEval_EvalCode+0xcc) [0x55d48c]
45  python() [0x6ab8d1]
46  python() [0x6a899c]
47  python() [0x6b9943]
48  python() [0x6b93e3]
49  python() [0x6b921e]
50  python(Py_RunMain+0x3c1) [0x6b86f1]
51  python(Py_BytesMain+0x2b) [0x6838eb]
52  /lib/x86_64-linux-gnu/libc.so.6(+0x29ca8) [0x7f50ec661ca8]
53  /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x85) [0x7f50ec661d65]
54  python(_start+0x21) [0x682c81]
=================================
Segmentation fault

Zhe minimal script is:

#!/usr/bin/env python
import torch
import torchvision

Two things make wonder:

Why torchvision depends on a video-broadcasting-library (libucs)?
What causes the trouble?

My system in bullets:

Debian 13 (Trixie)
GCC 14.2.0
Python 3.13
libc 2.41
libstdc++ 12.4.5

The post is quite long, if further information is needed I’ll provide it later.

Any suggestions, what went wrong?

Greets,

Jonathan

ptrblck · September 28, 2025, 2:07pm

It doesn’t and I guess you are loading UCS from UCX installed on your system. UCS has a fault handler and is showing up in the stacktrace as it did not remove the fault handling from the call trace. You can ignore it here since it’s not causing the segfault.
Unclear, but based on the calls I would start by checking if your custom builds are using the same CXX ABI first.

jdumke · September 29, 2025, 12:27am

Thanks for your reply.

At a first glance the ABI versions seeming to fit, but on a closer look …

Here is what is nm says:

for _C.so:

                 U _ZTVN10__cxxabiv117__class_type_infoE@CXXABI_1.3
                 U _ZTVN10__cxxabiv119__pointer_type_infoE@CXXABI_1.3
                 U _ZTVN10__cxxabiv120__function_type_infoE@CXXABI_1.3
                 U _ZTVN10__cxxabiv120__si_class_type_infoE@CXXABI_1.3

for libtorch_cpu.so:

                 U __cxa_allocate_exception@CXXABI_1.3
                 U __cxa_bad_cast@CXXABI_1.3
                 U __cxa_bad_typeid@CXXABI_1.3
                 U __cxa_begin_catch@CXXABI_1.3
                 U __cxa_call_terminate@CXXABI_1.3.15
                 U __cxa_demangle@CXXABI_1.3
                 U __cxa_end_catch@CXXABI_1.3
                 U __cxa_free_exception@CXXABI_1.3
                 U __cxa_guard_abort@CXXABI_1.3
                 U __cxa_guard_acquire@CXXABI_1.3
                 U __cxa_guard_release@CXXABI_1.3
                 U __cxa_init_primary_exception@CXXABI_1.3.11
                 w __cxa_pure_virtual@CXXABI_1.3
                 U __cxa_rethrow@CXXABI_1.3
                 U __cxa_thread_atexit@CXXABI_1.3.7
                 U __cxa_throw_bad_array_new_length@CXXABI_1.3.8
                 U __cxa_throw@CXXABI_1.3
                 U __dynamic_cast@CXXABI_1.3
                 U __gxx_personality_v0@CXXABI_1.3
                 U _ZdaPvm@CXXABI_1.3.9
                 U _ZdlPvm@CXXABI_1.3.9
                 U _ZdlPvmSt11align_val_t@CXXABI_1.3.11
                 U _ZNSt15__exception_ptr13exception_ptr10_M_releaseEv@CXXABI_1.3.13
                 U _ZNSt15__exception_ptr13exception_ptr9_M_addrefEv@CXXABI_1.3.13
                 U _ZNSt15__exception_ptr13exception_ptrC1EPv@CXXABI_1.3.11
                 U _ZnwmSt11align_val_t@CXXABI_1.3.11
                 U _ZSt11_Hash_bytesPKvmm@CXXABI_1.3.5
                 U _ZSt17current_exceptionv@CXXABI_1.3.3
                 U _ZSt17rethrow_exceptionNSt15__exception_ptr13exception_ptrE@CXXABI_1.3.3
                 U _ZTIb@CXXABI_1.3
                 U _ZTIc@CXXABI_1.3
                 U _ZTId@CXXABI_1.3
                 U _ZTIf@CXXABI_1.3
                 U _ZTIh@CXXABI_1.3
                 U _ZTIi@CXXABI_1.3
                 U _ZTIj@CXXABI_1.3
                 U _ZTIl@CXXABI_1.3
                 U _ZTIm@CXXABI_1.3
                 U _ZTIN10__cxxabiv115__forced_unwindE@CXXABI_1.3.2
                 U _ZTIPKc@CXXABI_1.3
                 U _ZTIv@CXXABI_1.3
                 U _ZTVN10__cxxabiv116__enum_type_infoE@CXXABI_1.3
                 U _ZTVN10__cxxabiv117__class_type_infoE@CXXABI_1.3
                 U _ZTVN10__cxxabiv119__pointer_type_infoE@CXXABI_1.3
                 U _ZTVN10__cxxabiv120__function_type_infoE@CXXABI_1.3
                 U _ZTVN10__cxxabiv120__si_class_type_infoE@CXXABI_1.3
                 U _ZTVN10__cxxabiv121__vmi_class_type_infoE@CXXABI_1.3

for libtorch_python.so

                 U __cxa_allocate_exception@CXXABI_1.3
                 U __cxa_bad_typeid@CXXABI_1.3
                 U __cxa_begin_catch@CXXABI_1.3
                 U __cxa_demangle@CXXABI_1.3
                 U __cxa_end_catch@CXXABI_1.3
                 U __cxa_free_exception@CXXABI_1.3
                 U __cxa_guard_abort@CXXABI_1.3
                 U __cxa_guard_acquire@CXXABI_1.3
                 U __cxa_guard_release@CXXABI_1.3
                 U __cxa_init_primary_exception@CXXABI_1.3.11
                 w __cxa_pure_virtual@CXXABI_1.3
                 U __cxa_rethrow@CXXABI_1.3
                 U __cxa_thread_atexit@CXXABI_1.3.7
                 U __cxa_throw_bad_array_new_length@CXXABI_1.3.8
                 U __cxa_throw@CXXABI_1.3
                 U __dynamic_cast@CXXABI_1.3
                 U __gxx_personality_v0@CXXABI_1.3
                 U _ZdaPvm@CXXABI_1.3.9
                 U _ZdlPvm@CXXABI_1.3.9
                 U _ZdlPvmSt11align_val_t@CXXABI_1.3.11
                 U _ZNSt15__exception_ptr13exception_ptr10_M_releaseEv@CXXABI_1.3.13
                 U _ZNSt15__exception_ptr13exception_ptr9_M_addrefEv@CXXABI_1.3.13
                 U _ZNSt15__exception_ptr13exception_ptrC1EPv@CXXABI_1.3.11
                 U _ZnwmSt11align_val_t@CXXABI_1.3.11
                 U _ZSt11_Hash_bytesPKvmm@CXXABI_1.3.5
                 U _ZSt17current_exceptionv@CXXABI_1.3.3
                 U _ZSt17rethrow_exceptionNSt15__exception_ptr13exception_ptrE@CXXABI_1.3.3
                 U _ZTIa@CXXABI_1.3
                 U _ZTIb@CXXABI_1.3
                 U _ZTId@CXXABI_1.3
                 U _ZTIDn@CXXABI_1.3.5
                 U _ZTIf@CXXABI_1.3
                 U _ZTIh@CXXABI_1.3
                 U _ZTIi@CXXABI_1.3
                 U _ZTIj@CXXABI_1.3
                 U _ZTIl@CXXABI_1.3
                 U _ZTIm@CXXABI_1.3
                 U _ZTIN10__cxxabiv115__forced_unwindE@CXXABI_1.3.2
                 U _ZTIPKc@CXXABI_1.3
                 U _ZTIPKv@CXXABI_1.3
                 U _ZTIs@CXXABI_1.3
                 U _ZTISt16nested_exception@CXXABI_1.3.5
                 U _ZTIv@CXXABI_1.3
                 U _ZTVN10__cxxabiv116__enum_type_infoE@CXXABI_1.3
                 U _ZTVN10__cxxabiv117__array_type_infoE@CXXABI_1.3
                 U _ZTVN10__cxxabiv117__class_type_infoE@CXXABI_1.3
                 U _ZTVN10__cxxabiv119__pointer_type_infoE@CXXABI_1.3
                 U _ZTVN10__cxxabiv120__function_type_infoE@CXXABI_1.3
                 U _ZTVN10__cxxabiv120__si_class_type_infoE@CXXABI_1.3
                 U _ZTVN10__cxxabiv121__vmi_class_type_infoE@CXXABI_1.3

jdumke · September 30, 2025, 11:09am

Okay I found the problem. The Building environment for torchvision 0.24.0 pulls torch 2.8.0 instead of locally built 2.9.0 while importing both modules the vtables don’t fit.

How can I convince pip to use my local version of torch and not a downloaded one?

ptrblck · September 30, 2025, 10:10pm

Try to use –no-build-isolation.

jdumke · September 30, 2025, 10:33pm

My actual pip call looks like:

pip wheel -w ~/test/vtt/dist/ --no-build-isolation --extra-index-url file:///home/jds/test/vtt/dist/simple/ ./vision/

I found the trick, due to the wheel-name:

torch-2.9.0a0+gitd7a703e-cp313-cp313-linux_x86_64.whl

I’ve to set following env-variable

export PYTORCH_VERSION=2.9.0a0

jdumke · October 4, 2025, 10:54pm

Okay, now I’ve got a new problem: the built and installed torchvision can’t be loaded. It brakes down with an std:bad_alloc and I can’t figure out where it comes from. This covers all versions of vision I built so far (0.23 – 0.25).