Hi, I’m deploying libtorch jit model on docker container, and i have succeed by code snnipet. But, when i move libtorch API based code to my c++ environment project, error occured as below.
Note: no matter i type “torch::set_num_threads(1);” or not , error still occured.
Could you run your script via gdb --args python script.py args and post the backtraces of all threads, please?
HI,here is the logs, and Context of the error code.
(gdb) r
Starting program: /root/workspace/apps/package-polaris/install/x64/bin/if-ads pnc
warning: Error disabling address space randomization: Operation not permitted
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
runs dir:/root/workspace/apps/package-polaris/install/x64/bin
bins dir:/root/workspace/apps/package-polaris/install/x64/bin
starting module pnc ...
mkdir ../log/ failed
EAXVA05_A
Begin LoadLibrary: libcontrol.so
node_name: control1
[New Thread 0x7f3ff1fb9700 (LWP 39453)]
[New Thread 0x7f3ff17b8700 (LWP 39454)]
[New Thread 0x7f3ff0fb7700 (LWP 39455)]
[New Thread 0x7f3ff07b6700 (LWP 39456)]
[New Thread 0x7f3feffb5700 (LWP 39457)]
[New Thread 0x7f3fef7b4700 (LWP 39458)]
[New Thread 0x7f3feefb3700 (LWP 39459)]
[New Thread 0x7f3fee7b2700 (LWP 39460)]
[New Thread 0x7f3fedfb1700 (LWP 39461)]
ControllerAgent Init Star!
Toml参数加载: 0.15
ControlComponent:Done
[New Thread 0x7f3fecfb2700 (LWP 39462)]
INFO : 2023-04-16 16:43:09.557:[TickFunc:170:39462] ##### TickFunc ENTRY PID[3975882496][39416] TID[39462], running at [OTHER/0], addr [0x7f3fecfaa540] #####
[New Thread 0x7f3fd7fff700 (LWP 39463)]
E20230416 16:43:09.558305 39462 posix_thread.h:241] set rt error: Operation not permitted, set rt policy: 2, priority max: 99, priority min: 1
[New Thread 0x7f3fd77fe700 (LWP 39464)]
E20230416 16:43:09.558712 39462 timing_wheel.cc:176] set timing wheel tick thread sched param failed
E20230416 16:43:09.559336 39463 thread_pool.cpp:88] success ----------------, ThreadPool[priority: 0(0-low, 1-mid,2-high)] set thread seched param, priority[: 0(0-low, 1-mid,2-high)], policy[: 0(0-other, 1-fifo, 2-rr)], thread num: 2
[New Thread 0x7f3fd6ffd700 (LWP 39465)]
E20230416 16:43:09.559391 39464 thread_pool.cpp:88] success ----------------, ThreadPool[priority: 0(0-low, 1-mid,2-high)] set thread seched param, priority[: 0(0-low, 1-mid,2-high)], policy[: 0(0-other, 1-fifo, 2-rr)], thread num: 2
[New Thread 0x7f3fd67fc700 (LWP 39466)]
E20230416 16:43:09.560139 39466 thread_pool.cpp:88] success ----------------, ThreadPool[priority: 1(0-low, 1-mid,2-high)] set thread seched param, priority[: 1(0-low, 1-mid,2-high)], policy[: 0(0-other, 1-fifo, 2-rr)], thread num: 1
[New Thread 0x7f3fd5ffb700 (LWP 39467)]
E20230416 16:43:09.560405 39467 thread_pool.cpp:88] success ----------------, ThreadPool[priority: 1(0-low, 1-mid,2-high)] set thread seched param, priority[: 1(0-low, 1-mid,2-high)], policy[: 0(0-other, 1-fifo, 2-rr)], thread num: 2
[New Thread 0x7f3fd57fa700 (LWP 39468)]
E20230416 16:43:09.560669 39468 thread_pool.cpp:88] success ----------------, ThreadPool[priority: 1(0-low, 1-mid,2-high)] set thread seched param, priority[: 1(0-low, 1-mid,2-high)], policy[: 0(0-other, 1-fifo, 2-rr)], thread num: 3
[New Thread 0x7f3fd4ff9700 (LWP 39469)]
E20230416 16:43:09.560891 39469 thread_pool.cpp:88] success ----------------, ThreadPool[priority: 1(0-low, 1-mid,2-high)] set thread seched param, priority[: 1(0-low, 1-mid,2-high)], policy[: 0(0-other, 1-fifo, 2-rr)], thread num: 4
[New Thread 0x7f3fcffff700 (LWP 39470)]
E20230416 16:43:09.561297 39470 posix_thread.h:241] set rt error: Operation not permitted, set rt policy: 2, priority max: 99, priority min: 1
E20230416 16:43:09.561339 39470 thread_pool.cpp:82] failed ----------------, ThreadPool[priority: 2(0-low, 1-mid,2-high)] set thread seched param, priority[: 2(0-low, 1-mid,2-high)], policy[: 2(0-other, 1-fifo, 2-rr)], thread num: 1
[New Thread 0x7f3fcf7fe700 (LWP 39471)]
There are still classobjs have not been deleted, classobj_ref_count_: 1
E20230416 16:43:09.561511 39471 posix_thread.h:241] set rt error: Operation not permitted, set rt policy: 2, priority max: 99, priority min: 1
E20230416 16:43:09.561553 39471 thread_pool.cpp:82] failed ----------------, ThreadPool[priority: 2(0-low, 1-mid,2-high)] set thread seched param, priority[: 2(0-low, 1-mid,2-high)], policy[: 2(0-other, 1-fifo, 2-rr)], thread num: 2
Begin LoadLibrary: libmodules_canbus.so
node_name: apollo_canbus2
[New Thread 0x7f3fcb54a700 (LWP 39512)]
[New Thread 0x7f3fcad49700 (LWP 39513)]
[New Thread 0x7f3fca548700 (LWP 39514)]
[New Thread 0x7f3fc9d47700 (LWP 39515)]
[New Thread 0x7f3fc9546700 (LWP 39516)]
[New Thread 0x7f3fc8d45700 (LWP 39517)]
[New Thread 0x7f3fb3fff700 (LWP 39518)]
[New Thread 0x7f3fb37fe700 (LWP 39519)]
[FastDdsQos:28]FastDdsQos Construct: default
[FastDdsQos:39]FastDdsQos Construct: xml_file[] participant_profile[] datawriter_profile[] datareader_profile[] topic_profile[]
[New Thread 0x7f3fb2ffd700 (LWP 39520)]
[New Thread 0x7f3fb27fc700 (LWP 39521)]
[New Thread 0x7f3fb1ffb700 (LWP 39522)]
[New Thread 0x7f3fb17fa700 (LWP 39523)]
[New Thread 0x7f3fb0ff9700 (LWP 39524)]
[New Thread 0x7f3f9bfff700 (LWP 39525)]
[New Thread 0x7f3f9b7fe700 (LWP 39526)]
[Thread 0x7f3fb27fc700 (LWP 39521) exited]
[Thread 0x7f3fb1ffb700 (LWP 39522) exited]
[Thread 0x7f3fb17fa700 (LWP 39523) exited]
[Thread 0x7f3fb0ff9700 (LWP 39524) exited]
[Thread 0x7f3f9bfff700 (LWP 39525) exited]
[Thread 0x7f3f9b7fe700 (LWP 39526) exited]
[Thread 0x7f3fb2ffd700 (LWP 39520) exited]
[FastDdsSubscriber:189]FastDdsSubscriber Construct: domain[0] topic[//apollo/control/v1]
[New Thread 0x7f3fb2ffd700 (LWP 39527)]
[New Thread 0x7f3f9b7fe700 (LWP 39528)]
[New Thread 0x7f3f9bfff700 (LWP 39529)]
[New Thread 0x7f3fb0ff9700 (LWP 39530)]
[New Thread 0x7f3fb27fc700 (LWP 39531)]
[New Thread 0x7f3fb1ffb700 (LWP 39532)]
[New Thread 0x7f3fb17fa700 (LWP 39533)]
[FastDdsQos:28]FastDdsQos Construct: default
[FastDdsQos:39]FastDdsQos Construct: xml_file[] participant_profile[] datawriter_profile[] datareader_profile[] topic_profile[]
[New Thread 0x7f3f9affd700 (LWP 39534)]
[New Thread 0x7f3f9a7fc700 (LWP 39535)]
[New Thread 0x7f3f99ffb700 (LWP 39536)]
[New Thread 0x7f3f997fa700 (LWP 39537)]
[New Thread 0x7f3f98ff9700 (LWP 39538)]
[New Thread 0x7f3f83fff700 (LWP 39539)]
[New Thread 0x7f3f837fe700 (LWP 39540)]
[Thread 0x7f3f9a7fc700 (LWP 39535) exited]
[Thread 0x7f3f99ffb700 (LWP 39536) exited]
[Thread 0x7f3f997fa700 (LWP 39537) exited]
[Thread 0x7f3f98ff9700 (LWP 39538) exited]
[Thread 0x7f3f83fff700 (LWP 39539) exited]
[Thread 0x7f3f837fe700 (LWP 39540) exited]
[Thread 0x7f3f9affd700 (LWP 39534) exited]
[FastDdsPublisher:148]FastDdsPublisher Construct: domain[0] topic[//apollo/canbus/chassis/v1]
[New Thread 0x7f3f9affd700 (LWP 39541)]
[New Thread 0x7f3f837fe700 (LWP 39542)]
[New Thread 0x7f3f83fff700 (LWP 39543)]
[New Thread 0x7f3f98ff9700 (LWP 39544)]
[New Thread 0x7f3f9a7fc700 (LWP 39545)]
[New Thread 0x7f3f99ffb700 (LWP 39546)]
[New Thread 0x7f3f997fa700 (LWP 39547)]
[~FastDdsSubscriber:230]FastDdsSubscriber Destruct: domain[0] topic[//apollo/control/v1]
[Thread 0x7f3f9b7fe700 (LWP 39528) exited]
[Thread 0x7f3f9bfff700 (LWP 39529) exited]
[Thread 0x7f3fb0ff9700 (LWP 39530) exited]
[Thread 0x7f3fb27fc700 (LWP 39531) exited]
[Thread 0x7f3fb1ffb700 (LWP 39532) exited]
[Thread 0x7f3fb17fa700 (LWP 39533) exited]
[Thread 0x7f3fb2ffd700 (LWP 39527) exited]
There are still classobjs have not been deleted, classobj_ref_count_: 1
Begin LoadLibrary: libmodules_localization.so
node_name: apollo_localization3
[New Thread 0x7f3fb2ffd700 (LWP 39549)]
[New Thread 0x7f3fb17fa700 (LWP 39550)]
[New Thread 0x7f3fb1ffb700 (LWP 39551)]
[New Thread 0x7f3fb27fc700 (LWP 39553)]
[New Thread 0x7f3f9bfff700 (LWP 39555)]
[New Thread 0x7f3f9b7fe700 (LWP 39556)]
[New Thread 0x7f3f82ffd700 (LWP 39557)]
[New Thread 0x7f3f827fc700 (LWP 39558)]
[FastDdsQos:28]FastDdsQos Construct: default
[FastDdsQos:39]FastDdsQos Construct: xml_file[] participant_profile[] datawriter_profile[] datareader_profile[] topic_profile[]
[New Thread 0x7f3f81ffb700 (LWP 39559)]
[New Thread 0x7f3f817fa700 (LWP 39560)]
[New Thread 0x7f3f80ff9700 (LWP 39561)]
[New Thread 0x7f3f6bfff700 (LWP 39562)]
[New Thread 0x7f3f6b7fe700 (LWP 39563)]
[New Thread 0x7f3f6affd700 (LWP 39564)]
[New Thread 0x7f3f6a7fc700 (LWP 39565)]
[Thread 0x7f3f817fa700 (LWP 39560) exited]
[Thread 0x7f3f80ff9700 (LWP 39561) exited]
[Thread 0x7f3f6bfff700 (LWP 39562) exited]
[Thread 0x7f3f6b7fe700 (LWP 39563) exited]
[Thread 0x7f3f6affd700 (LWP 39564) exited]
[Thread 0x7f3f6a7fc700 (LWP 39565) exited]
[Thread 0x7f3f81ffb700 (LWP 39559) exited]
[FastDdsPublisher:148]FastDdsPublisher Construct: domain[0] topic[//apollo/localization/pose/v1]
[New Thread 0x7f3f81ffb700 (LWP 39566)]
[New Thread 0x7f3f6a7fc700 (LWP 39567)]
[New Thread 0x7f3f6affd700 (LWP 39568)]
[New Thread 0x7f3f6b7fe700 (LWP 39569)]
[New Thread 0x7f3f817fa700 (LWP 39570)]
[New Thread 0x7f3f80ff9700 (LWP 39571)]
[New Thread 0x7f3f6bfff700 (LWP 39572)]
[FastDdsQos:28]FastDdsQos Construct: default
[FastDdsQos:39]FastDdsQos Construct: xml_file[] participant_profile[] datawriter_profile[] datareader_profile[] topic_profile[]
[New Thread 0x7f3f69ffb700 (LWP 39573)]
[New Thread 0x7f3f697fa700 (LWP 39574)]
[New Thread 0x7f3f68ff9700 (LWP 39575)]
[New Thread 0x7f3f53fff700 (LWP 39576)]
[New Thread 0x7f3f537fe700 (LWP 39577)]
[New Thread 0x7f3f52ffd700 (LWP 39578)]
[New Thread 0x7f3f527fc700 (LWP 39579)]
[Thread 0x7f3f697fa700 (LWP 39574) exited]
[Thread 0x7f3f53fff700 (LWP 39576) exited]
[Thread 0x7f3f68ff9700 (LWP 39575) exited]
[Thread 0x7f3f537fe700 (LWP 39577) exited]
[Thread 0x7f3f52ffd700 (LWP 39578) exited]
[Thread 0x7f3f527fc700 (LWP 39579) exited]
[Thread 0x7f3f69ffb700 (LWP 39573) exited]
[FastDdsPublisher:148]FastDdsPublisher Construct: domain[0] topic[//apollo/localization/msf_gnss/v1]
[New Thread 0x7f3f69ffb700 (LWP 39580)]
[New Thread 0x7f3f527fc700 (LWP 39581)]
[New Thread 0x7f3f52ffd700 (LWP 39582)]
[New Thread 0x7f3f537fe700 (LWP 39583)]
[New Thread 0x7f3f697fa700 (LWP 39584)]
[New Thread 0x7f3f68ff9700 (LWP 39585)]
[New Thread 0x7f3f53fff700 (LWP 39586)]
There are still classobjs have not been deleted, classobj_ref_count_: 1
Begin LoadLibrary: libmodules_planning.so
node_name: apollo_planning4
[New Thread 0x7f3f1ea21700 (LWP 39680)]
[New Thread 0x7f3f1e220700 (LWP 39681)]
[New Thread 0x7f3f1da1f700 (LWP 39682)]
[New Thread 0x7f3f1d21e700 (LWP 39683)]
[New Thread 0x7f3f1ca1d700 (LWP 39684)]
[New Thread 0x7f3f1c21c700 (LWP 39685)]
[New Thread 0x7f3f1ba1b700 (LWP 39686)]
[New Thread 0x7f3f1b21a700 (LWP 39687)]
CanbusComponent, Proc:
[New Thread 0x7f3f1aa19700 (LWP 39688)]
[FastDdsQos:28]FastDdsQos Construct: default
[FastDdsQos:39]FastDdsQos Construct: xml_file[] participant_profile[] datawriter_profile[] datareader_profile[] topic_profile[]
[New Thread 0x7f3f1a218700 (LWP 39689)]
[New Thread 0x7f3f19a17700 (LWP 39690)]
[New Thread 0x7f3f19216700 (LWP 39691)]
[New Thread 0x7f3f18a15700 (LWP 39692)]
[New Thread 0x7f3f0bfff700 (LWP 39693)]
[New Thread 0x7f3f0b7fe700 (LWP 39694)]
[New Thread 0x7f3f0affd700 (LWP 39695)]
[Thread 0x7f3f19a17700 (LWP 39690) exited]
[Thread 0x7f3f19216700 (LWP 39691) exited]
[Thread 0x7f3f18a15700 (LWP 39692) exited]
[Thread 0x7f3f0bfff700 (LWP 39693) exited]
[Thread 0x7f3f0b7fe700 (LWP 39694) exited]
[Thread 0x7f3f0affd700 (LWP 39695) exited]
[Thread 0x7f3f1a218700 (LWP 39689) exited]
[FastDdsSubscriber:189]FastDdsSubscriber Construct: domain[0] topic[//apollo/prediction/v1]
[New Thread 0x7f3f1a218700 (LWP 39696)]
[New Thread 0x7f3f0affd700 (LWP 39697)]
[New Thread 0x7f3f0b7fe700 (LWP 39698)]
[New Thread 0x7f3f0bfff700 (LWP 39699)]
[New Thread 0x7f3f19a17700 (LWP 39700)]
[New Thread 0x7f3f19216700 (LWP 39701)]
[New Thread 0x7f3f18a15700 (LWP 39702)]
[FastDdsQos:28]FastDdsQos Construct: default
[FastDdsQos:39]FastDdsQos Construct: xml_file[] participant_profile[] datawriter_profile[] datareader_profile[] topic_profile[]
[New Thread 0x7f3f0a6ea700 (LWP 39703)]
[New Thread 0x7f3f09ee9700 (LWP 39704)]
[New Thread 0x7f3f095d6700 (LWP 39705)]
[New Thread 0x7f3f08c3a700 (LWP 39706)]
[New Thread 0x7f3eebfff700 (LWP 39707)]
[New Thread 0x7f3eeb7fe700 (LWP 39708)]
[New Thread 0x7f3eeaffd700 (LWP 39709)]
[Thread 0x7f3f09ee9700 (LWP 39704) exited]
[Thread 0x7f3f095d6700 (LWP 39705) exited]
[Thread 0x7f3f08c3a700 (LWP 39706) exited]
[Thread 0x7f3eebfff700 (LWP 39707) exited]
[Thread 0x7f3eeb7fe700 (LWP 39708) exited]
[Thread 0x7f3eeaffd700 (LWP 39709) exited]
[Thread 0x7f3f0a6ea700 (LWP 39703) exited]
[FastDdsSubscriber:189]FastDdsSubscriber Construct: domain[0] topic[//apollo/canbus/chassis/v1]
[New Thread 0x7f3f0a6ea700 (LWP 39710)]
[New Thread 0x7f3eeaffd700 (LWP 39711)]
[New Thread 0x7f3eeb7fe700 (LWP 39712)]
[New Thread 0x7f3eebfff700 (LWP 39713)]
[New Thread 0x7f3f09ee9700 (LWP 39714)]
[New Thread 0x7f3f095d6700 (LWP 39715)]
[New Thread 0x7f3f08c3a700 (LWP 39716)]
[FastDdsQos:28]FastDdsQos Construct: default
[FastDdsQos:39]FastDdsQos Construct: xml_file[] participant_profile[] datawriter_profile[] datareader_profile[] topic_profile[]
[New Thread 0x7f3ee9dd1700 (LWP 39717)]
[New Thread 0x7f3ee94be700 (LWP 39718)]
[New Thread 0x7f3ee8c34700 (LWP 39719)]
[New Thread 0x7f3ed3fff700 (LWP 39720)]
[New Thread 0x7f3ed3775700 (LWP 39721)]
[New Thread 0x7f3ed2e62700 (LWP 39722)]
[New Thread 0x7f3ed25d8700 (LWP 39723)]
[on_subscription_matched:72]FastDdsSubscriber //apollo/canbus/chassis/v1 SubscriptionMatchedStatus: current matched count is increased. total_count:1
[on_publication_matched:99]FastDdsPublisher //apollo/canbus/chassis/v1 PublicationMatchedStatus: current matched count is increased. total_count:1
[Thread 0x7f3ee94be700 (LWP 39718) exited]
[Thread 0x7f3ee8c34700 (LWP 39719) exited]
[Thread 0x7f3ed3fff700 (LWP 39720) exited]
[Thread 0x7f3ed3775700 (LWP 39721) exited]
[Thread 0x7f3ed2e62700 (LWP 39722) exited]
[Thread 0x7f3ed25d8700 (LWP 39723) exited]
[Thread 0x7f3ee9dd1700 (LWP 39717) exited]
[FastDdsSubscriber:189]FastDdsSubscriber Construct: domain[0] topic[//apollo/localization/pose/v1]
[New Thread 0x7f3ee9dd1700 (LWP 39724)]
[New Thread 0x7f3ed25d8700 (LWP 39725)]
[New Thread 0x7f3ed2e62700 (LWP 39726)]
[New Thread 0x7f3ed3775700 (LWP 39727)]
[New Thread 0x7f3ee94be700 (LWP 39728)]
[New Thread 0x7f3ee8c34700 (LWP 39729)]
[New Thread 0x7f3ed3fff700 (LWP 39730)]
[FastDdsQos:28]FastDdsQos Construct: default
[FastDdsQos:39]FastDdsQos Construct: xml_file[] participant_profile[] datawriter_profile[] datareader_profile[] topic_profile[]
[New Thread 0x7f3ed1435700 (LWP 39731)]
[New Thread 0x7f3ed0bab700 (LWP 39732)]
[New Thread 0x7f3ebbfff700 (LWP 39733)]
[New Thread 0x7f3ebb7fe700 (LWP 39734)]
[New Thread 0x7f3ebaeeb700 (LWP 39735)]
[New Thread 0x7f3eba661700 (LWP 39736)]
[on_publication_matched:99]FastDdsPublisher //apollo/localization/pose/v1 PublicationMatchedStatus: current matched count is increased. total_count:1
[New Thread 0x7f3eb9e60700 (LWP 39737)]
[Thread 0x7f3ed0bab700 (LWP 39732) exited]
[Thread 0x7f3ebbfff700 (LWP 39733) exited]
[Thread 0x7f3ebb7fe700 (LWP 39734) exited]
[Thread 0x7f3ebaeeb700 (LWP 39735) exited]
[Thread 0x7f3eba661700 (LWP 39736) exited]
[Thread 0x7f3eb9e60700 (LWP 39737) exited]
[Thread 0x7f3ed1435700 (LWP 39731) exited]
[FastDdsSubscriber:189]FastDdsSubscriber Construct: domain[0] topic[//apollo/routing_response/v1]
[New Thread 0x7f3ed1435700 (LWP 39738)]
[New Thread 0x7f3eb9e60700 (LWP 39739)]
[New Thread 0x7f3eba661700 (LWP 39740)]
[New Thread 0x7f3ebaeeb700 (LWP 39742)]
[New Thread 0x7f3ebbfff700 (LWP 39743)]
[New Thread 0x7f3ebb7fe700 (LWP 39746)]
[New Thread 0x7f3eb8c83700 (LWP 39747)]
[FastDdsQos:28]FastDdsQos Construct: default
[FastDdsQos:39]FastDdsQos Construct: xml_file[] participant_profile[] datawriter_profile[] datareader_profile[] topic_profile[]
[New Thread 0x7f3ea7fff700 (LWP 39748)]
[New Thread 0x7f3ea77fe700 (LWP 39749)]
[New Thread 0x7f3ea6ffd700 (LWP 39750)]
[New Thread 0x7f3ea67fc700 (LWP 39751)]
[New Thread 0x7f3ea5ffb700 (LWP 39752)]
[New Thread 0x7f3ea57fa700 (LWP 39753)]
Location, Proc:
[New Thread 0x7f3ea4f70700 (LWP 39754)]
[Thread 0x7f3ea77fe700 (LWP 39749) exited]
[Thread 0x7f3ea6ffd700 (LWP 39750) exited]
[Thread 0x7f3ea67fc700 (LWP 39751) exited]
[Thread 0x7f3ea5ffb700 (LWP 39752) exited]
[Thread 0x7f3ea57fa700 (LWP 39753) exited]
[Thread 0x7f3ea4f70700 (LWP 39754) exited]
[Thread 0x7f3ea7fff700 (LWP 39748) exited]
[FastDdsSubscriber:189]FastDdsSubscriber Construct: domain[0] topic[//apollo/perception/traffic_light/v1]
[New Thread 0x7f3ea7fff700 (LWP 39755)]
[New Thread 0x7f3ea4f70700 (LWP 39756)]
[New Thread 0x7f3ea57fa700 (LWP 39757)]
[New Thread 0x7f3ea5ffb700 (LWP 39758)]
[New Thread 0x7f3ea7775700 (LWP 39759)]
[New Thread 0x7f3ea6f74700 (LWP 39760)]
[New Thread 0x7f3e83fff700 (LWP 39761)]
[FastDdsQos:28]FastDdsQos Construct: default
[FastDdsQos:39]FastDdsQos Construct: xml_file[] participant_profile[] datawriter_profile[] datareader_profile[] topic_profile[]
[New Thread 0x7f3e837fe700 (LWP 39762)]
[on_subscription_matched:72]FastDdsSubscriber //apollo/localization/pose/v1 SubscriptionMatchedStatus: current matched count is increased. total_count:1
[New Thread 0x7f3e82ffd700 (LWP 39763)]
[New Thread 0x7f3e827fc700 (LWP 39764)]
[New Thread 0x7f3e81ffb700 (LWP 39765)]
[New Thread 0x7f3e817fa700 (LWP 39766)]
[New Thread 0x7f3e80e5e700 (LWP 39767)]
[New Thread 0x7f3e805d4700 (LWP 39768)]
[Thread 0x7f3e82ffd700 (LWP 39763) exited]
[Thread 0x7f3e827fc700 (LWP 39764) exited]
[Thread 0x7f3e81ffb700 (LWP 39765) exited]
[Thread 0x7f3e817fa700 (LWP 39766) exited]
[Thread 0x7f3e80e5e700 (LWP 39767) exited]
[Thread 0x7f3e805d4700 (LWP 39768) exited]
[Thread 0x7f3e837fe700 (LWP 39762) exited]
[FastDdsSubscriber:189]FastDdsSubscriber Construct: domain[0] topic[//apollo/planning/pad/v1]
[New Thread 0x7f3e837fe700 (LWP 39769)]
[New Thread 0x7f3e805d4700 (LWP 39770)]
[New Thread 0x7f3e80e5e700 (LWP 39771)]
[New Thread 0x7f3e817fa700 (LWP 39772)]
[New Thread 0x7f3e82ffd700 (LWP 39773)]
[New Thread 0x7f3e827fc700 (LWP 39774)]
[New Thread 0x7f3e81ffb700 (LWP 39775)]
[FastDdsQos:28]FastDdsQos Construct: default
[FastDdsQos:39]FastDdsQos Construct: xml_file[] participant_profile[] datawriter_profile[] datareader_profile[] topic_profile[]
[New Thread 0x7f3e7f086700 (LWP 39776)]
[New Thread 0x7f3e7e773700 (LWP 39777)]
[New Thread 0x7f3e7de60700 (LWP 39778)]
[New Thread 0x7f3e7d4c4700 (LWP 39779)]
[New Thread 0x7f3e7cc3a700 (LWP 39780)]
[New Thread 0x7f3e67fff700 (LWP 39781)]
[New Thread 0x7f3e677fe700 (LWP 39782)]
[Thread 0x7f3e7e773700 (LWP 39777) exited]
[Thread 0x7f3e7de60700 (LWP 39778) exited]
[Thread 0x7f3e7d4c4700 (LWP 39779) exited]
[Thread 0x7f3e67fff700 (LWP 39781) exited]
[Thread 0x7f3e7cc3a700 (LWP 39780) exited]
[Thread 0x7f3e677fe700 (LWP 39782) exited]
[Thread 0x7f3e7f086700 (LWP 39776) exited]
[FastDdsSubscriber:189]FastDdsSubscriber Construct: domain[0] topic[//apollo/storytelling/v1]
[New Thread 0x7f3e7f086700 (LWP 39783)]
[New Thread 0x7f3e677fe700 (LWP 39784)]
[New Thread 0x7f3e67fff700 (LWP 39785)]
[New Thread 0x7f3e7cc3a700 (LWP 39786)]
[New Thread 0x7f3e7de60700 (LWP 39787)]
[New Thread 0x7f3e7d4c4700 (LWP 39788)]
[New Thread 0x7f3e66c38700 (LWP 39789)]
[FastDdsQos:28]FastDdsQos Construct: default
[FastDdsQos:39]FastDdsQos Construct: xml_file[] participant_profile[] datawriter_profile[] datareader_profile[] topic_profile[]
[New Thread 0x7f3e66437700 (LWP 39790)]
[New Thread 0x7f3e65c36700 (LWP 39791)]
[New Thread 0x7f3e653ac700 (LWP 39792)]
[New Thread 0x7f3e648fe700 (LWP 39793)]
[New Thread 0x7f3e4ff76700 (LWP 39794)]
[New Thread 0x7f3e4f4c8700 (LWP 39795)]
[New Thread 0x7f3e4eb2c700 (LWP 39796)]
[Thread 0x7f3e65c36700 (LWP 39791) exited]
[Thread 0x7f3e653ac700 (LWP 39792) exited]
[Thread 0x7f3e648fe700 (LWP 39793) exited]
[Thread 0x7f3e4f4c8700 (LWP 39795) exited]
[Thread 0x7f3e4ff76700 (LWP 39794) exited]
[Thread 0x7f3e4eb2c700 (LWP 39796) exited]
[Thread 0x7f3e66437700 (LWP 39790) exited]
[FastDdsPublisher:148]FastDdsPublisher Construct: domain[0] topic[//apollo/planning/v1]
[New Thread 0x7f3e66437700 (LWP 39797)]
[New Thread 0x7f3e4eb2c700 (LWP 39798)]
[New Thread 0x7f3e4f4c8700 (LWP 39799)]
[New Thread 0x7f3e4ff76700 (LWP 39800)]
[New Thread 0x7f3e65c36700 (LWP 39801)]
[New Thread 0x7f3e648fe700 (LWP 39802)]
[New Thread 0x7f3e4d997700 (LWP 39803)]
[FastDdsQos:28]FastDdsQos Construct: default
[FastDdsQos:39]FastDdsQos Construct: xml_file[] participant_profile[] datawriter_profile[] datareader_profile[] topic_profile[]
[New Thread 0x7f3e4d084700 (LWP 39805)]
[New Thread 0x7f3e4c883700 (LWP 39806)]
[New Thread 0x7f3e37f76700 (LWP 39807)]
[New Thread 0x7f3e374c8700 (LWP 39808)]
[New Thread 0x7f3e36b2c700 (LWP 39809)]
[New Thread 0x7f3e36107700 (LWP 39812)]
[New Thread 0x7f3e3587d700 (LWP 39813)]
[Thread 0x7f3e4c883700 (LWP 39806) exited]
[Thread 0x7f3e37f76700 (LWP 39807) exited]
[Thread 0x7f3e374c8700 (LWP 39808) exited]
[Thread 0x7f3e36107700 (LWP 39812) exited]
[Thread 0x7f3e36b2c700 (LWP 39809) exited]
[Thread 0x7f3e3587d700 (LWP 39813) exited]
[Thread 0x7f3e4d084700 (LWP 39805) exited]
[FastDdsPublisher:148]FastDdsPublisher Construct: domain[0] topic[//apollo/routing_request/v1]
[New Thread 0x7f3e4d084700 (LWP 39814)]
[New Thread 0x7f3e3587d700 (LWP 39815)]
[New Thread 0x7f3e36107700 (LWP 39816)]
[New Thread 0x7f3e36b2c700 (LWP 39817)]
[New Thread 0x7f3e4c883700 (LWP 39818)]
[New Thread 0x7f3e37f76700 (LWP 39819)]
[New Thread 0x7f3e374c8700 (LWP 39820)]
[FastDdsQos:28]FastDdsQos Construct: default
[FastDdsQos:39]FastDdsQos Construct: xml_file[] participant_profile[] datawriter_profile[] datareader_profile[] topic_profile[]
[New Thread 0x7f3e27f76700 (LWP 39821)]
[New Thread 0x7f3e276ec700 (LWP 39822)]
[New Thread 0x7f3e26d50700 (LWP 39823)]
[New Thread 0x7f3e262a2700 (LWP 39824)]
[New Thread 0x7f3e25a18700 (LWP 39825)]
[New Thread 0x7f3e24fcc700 (LWP 39827)]
[New Thread 0x7f3e1ffff700 (LWP 39828)]
[Thread 0x7f3e276ec700 (LWP 39822) exited]
[Thread 0x7f3e26d50700 (LWP 39823) exited]
[Thread 0x7f3e262a2700 (LWP 39824) exited]
[Thread 0x7f3e25a18700 (LWP 39825) exited]
[Thread 0x7f3e24fcc700 (LWP 39827) exited]
[Thread 0x7f3e1ffff700 (LWP 39828) exited]
[Thread 0x7f3e27f76700 (LWP 39821) exited]
[FastDdsPublisher:148]FastDdsPublisher Construct: domain[0] topic[//apollo/planning/learning_data/v1]
[New Thread 0x7f3e27f76700 (LWP 39831)]
[New Thread 0x7f3e1ffff700 (LWP 39832)]
[New Thread 0x7f3e24fcc700 (LWP 39833)]
[New Thread 0x7f3e25a18700 (LWP 39834)]
[New Thread 0x7f3e276ec700 (LWP 39835)]
[New Thread 0x7f3e262a2700 (LWP 39836)]
[New Thread 0x7f3e1f7fe700 (LWP 39837)]
There are still classobjs have not been deleted, classobj_ref_count_: 1
Begin LoadLibrary: libmodules_routing.so
node_name: apollo_routing5
[New Thread 0x7f3e1dfcb700 (LWP 39842)]
[New Thread 0x7f3e1d7ca700 (LWP 39843)]
[New Thread 0x7f3e1cfc9700 (LWP 39844)]
[New Thread 0x7f3e03fff700 (LWP 39845)]
[New Thread 0x7f3e037fe700 (LWP 39846)]
[New Thread 0x7f3e02ffd700 (LWP 39847)]
[New Thread 0x7f3e027fc700 (LWP 39848)]
[New Thread 0x7f3e01ffb700 (LWP 39849)]
[FastDdsQos:28]FastDdsQos Construct: default
[FastDdsQos:39]FastDdsQos Construct: xml_file[] participant_profile[] datawriter_profile[] datareader_profile[] topic_profile[]
[New Thread 0x7f3e017fa700 (LWP 39850)]
[New Thread 0x7f3e00ff9700 (LWP 39851)]
[New Thread 0x7f3debfff700 (LWP 39852)]
[New Thread 0x7f3deb7fe700 (LWP 39853)]
[New Thread 0x7f3deaffd700 (LWP 39854)]
[New Thread 0x7f3dea7fc700 (LWP 39855)]
[New Thread 0x7f3de9ffb700 (LWP 39856)]
CanbusComponent, Proc:
[Thread 0x7f3e00ff9700 (LWP 39851) exited]
[Thread 0x7f3debfff700 (LWP 39852) exited]
[Thread 0x7f3deb7fe700 (LWP 39853) exited]
[Thread 0x7f3deaffd700 (LWP 39854) exited]
[Thread 0x7f3dea7fc700 (LWP 39855) exited]
[Thread 0x7f3de9ffb700 (LWP 39856) exited]
[Thread 0x7f3e017fa700 (LWP 39850) exited]
[FastDdsSubscriber:189]FastDdsSubscriber Construct: domain[0] topic[//apollo/routing_request/v1]
[New Thread 0x7f3e017fa700 (LWP 39857)]
[New Thread 0x7f3de9ffb700 (LWP 39858)]
[New Thread 0x7f3dea7fc700 (LWP 39859)]
[New Thread 0x7f3deaffd700 (LWP 39860)]
[New Thread 0x7f3e00ff9700 (LWP 39861)]
[New Thread 0x7f3debfff700 (LWP 39862)]
[New Thread 0x7f3deb7fe700 (LWP 39863)]
[FastDdsQos:28]FastDdsQos Construct: default
[FastDdsQos:39]FastDdsQos Construct: xml_file[] participant_profile[] datawriter_profile[] datareader_profile[] topic_profile[]
[New Thread 0x7f3de97fa700 (LWP 39864)]
[New Thread 0x7f3de8ff9700 (LWP 39865)]
[New Thread 0x7f3dd3fff700 (LWP 39866)]
[New Thread 0x7f3dd37fe700 (LWP 39867)]
[New Thread 0x7f3dd2ffd700 (LWP 39868)]
[New Thread 0x7f3dd27fc700 (LWP 39869)]
[New Thread 0x7f3dd1ffb700 (LWP 39870)]
[Thread 0x7f3de8ff9700 (LWP 39865) exited]
[Thread 0x7f3dd3fff700 (LWP 39866) exited]
[Thread 0x7f3dd37fe700 (LWP 39867) exited]
[Thread 0x7f3dd27fc700 (LWP 39869) exited]
[Thread 0x7f3dd2ffd700 (LWP 39868) exited]
[Thread 0x7f3dd1ffb700 (LWP 39870) exited]
[Thread 0x7f3de97fa700 (LWP 39864) exited]
[FastDdsPublisher:148]FastDdsPublisher Construct: domain[0] topic[//apollo/routing_response/v1]
[on_subscription_matched:72]FastDdsSubscriber //apollo/routing_request/v1 SubscriptionMatchedStatus: current matched count is increased. total_count:1
[on_publication_matched:99]FastDdsPublisher //apollo/routing_request/v1 PublicationMatchedStatus: current matched count is increased. total_count:1
[on_subscription_matched:72]FastDdsSubscriber //apollo/routing_request/v1 SubscriptionMatchedStatus: current matched count is increased. total_count:2
[New Thread 0x7f3de97fa700 (LWP 39871)]
[New Thread 0x7f3dd1ffb700 (LWP 39872)]
[New Thread 0x7f3dd27fc700 (LWP 39873)]
[New Thread 0x7f3dd2ffd700 (LWP 39874)]
[New Thread 0x7f3de8ff9700 (LWP 39875)]
[New Thread 0x7f3dd3fff700 (LWP 39876)]
[New Thread 0x7f3dd37fe700 (LWP 39877)]
[FastDdsQos:28]FastDdsQos Construct: default
[FastDdsQos:39]FastDdsQos Construct: xml_file[] participant_profile[] datawriter_profile[] datareader_profile[] topic_profile[]
[New Thread 0x7f3dd0f6a700 (LWP 39878)]
[New Thread 0x7f3dbbfff700 (LWP 39879)]
[New Thread 0x7f3dbb7fe700 (LWP 39880)]
[New Thread 0x7f3dbaf74700 (LWP 39881)]
[New Thread 0x7f3dba661700 (LWP 39882)]
[New Thread 0x7f3db9d34700 (LWP 39883)]
[New Thread 0x7f3db937e700 (LWP 39884)]
[Thread 0x7f3dbbfff700 (LWP 39879) exited]
[Thread 0x7f3dbb7fe700 (LWP 39880) exited]
[Thread 0x7f3dbaf74700 (LWP 39881) exited]
[Thread 0x7f3db9d34700 (LWP 39883) exited]
[Thread 0x7f3dba661700 (LWP 39882) exited]
[Thread 0x7f3db937e700 (LWP 39884) exited]
[Thread 0x7f3dd0f6a700 (LWP 39878) exited]
[FastDdsPublisher:148]FastDdsPublisher Construct: domain[0] topic[//apollo/routing_response_history/v1]
[New Thread 0x7f3dd0f6a700 (LWP 39885)]
[New Thread 0x7f3db937e700 (LWP 39886)]
[on_subscription_matched:72]FastDdsSubscriber //apollo/routing_response/v1 SubscriptionMatchedStatus: current matched count is increased. total_count:1
[New Thread 0x7f3db9d34700 (LWP 39887)]
[on_publication_matched:99]FastDdsPublisher //apollo/routing_response/v1 PublicationMatchedStatus: current matched count is increased. total_count:1
[New Thread 0x7f3dba661700 (LWP 39889)]
[New Thread 0x7f3dbbec6700 (LWP 39890)]
[New Thread 0x7f3dbaf74700 (LWP 39893)]
[New Thread 0x7f3da7fff700 (LWP 39894)]
There are still classobjs have not been deleted, classobj_ref_count_: 1
Begin LoadLibrary: libmodules_prediction.so
Location, Proc:
E20230416 16:43:25.183076 39467 planning_component.cc:371] []routing not ready; skip the planning cycle.
[Publish:283]send data size is 31
[Publish:288]send data size is [31]
[Publish:294]FastDdsPublisher pub data is successed!
E20230416 16:43:25.184424 39467 planning_component.cc:259] []Input check failed
CanbusComponent, Proc:
RoutingComponent::Proc()_start
Location, Proc:
E20230416 16:43:26.181955 39468 pnc_map.cc:269] []Route is empty.
E20230416 16:43:26.182494 39468 planning_component.cc:371] []routing not ready; skip the planning cycle.
[Publish:283]send data size is 57
[Publish:288]send data size is [57]
[Publish:294]FastDdsPublisher pub data is successed!
E20230416 16:43:26.182761 39468 planning_component.cc:259] []Input check failed
CanbusComponent, Proc:
RoutingComponent::Proc()_start
Location, Proc:
E20230416 16:43:27.181689 39469 pnc_map.cc:269] []Route is empty.
E20230416 16:43:27.182014 39469 planning_component.cc:371] []routing not ready; skip the planning cycle.
[Publish:283]send data size is 57
[Publish:288]send data size is [57]
[Publish:294]FastDdsPublisher pub data is successed!
E20230416 16:43:27.182266 39469 planning_component.cc:259] []Input check failed
CanbusComponent, Proc:
node_name: apollo_prediction6
[New Thread 0x7f3d934df700 (LWP 39919)]
[New Thread 0x7f3d92cde700 (LWP 39920)]
[New Thread 0x7f3d924dd700 (LWP 39921)]
[New Thread 0x7f3d91cdc700 (LWP 39922)]
[New Thread 0x7f3d914db700 (LWP 39923)]
[New Thread 0x7f3d90cda700 (LWP 39924)]
[New Thread 0x7f3d904d9700 (LWP 39925)]
[New Thread 0x7f3d8fcd8700 (LWP 39926)]
PredictionComponent::Init in!
E20230416 16:43:27.931082 39416 hdmap_util.cc:40] []No existing file found in ../conf/configfile/map/data/lx/sim_map.bin|sim_map.txt. Fallback to first candidate as default result.
E20230416 16:43:27.931314 39416 file.cc:100] []Failed to open file ../conf/configfile/map/data/lx/sim_map.bin in binary mode.
E20230416 16:43:27.931334 39416 file.cc:74] []Failed to open file ../conf/configfile/map/data/lx/sim_map.bin in text mode.
E20230416 16:43:27.931353 39416 hdmap_util.cc:75] []Failed to load HDMap ../conf/configfile/map/data/lx/sim_map.bin
Prediction:RegisterEvaluators::Init-11
E20230416 16:43:27.939330 39416 hdmap_util.cc:40] []No existing file found in ../conf/configfile/map/data/lx/sim_map.bin|sim_map.txt. Fallback to first candidate as default result.
E20230416 16:43:27.939487 39416 file.cc:100] []Failed to open file ../conf/configfile/map/data/lx/sim_map.bin in binary mode.
E20230416 16:43:27.939503 39416 file.cc:74] []Failed to open file ../conf/configfile/map/data/lx/sim_map.bin in text mode.
E20230416 16:43:27.939515 39416 hdmap_util.cc:75] []Failed to load HDMap ../conf/configfile/map/data/lx/sim_map.bin
Prediction:RegisterEvaluators::Init-12
E20230416 16:43:27.946803 39416 hdmap_util.cc:40] []No existing file found in ../conf/configfile/map/data/lx/sim_map.bin|sim_map.txt. Fallback to first candidate as default result.
E20230416 16:43:27.946965 39416 file.cc:100] []Failed to open file ../conf/configfile/map/data/lx/sim_map.bin in binary mode.
E20230416 16:43:27.946981 39416 file.cc:74] []Failed to open file ../conf/configfile/map/data/lx/sim_map.bin in text mode.
E20230416 16:43:27.946993 39416 hdmap_util.cc:75] []Failed to load HDMap ../conf/configfile/map/data/lx/sim_map.bin
Prediction
=============================
start load jit model:/root/workspace/apps/my_model_conversion/model/iknownet_jit_script_pytorch1.12.1.pt
RoutingComponent::Proc()_start
Location, Proc:
[23, 16, 3]
start inferenc:/root/workspace/apps/my_model_conversion/model/iknownet_jit_script_pytorch1.12.1.pt
E20230416 16:43:28.181658 39466 pnc_map.cc:269] []Route is empty.
E20230416 16:43:28.182022 39466 planning_component.cc:371] []routing not ready; skip the planning cycle.
[Publish:283]send data size is 57
[Publish:288]send data size is [57]
[Publish:294]FastDdsPublisher pub data is successed!
E20230416 16:43:28.182356 39466 planning_component.cc:259] []Input check failed
Thread 1 "if-ads" received signal SIGSEGV, Segmentation fault.
__memset_avx2_erms () at ../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S:145
145 ../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S: No such file or directory.
(gdb)
You didn’t post any backtraces. I would also recommend posting code snippets directly by wrapping them into three backticks ``` instead of images.
How about this time’s log ?
There is no problem running the code snippet alone in a single project, but when integrated into the target project, the error occurs. i have check setting of subdirectory's cmakelist.txt file, there is no difference.
code snippets
void IKnownetEvaluator::LoadModel() {
const std::string model_path= "/root/workspace/apps/my_model_conversion/model/iknownet_jit_script_pytorch1.12.1.pt";//模型路径
std::cout<<"Prediction\n============================="<<std::endl;
//sleep(10);
test(model_path);
std::cout<<"Prediction\n============================="<<std::endl;
//sleep(10);
}
void IKnownetEvaluator::test(std::string model_path)
{
int iact_num = 23;
int imap_num = 2;
torch::Device device_(torch::kCPU);
torch::jit::script::Module module;// = torch::jit::load(model_path);
if (torch::cuda::is_available())
{
std::cout << "CUDA is available" << std::endl;
device_ = torch::Device(torch::kCUDA);
module =
torch::jit::load(model_path, device_);
} else {
std::cout <<"start load jit model:" << model_path <<std::endl;
module =
torch::jit::load(model_path, device_);
}
// 3.0 data packing
//torch::set_num_threads(1);
// actor_data: Dict[str, List[Tensor]] [batch_size * [num_agents, num_hist, dim] ]
std::unordered_map<std::string, std::vector<torch::Tensor>> actors_data;
actors_data["types_int"] = {1, torch::randn({iact_num, 1}).to(device_)}; // [5, 1]
actors_data["history_speeds"] = {1, torch::randn({iact_num, 16, 2}).to(device_)}; // [5, 16, 2]
actors_data["history_headings"] = {1, torch::randn({iact_num, 16, 1}).to(device_)}; // [5, 16, 1]
actors_data["history_trajs"] = {1, torch::randn({iact_num, 16, 1}).to(device_)}; // [5, 16, 3]
actors_data["history_trajs_relative"] = {1, torch::randn({iact_num, 16, 3}).to(device_)}; // [5, 16, 3]
std::cout<<actors_data["history_trajs_relative"][0].sizes()<<std::endl; // batch 0 [12, 16, 3]
// lane_data: Dict[str, List[List[Tensor]]] [batch_size * [num_lanes * [num_points, dim] ] ]
std::unordered_map<std::string, std::vector<std::vector<torch::Tensor>>> lanes_data; //[0][32, 6] [1][32, 6, 60, 2]
lanes_data["centerlines"] = {imap_num, {torch::randn({10,2}).to(device_), torch::randn({10,2}).to(device_), torch::randn({10,2}).to(device_)}};
std::vector<torch::jit::IValue> inputs;
inputs.push_back(actors_data);
inputs.push_back(lanes_data);
std::cout <<"start inferenc:" << model_path <<std::endl;
auto outputs = module.forward(inputs).toTuple();
std::cout <<"end inferenc:" << model_path <<std::endl;
torch::Tensor trajs_prob = outputs->elements()[0].toTensor();
//std::cout << outputs.device() << std::endl;
std::cout << trajs_prob.device() << std::endl;
torch::Tensor trajs_pred = outputs->elements()[1].toTensor();
std::cout<<"probabilites: "<<trajs_prob.sizes()<<std::endl; // [2, 6]
std::cout<<"trajectories: "<<trajs_pred.sizes()<<std::endl; // [2, 6, 60, 2]
logs
=============================
start load jit model:/root/workspace/apps/my_model_conversion/model/iknownet_jit_script_pytorch1.12.1.pt
[23, 16, 3]
start inferenc:/root/workspace/apps/my_model_conversion/model/iknownet_jit_script_pytorch1.12.1.pt
Thread 1 "if-ads" received signal SIGSEGV, Segmentation fault.
__memset_avx2_erms () at ../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S:145
145 ../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S: No such file or directory.
(gdb) backtrace
#0 0x00007fda4d15cded in __memset_avx2_erms () at ../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S:145
#1 0x00007fda4b214c00 in sgemm_(char const*, char const*, la_uint_t const*, la_uint_t const*, la_uint_t const*, float const*, float const*, la_uint_t const*, float const*, la_uint_t const*, float const*, float*, la_uint_t const*) (TRANSA=0x7ffdce9df876 "TN\200", TRANSB=<optimized out>, M=0x7ffdce9df878, N=<optimized out>, K=0x7ffdce9df880, ALPHA=0x7ffdce9df890, A=0x565197d36380, LDA=0x7ffdce9df884, B=0x565199dda080, LDB=0x7ffdce9df888, BETA=0x7ffdce9df8b0, C=0x56519b31db40, LDC=0x7ffdce9df88c) at /root/workspace/apps/qpOASES/src/BLASReplacement.cpp:102
#2 0x00007fda115b3761 in at::native::cpublas::gemm(at::native::TransposeType, at::native::TransposeType, long, long, long, float, float const*, long, float const*, long, float, float*, long) () at /usr/local/libtorch/lib/libtorch_cpu.so
#3 0x00007fda116c097d in at::native::addmm_impl_cpu_(at::Tensor&, at::Tensor const&, at::Tensor, at::Tensor, c10::Scalar const&, c10::Scalar const&) ()
at /usr/local/libtorch/lib/libtorch_cpu.so
#4 0x00007fda116c1a4b in at::native::structured_mm_out_cpu::impl(at::Tensor const&, at::Tensor const&, at::Tensor const&) () at /usr/local/libtorch/lib/libtorch_cpu.so
#5 0x00007fda1215e199 in at::(anonymous namespace)::wrapper_mm(at::Tensor const&, at::Tensor const&) () at /usr/local/libtorch/lib/libtorch_cpu.so
#6 0x00007fda1215e203 in c10::impl::wrap_kernel_functor_unboxed_<c10::impl::detail::WrapFunctionIntoFunctor_<c10::CompileTimeFunctionPointer<at::Tensor (at::Tensor const&, at::Tensor const&), &at::(anonymous namespace)::wrapper_mm>, at::Tensor, c10::guts::typelist::typelist<at::Tensor const&, at::Tensor const&> >, at::Tensor (at::Tensor const&, at::Tensor const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&) () at /usr/local/libtorch/lib/libtorch_cpu.so
#7 0x00007fda11f9a5b2 in at::Tensor c10::Dispatcher::redispatch<at::Tensor, at::Tensor const&, at::Tensor const&>(c10::TypedOperatorHandle<at::Tensor (at::Tensor const&, at::Tensor const&)> const&, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&) const [clone .isra.203] () at /usr/local/libtorch/lib/libtorch_cpu.so
#8 0x00007fda11f9ab36 in at::_ops::mm::redispatch(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&) () at /usr/local/libtorch/lib/libtorch_cpu.so
#9 0x00007fda133d6b5f in torch::autograd::VariableType::(anonymous namespace)::mm(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&) ()
at /usr/local/libtorch/lib/libtorch_cpu.so
#10 0x00007fda133d79e6 in c10::impl::wrap_kernel_functor_unboxed_<c10::impl::detail::WrapFunctionIntoFunctor_<c10::CompileTimeFunctionPointer<at::Tensor (c10::DispatchKeySet, at::Tensor const&, at::Tensor const&), &torch::autograd::VariableType::(anonymous namespace)::mm>, at::Tensor, c10::guts::typelist::typelist<c10::DispatchKeySet, at::Tensor const&, at::Tensor const&> >, at::Tensor (c10::DispatchKeySet, at::Tensor const&, at::Tensor const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&) ()
at /usr/local/libtorch/lib/libtorch_cpu.so
#11 0x00007fda11fe499f in at::_ops::mm::call(at::Tensor const&, at::Tensor const&) () at /usr/local/libtorch/lib/libtorch_cpu.so
#12 0x00007fda116c32c5 in at::native::_matmul_impl(at::Tensor&, at::Tensor const&, at::Tensor const&) () at /usr/local/libtorch/lib/libtorch_cpu.so
#13 0x00007fda116c379f in at::native::matmul(at::Tensor const&, at::Tensor const&) () at /usr/local/libtorch/lib/libtorch_cpu.so
#14 0x00007fda123cff63 in c10::impl::wrap_kernel_functor_unboxed_<c10::impl::detail::WrapFunctionIntoFunctor_<c10::CompileTimeFunctionPointer<at::Tensor (at::Tensor const&, at::Tensor const&), &at::(anonymous namespace)::(anonymous namespace)::wrapper__matmul>, at::Tensor, c10::guts::typelist::typelist<at::Tensor const&, at::Tensor const&> >, at::Tensor (at::Tensor const&, at::Tensor const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&) () at /usr/local/libtorch/lib/libtorch_cpu.so
#15 0x00007fda120b81ef in at::_ops::matmul::call(at::Tensor const&, at::Tensor const&) () at /usr/local/libtorch/lib/libtorch_cpu.so
#16 0x00007fda116a742e in at::native::linear(at::Tensor const&, at::Tensor const&, c10::optional<at::Tensor> const&) () at /usr/local/libtorch/lib/libtorch_cpu.so
#17 0x00007fda1244eff0 in c10::impl::make_boxed_from_unboxed_functor<c10::impl::detail::WrapFunctionIntoFunctor_<c10::CompileTimeFunctionPointer<at::Tensor (at::Tensor const&, at::Tensor const&, c10::optional<at::Tensor> const&), &at::(anonymous namespace)::(anonymous namespace)::wrapper__linear>, at::Tensor, c10::guts::typelist::typelist<at::Tensor const&, at::Tensor const&, c10::optional<at::Tensor> const&> >, false>::call(c10::OperatorKernel*, c10::OperatorHandle const&, c10::DispatchKeySet, std::vector<c10::IValue, std::allocator<c10::IValue> >*) ()
at /usr/local/libtorch/lib/libtorch_cpu.so
#18 0x00007fda11329c69 in c10::Dispatcher::callBoxed(c10::OperatorHandle const&, std::vector<c10::IValue, std::allocator<c10::IValue> >*) const ()
at /usr/local/libtorch/lib/libtorch_cpu.so
#19 0x00007fda13e45e23 in torch::jit::InterpreterStateImpl::runImpl(std::vector<c10::IValue, std::allocator<c10::IValue> >&) () at /usr/local/libtorch/lib/libtorch_cpu.so
#20 0x00007fda13e341b2 in torch::jit::InterpreterState::run(std::vector<c10::IValue, std::allocator<c10::IValue> >&) () at /usr/local/libtorch/lib/libtorch_cpu.so
#21 0x00007fda13e272d6 in torch::jit::GraphExecutorImplBase::run(std::vector<c10::IValue, std::allocator<c10::IValue> >&) () at /usr/local/libtorch/lib/libtorch_cpu.so
#22 0x00007fda13aba520 in torch::jit::Method::operator()(std::vector<c10::IValue, std::allocator<c10::IValue> >, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, c10::IValue, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, c10::IValue> > > const&) const () at /usr/local/libtorch/lib/libtorch_cpu.so
#23 0x00007fda2a24239a in torch::jit::Module::forward(std::vector<c10::IValue, std::allocator<c10::IValue> >, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, c10::IValue, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, c10::IValue> > > const&) (this=0x7ff---Type <return> to continue, or q <return> to quit---return
dce9e1930, inputs=std::vector of length 0, capacity 0, kwargs=std::unordered_map with 0 elements) at /usr/local/libtorch/include/torch/csrc/jit/api/module.h:114
#24 0x00007fda2a25f8b1 in apollo::prediction::IKnownetEvaluator::test(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) (this=0x5651979d6980, model_path="/root/workspace/apps/my_model_conversion/model/iknownet_jit_script_pytorch1.12.1.pt") at ../../modules/prediction/evaluator/vehicle/iknownet_evaluator.cc:499
#25 0x00007fda2a260058 in apollo::prediction::IKnownetEvaluator::LoadModel() (this=0x5651979d6980) at ../../modules/prediction/evaluator/vehicle/iknownet_evaluator.cc:515
#26 0x00007fda2a2580a8 in apollo::prediction::IKnownetEvaluator::IKnownetEvaluator() (this=0x5651979d6980) at ../../modules/prediction/evaluator/vehicle/iknownet_evaluator.cc:40
#27 0x00007fda2a2260f6 in apollo::prediction::EvaluatorManager::CreateEvaluator(apollo::prediction::ObstacleConf_EvaluatorType const&) (this=0x565195e2b580, type=@0x7fda2a403358: apollo::prediction::ObstacleConf_EvaluatorType_IKNOWNET_EVALUATOR) at ../../modules/prediction/evaluator/evaluator_manager.cc:536
#28 0x00007fda2a2262a5 in apollo::prediction::EvaluatorManager::RegisterEvaluator(apollo::prediction::ObstacleConf_EvaluatorType const&) (this=0x565195e2b580, type=@0x7fda2a403358: apollo::prediction::ObstacleConf_EvaluatorType_IKNOWNET_EVALUATOR) at ../../modules/prediction/evaluator/evaluator_manager.cc:548
#29 0x00007fda2a2237b2 in apollo::prediction::EvaluatorManager::RegisterEvaluators() (this=0x565195e2b580) at ../../modules/prediction/evaluator/evaluator_manager.cc:146
#30 0x00007fda2a223b7e in apollo::prediction::EvaluatorManager::Init(apollo::prediction::PredictionConf const&) (this=0x565195e2b580, config=...)
at ../../modules/prediction/evaluator/evaluator_manager.cc:171
#31 0x00007fda2a18ee2f in apollo::prediction::MessageProcess::InitEvaluators(apollo::prediction::EvaluatorManager*, apollo::prediction::PredictionConf const&) (evaluator_manager=0x565195e2b580, prediction_conf=...) at ../../modules/prediction/common/message_process.cc:87
#32 0x00007fda2a18e92b in apollo::prediction::MessageProcess::Init(apollo::prediction::ContainerManager*, apollo::prediction::EvaluatorManager*, apollo::prediction::PredictorManager*, apollo::prediction::PredictionConf const&) (container_manager=0x565195e1a8f0, evaluator_manager=0x565195e2b580, predictor_manager=0x5651979889a0, prediction_conf=...)
at ../../modules/prediction/common/message_process.cc:59
#33 0x00007fda2a13ba3e in apollo::prediction::PredictionComponent::Init() (this=0x56519786b970) at ../../modules/prediction/prediction_component.cc:85
#34 0x00007fda4e0a639c in vcl::comp::TimerComponent::Initialize(vcl::comp::ComponentConfig const&) (this=0x56519786b970, config=...) at ../../vcl/src/component/timer_component.cpp:38
#35 0x00007fda4dfb901f in vcl::comp::ComponentManager::StartOneTimerComponent(std::shared_ptr<vcl::comp::ComponentInfo>) (this=0x565195c03710, component=std::shared_ptr<vcl::comp::ComponentInfo> (use count 5, weak count 0) = {...}) at ../../vcl/src/component/component_manager.cpp:168
#36 0x00007fda4dfb87be in vcl::comp::ComponentManager::StartTimerComponent() (this=0x565195c03710) at ../../vcl/src/component/component_manager.cpp:144
#37 0x00005651954105a3 in CreateComponents(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) (aot_name="pnc") at ../../src/mainboard/src/init.cpp:76
#38 0x000056519541082e in Init(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) (aot_name="pnc") at ../../src/mainboard/src/init.cpp:142
#39 0x000056519540bb18 in main(int, char**) (argc=2, argv=0x7ffdce9e2aa8) at ../../src/mainboard/main.cpp:86
Yes, this stacktrace shows the segfault is raised in #1 0x00007fda4b214c00 in sgemm_(char const*, char const*, la_uint_t const*, la_uint_t const*, la_uint_t const*, float const*, float const*, la_uint_t const*, float const*, la_uint_t const*, float const*, float*, la_uint_t const*) [...] at /root/workspace/apps/qpOASES/src/BLASReplacement.cpp:102 called from at::native::cpublas::gemm.
I don’t know how you’ve built PyTorch but it seems you might have replaced the native gemm calls with the qpOASES library?
The relevant code seems to be here.
I need to confirm the method used to compile pytorch, but, as far as I know, there is no reason or necessity to modify the source code. If I build a separate project to run, no error will be reported, these two phenomena cannot be explained.
Code below will goes successfuly, although they depend on same libtorch library.
source code.
#include <iostream>
#include <string>
#include <utility>
#include <vector>
#include <memory>
#include <cstring>
#include <time.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/time.h>
#include <torch/script.h> // One-stop header.
//#include "torch/extension.h"
#include <torch/cuda.h>
#include <numeric>
#include <algorithm>
#include <fstream>
using namespace std;
void test(string model_path)
{
int iact_num = 23;
int imap_num = 2;
torch::Device device_(torch::kCPU);
torch::jit::script::Module module;// = torch::jit::load(model_path);
if (torch::cuda::is_available())
{
std::cout << "CUDA is available" <<endl;
device_ = torch::Device(torch::kCUDA);
module =
torch::jit::load(model_path, device_);
} else {
module =
torch::jit::load(model_path, device_);
}
// 3.0 data packing
// actor_data: Dict[str, List[Tensor]] [batch_size * [num_agents, num_hist, dim] ]
std::unordered_map<std::string, std::vector<torch::Tensor>> actors_data;
actors_data["types_int"] = {1, torch::randn({iact_num, 1}).to(device_)}; // [5, 1]
actors_data["history_speeds"] = {1, torch::randn({iact_num, 16, 2}).to(device_)}; // [5, 16, 2]
actors_data["history_headings"] = {1, torch::randn({iact_num, 16, 1}).to(device_)}; // [5, 16, 1]
actors_data["history_trajs"] = {1, torch::randn({iact_num, 16, 1}).to(device_)}; // [5, 16, 3]
actors_data["history_trajs_relative"] = {1, torch::randn({iact_num, 16, 3}).to(device_)}; // [5, 16, 3]
std::cout<<actors_data["history_trajs_relative"][0].sizes()<<std::endl; // batch 0 [12, 16, 3]
std::unordered_map<std::string, std::vector<std::vector<torch::Tensor>>> lanes_data; //[0][32, 6] [1][32, 6, 60, 2]
lanes_data["centerlines"] = {imap_num, {torch::randn({10,2}).to(device_), torch::randn({10,2}).to(device_), torch::randn({10,2}).to(device_)}};
std::vector<torch::jit::IValue> inputs;
inputs.push_back(actors_data);
inputs.push_back(lanes_data);
auto outputs = module.forward(inputs).toTuple();
torch::Tensor trajs_prob = outputs->elements()[0].toTensor();
//std::cout << outputs.device() << std::endl;
std::cout << trajs_prob.device() << std::endl;
torch::Tensor trajs_pred = outputs->elements()[1].toTensor();
std::cout<<"probabilites: "<<trajs_prob.sizes()<<std::endl; // [2, 6]
std::cout<<"trajectories: "<<trajs_pred.sizes()<<std::endl; // [2, 6, 60, 2]
}
int main(int argc, char* argv[])
{
const std::string model_path= "../model/iknownet_jit_script_pytorch1.12.1.pt";//模型路径
test(model_path);
return 0;
}
CMakeLists.txt
cmake_minimum_required(VERSION 3.0 FATAL_ERROR)
project(custom_ops)
set(CMAKE_CXX_STANDARD 14)
link_directories(/usr/local/libtorch/lib)
include_directories(/usr/local/libtorch/include)
include_directories(/usr/local/libtorch/include/torch/csrc/api/include)
include_directories(${OpenCV_LIBS})
add_executable(example-app iknownet_inference_gpu_device_test.cpp)
target_link_libraries(example-app
${OpenCV_LIBS}
${Boost_LIBRARIES}
torch
c10
torch_cpu
#torch_python
torch_global_deps
)
output
root@ubuntu-ThinkStation-P920:~/workspace/apps/my_model_conversion/build# ./example-app
[23, 16, 3]
cpu
probabilites: [1, 6]
trajectories: [1, 6, 60, 2]
root@ubuntu-ThinkStation-P920:~/workspace/apps/my_model_conversion/build#
Sorry, but I don’t fully understand the comment.
Based on the backtrace it seems you are using qpOASES for the matmul calls and call them from libtorch, which is then causing the segfault.
Your current (working) example doesn’t show any qpOASES usage, so I still think the interaction or qpOASES itself is causing the issue.
Thank you. I am trying to rebuild pytorch1.12.1 again to make no use of qpOASES. Last comment means that there are two version inference codes, the one is integrated to a big project, when build and run the big project the libtorch related submodule report that error.The other is a small project which only run inference in main() function, and this version code runs successfully. So, same infenence API on same jit model, produced different results. Last comment’s CMakeLists.txt and source code demonstrate the second situation.
------add———
When I use the official libtorch library[https://download.pytorch.org/libtorch/cpu/libtorch-shared-with-deps-1.12.1%2Bcpu.zip], it also reported the same error, I have been powerless.Maybe it’s not the library’s fault, but my project’s runtime environment.
------add-------
I have debug integrated version code snippet and lonely libtorch demo code, below is running time model value, left is so strange, which shows the model instance is unnormal.
i just try to convert resnet model to jit format, and load the jit model, both export and import libtorch using torch1.12.1, and get this error.
The current error is raised as Unrecognized data format so you might need to check what exactly you are trying to load and make sure it’s compatible.
To the original issue: it’s still unclear if you are seeing the segfault in a pure libtorch application or not.
The stacktrace you have provided points to qpOASES.
As for the original issue, occured in forward() stage, it is obvious that model loaded form .pt file is uncorrect, so discuss the first issue is meaningless. Besize, the jit model is exported by libtorch1.12.1 and i have run complete forward pipeline in a single main() function successfully.
I am sure the libtorch is official offered. I have no idea about why there is any relationship with qpOASES.