Intermittent NvMapMemAlloc error 12 and CUDA allocator crash during PyTorch inference on Jetson Orin Nano

vision
1

Hi,

I’m running a PyTorch YOLO-based inference on a Jetson Orin Nano Super, and I frequently get these errors (not always, but randomly):

NvMapMemAllocInternalTagged: 1075072515 error 12
NvMapMemHandleAlloc: error 0
Error : NVML_SUCCESS == r INTERNAL ASSERT FAILED at "/opt/pytorch/pytorch/c10/cuda/CUDACachingAllocator.cpp":838, please report a bug to PyTorch.

I tried the following, but the issue still occurs:

  • with torch.no_grad() during inference

  • os.environ['PYTORCH_CUDA_ALLOC_CONF'] = 'expandable_segments:True'

  • Full cleanup using torch.cuda.empty_cache(), gc.collect(), and reloading the model

The error isn’t always caught by try/except and sometimes crashes the process.

Setup:

  • Jetson Orin Nano

  • JetPack 6.2.1

  • PyTorch (from NVIDIA SDK): 2.5.0a0+872d972e41.nv24.08

  • Model: YOLO (tracking mode)

Questions:

  1. Is this a PyTorch issue or a Jetson memory allocator issue (NvMapMemAlloc)?

  2. Any known fix or configuration to prevent this intermittent error?

Thanks in advance for any suggestions.

2

Are you running out of memory? Also, which build are you using?

3

Hi,

I’m having the same issue.

Error:

NvMapMemAllocInternalTagged: 1075072515 error 12
NvMapMemHandleAlloc: error 0
NvMapMemAllocInternalTagged: 1075072515 error 12
NvMapMemHandleAlloc: error 0
NvMapMemAllocInternalTagged: 1075072515 error 12
NvMapMemHandleAlloc: error 0
NvMapMemAllocInternalTagged: 1075072515 error 12
NvMapMemHandleAlloc: error 0
NvMapMemAllocInternalTagged: 1075072515 error 12
NvMapMemHandleAlloc: error 0

Error: NVML_SUCCESS == r INTERNAL ASSERT FAILED at "/opt/pytorch/c10/cuda/CUDACachingAllocator.cpp":1131, please report a bug to PyTorch.

System Information:

Jetson Model: NVIDIA Jetson Orin Nano Engineering Reference Developer Kit SuperLinux, Ubuntu 22.04.5 LTS

CPU Model: ARMv8 Processor rev 1 (v8l)CPU Cores: 6Architecture:                       aarch64

CUDA Compiler: Cuda compilation tools, release 12.6, V12.6.68

cuDNN Version:CUDNN_MAJOR 9CUDNN_MINOR 3CUDNN_PATCHLEVEL 0CUDNN_VERSION (CUDNN_MAJOR

TensorRT:libnvinfer-bin 10.3.0.30-1+cuda12.5libnvinfer-dev 10.3.0.30-1+cuda12.5libnvinfer-dispatch-dev 10.3.0.30-1+cuda12.5

Python Version: 3.10.12

PyTorch Version: 2.8.0PyTorch CUDA Available: TruePyTorch CUDA Version: 12.6PyTorch cuDNN Version: 90300PyTorch cuDNN Enabled: True

GPU Count: 1

GPU 0:Name: OrinCompute Capability: (8, 7)Total Memory: 7.44 GBMulti Processor Count: 8

TorchVision Version: 0.23.0

Any help or workarounds would be greatly appreciated.

Thanks in advance!

4

Thanks for the response.

I’ve monitored GPU and system memory in real time using jtop, and it doesn’t appear to be a straightforward out-of-memory condition; the crash occurs even when there’s available GPU memory.

Here are my setup details:

  • Device: Jetson Orin Nano Super (8 GB RAM)

  • JetPack: 6.2.1

  • PyTorch: 2.5.0a0+872d972e41.nv24.8

  • CUDA: 12.6

  • TorchVision: 0.19.1

  • RAM: 8 GB

  • Storage: 60 GB SD card + 1 TB SSD

  • Swap Memory: 25 GB