# If your PyTorch process runs out of memory after hours of switching between models, the problem isn't PyTorch — it's glibc's malloc arena allocator

**URL:** <https://discuss.pytorch.org/t/if-your-pytorch-process-runs-out-of-memory-after-hours-of-switching-between-models-the-problem-isnt-pytorch-its-glibcs-malloc-arena-allocator/224743>\
**Category:** deployment\
**Created:** [March 29, 2026, 3:46am UTC](https://discuss.pytorch.org/t/if-your-pytorch-process-runs-out-of-memory-after-hours-of-switching-between-models-the-problem-isnt-pytorch-its-glibcs-malloc-arena-allocator/224743 "2026-03-29T03:46:59Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![brjen](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/brjen/32/79460_2.png) [@brjen](https://discuss.pytorch.org/u/brjen)\
**Post date:** [March 29, 2026, 3:46am UTC](https://discuss.pytorch.org/t/if-your-pytorch-process-runs-out-of-memory-after-hours-of-switching-between-models-the-problem-isnt-pytorch-its-glibcs-malloc-arena-allocator/224743/1 "2026-03-29T03:46:59Z")

</div>

### The problem

When PyTorch loads a model, glibc allocates large memory arenas via `malloc`. When you unload with `del model` + `gc.collect()` + `torch.cuda.empty_cache()`, Python releases its references — but glibc keeps the arenas because small residual allocations pin entire chunks. Memory grows with every model switch and is never returned to the OS.

This affects anyone running long-lived inference servers, Gradio apps, ComfyUI, or any pipeline that loads/unloads multiple models.

### The fix

```bash
export MALLOC_MMAP_THRESHOLD_=65536
export MALLOC_TRIM_THRESHOLD_=65536

```

Set these before launching Python. Forces allocations \>64KB to use `mmap()` instead of arenas. `mmap` pages are returned to the OS immediately on free — no fragmentation.

### Proof

- **Before:** RSS grew ~3GB per model switch, OOM after 17 hours and 107 switches
- **After:** RSS flat at 955MB across 107 consecutive switches between 13 different checkpoints (SDXL, Flux, PixArt, SD 1.5, Playground v2.5)
- Tested with diffusers/FastAPI on an AMD RX 7800 XT (ROCm) and NVIDIA GTX 1080 Ti (CUDA)

No code changes. No hook removal. No gc hacks. Just two environment variables.

Full write-up with methodology and data: [GitHub - brjen/pytorch-memory-fix: Two environment variables that fix PyTorch/glibc memory creep on Linux. Zero code changes. Zero performance cost. · GitHub](https://github.com/brjen/pytorch-memory-fix)

---

<div class="post-metadata">

**Author:** ![Aditya\_Mehra](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/aditya_mehra/32/79672_2.png) [@Aditya\_Mehra](https://discuss.pytorch.org/u/Aditya_Mehra)\
**Post date:** [May 22, 2026, 12:58am UTC](https://discuss.pytorch.org/t/if-your-pytorch-process-runs-out-of-memory-after-hours-of-switching-between-models-the-problem-isnt-pytorch-its-glibcs-malloc-arena-allocator/224743/2 "2026-05-22T00:58:26Z")

</div>

This is a great write-up — the malloc arena fragmentation issue is one of the more insidious problems in long-running PyTorch serving processes and it is underdiagnosed.

A few things to add from production experience:

**Monitoring:** You can track this in production by comparing torch.cuda.memory\_allocated() (PyTorch’s view of GPU memory) against the RSS reported by the OS (/proc/self/status -\>VmRSS). If RSS grows steadily while PyTorch’s allocator stats stay flat, the leak is in the CPU-side allocator, not the GPU side.

**Alternative to MALLOC\_ARENA\_MAX:** jemalloc or tcmalloc are drop-in replacements that handle arena fragmentation much better for long-running processes. In serving environments:

**# Using jemalloc**

**LD\_PRELOAD=/usr/lib/x86\_64-linux-gnu/libjemalloc.so python serve.py**

**# Using tcmalloc**

**LD\_PRELOAD=/usr/lib/x86\_64-linux-gnu/libtcmalloc.so python serve.py**

Both are widely used in production ML serving. jemalloc in particular is the default allocator for Redis and Rust for exactly this reason.

In my opinion for anyone diagnosing memory growth in long-running PyTorch processes, the debugging order should be:

1. torch.cuda.memory\_summary() — check for GPU-side fragmentation

2. gc.get\_objects() filtered by tensor type — check for Python-side tensor leaks

3. RSS monitoring — check for CPU allocator fragmentation (this post)

4. /proc/self/maps — check for memory-mapped file accumulation

Thanks for sharing this — it is the kind of operational knowledge that rarely makes it into tutorials.

/Aditya