# Assignment to split tensor causes memory leak

**URL:** <https://discuss.pytorch.org/t/assignment-to-split-tensor-causes-memory-leak/74875>\
**Category:** Memory Format\
**Created:** [March 31, 2020, 6:58am UTC](https://discuss.pytorch.org/t/assignment-to-split-tensor-causes-memory-leak/74875 "2020-03-31T06:58:11Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![DominikP](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/d/ecd19e/32.png) [@DominikP](https://discuss.pytorch.org/u/DominikP)\
**Post date:** [March 31, 2020, 6:58am UTC](https://discuss.pytorch.org/t/assignment-to-split-tensor-causes-memory-leak/74875/1 "2020-03-31T06:58:11Z")

</div>

I want to train a network for smoothing a point cloud.  
It is applied independently for each point and has the surrounding points as input.  
The function below performs the actual smoothing by splitting the points (_x_) and the neighborhoods (_y_) into batches and then calling the model.

```auto
def do_smoothing(x, y, model, batch_size):
    model.eval()
    torch.no_grad() # Make sure that no gradient memory is allocated
    print('Used Memory: {:3f}'.format(torch.cuda.memory_allocated('cuda') * 1e-9 ))
    xbatches = list(torch.split(x, batch_size))
    ybatches = list(torch.split(y, batch_size))
    assert(len(xbatches) == len(ybatches))
    print('Used Memory: {:3f}'.format(torch.cuda.memory_allocated('cuda') * 1e-9 ))
    for i in range(len(xbatches)):
        print("Batch {} [Memory: {:3f}]".format(i + 1, torch.cuda.memory_allocated('cuda') * 1e-9))
        tmp = self.forward(_xbatches[i])
        xbatches[i] += tmp
        del tmp
        print("Batch {} [Memory: {:3f}]".format(i + 1, torch.cuda.memory_allocated('cuda') * 1e-9))
        print("------------------------------------")
    return x

```

Unfortunately, the memory consumption is very high leading to an out of memory error. The console output is:

```auto
Used Memory: 0.256431
Used Memory: 0.256431
Prior batch 1 [Memory: 0.256431]
Post batch 1 [Memory: 2.036074]
------------------------------------
Prior batch 2 [Memory: 2.036074]
 -> Out of memory error

```

If I remove the line `xbatches[i] += tmp` the allocated memory is not changing (as expected).  
If I also remove the line `del tmp` on the other hand the code once again allocates huge amounts of GPU memory.

I assume that [torch.split](https://pytorch.org/docs/stable/torch.html#torch.split) creates views onto the tensor and therefore the update should not use additional memory.  
Did I miss anything or is this unintended behavior?

I am using pytorch 1.4.0 with CUDA 10.1 on a Windows 10 machine.

Thanks in advance for any tips.

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [March 31, 2020, 7:41am UTC](https://discuss.pytorch.org/t/assignment-to-split-tensor-causes-memory-leak/74875/2 "2020-03-31T07:41:08Z")

</div>

Could you try to use `torch.no_grad()` in a `with` statement:

```python
model.eval()
with torch.no_grad():
    xbatches = ...

```

If I’m not mistaken, your current approach shouldn’t change the gradient behavior.

---

<div class="post-metadata">

**Author:** ![DominikP](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/d/ecd19e/32.png) [@DominikP](https://discuss.pytorch.org/u/DominikP)\
**Post date:** [March 31, 2020, 7:51am UTC](https://discuss.pytorch.org/t/assignment-to-split-tensor-causes-memory-leak/74875/3 "2020-03-31T07:51:50Z")

</div>

Thank you very much, this indeed fixes the problem.

---

<div class="post-metadata">

**Author:** ![Riccorl](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/riccorl/32/35686_2.png) [@Riccorl](https://discuss.pytorch.org/u/Riccorl)\
**Post date:** [November 3, 2022, 6:53pm UTC](https://discuss.pytorch.org/t/assignment-to-split-tensor-causes-memory-leak/74875/4 "2022-11-03T18:53:36Z")

</div>

I know it is an old question, but I have the same problem. The proposed solution for me doesn’t work because I need to split tensors during training. Is there a way to split a tensor without getting OOM errors?

---

<div class="post-metadata">

**Author:** ![nuoyi\_ma](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/nuoyi_ma/32/15447_2.png) [@nuoyi\_ma](https://discuss.pytorch.org/u/nuoyi_ma)\
**Post date:** [August 13, 2023, 8:23am UTC](https://discuss.pytorch.org/t/assignment-to-split-tensor-causes-memory-leak/74875/5 "2023-08-13T08:23:26Z")

</div>

Have you found a solution?