# Freeing buffer strange behavior

**URL:** <https://discuss.pytorch.org/t/freeing-buffer-strange-behavior/31955>\
**Category:** autograd\
**Created:** [December 12, 2018, 1:12pm UTC](https://discuss.pytorch.org/t/freeing-buffer-strange-behavior/31955 "2018-12-12T13:12:09Z")\
**Posts on this page:** 1\
**Showing post:** 8

<div class="post-metadata">

**Author:** ![albanD](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/32/215_2.png) [@albanD](https://discuss.pytorch.org/u/albanD)\
**Post date:** [December 14, 2018, 1:13pm UTC](https://discuss.pytorch.org/t/freeing-buffer-strange-behavior/31955/8 "2018-12-14T13:13:02Z")

</div>

I’ll add more informations here for future reference.

The corresponding PR to fix it is [here](https://github.com/pytorch/pytorch/pull/15219).

Smallest repro code:

```python
import torch
from torch import nn, cuda
from torch.autograd import Variable, grad
from torch.nn import functional as F

# Debug stuff
import torchviz
torch.autograd.set_detect_anomaly(True)

inputs = torch.ones((1, 3, 256, 256), requires_grad=True)

tmp1 = (inputs+1).view_as(inputs)
tmp2 = F.threshold(tmp1, 0., 0., True)
prob_interpolated = torch.sigmoid(tmp2)

gradients = grad(outputs=prob_interpolated, inputs=inputs,
                 grad_outputs=torch.ones(prob_interpolated.size()),
                 create_graph=True, retain_graph=True)[0]

gradient_penalty = gradients.sum()

# Debug graph
torchviz.make_dot(gradient_penalty).view()
gradient_penalty.backward()

```

The computational graph generated is:

 ![image](https://discuss.pytorch.org/uploads/default/original/2X/b/bde9bc1e7d94e038fd340b68931fc19a6feb5af2.png)

The interesting part is the branch on the right that links `ThresholdBackwardBackward` directly to `ThresholdBackward` while `ThresholdBackward` is already wrapped inside the first `CopySlices`.

The thing is that part of the `threshold_` function code is:

```cpp
  baseType->threshold_(self_, threshold, value);
  increment_version(self);
  rebase_history(flatten_tensor_args( self ), grad_fn);
  if (tracer_state) {
    jit::tracer::setTracingState(std::move(tracer_state));
    jit::tracer::addOutput(node, self);
  }
  if (grad_fn) {
    grad_fn->result_ = SavedVariable(self, true);
  }

```

As you can see, `self` is considered as an output of `grad_fn` when saved. And so when `ThresholdBackward` is called to generate `ThresholdBackwardBackward`, `self` is associated to `ThresholdBackward` and thus the graph above.

The thing is that after the rebase\_history, `self` is not an output of `grad_fn` anymore, it’s an output of the rewritten graph.

Changing the save to

```auto
grad_fn->result_ = SavedVariable(self, !as_variable_ref(self).is_view());

```

Make sure that in the case where `self`’s history is rewritten, we don’t consider it as an output of `grad_fn` anymore.

After the fix in the PR, the new graph is as expected:

 ![image](https://discuss.pytorch.org/uploads/default/original/2X/6/6fcd86058c3488ee58acb8faae812a673e2a49f1.png)

---

_[View the full topic](https://discuss.pytorch.org/t/freeing-buffer-strange-behavior/31955)._