# Query regarding register\_backward\_hook

**URL:** <https://discuss.pytorch.org/t/query-regarding-register-backward-hook/14184>\
**Category:** vision\
**Created:** [March 1, 2018, 7:29am UTC](https://discuss.pytorch.org/t/query-regarding-register-backward-hook/14184 "2018-03-01T07:29:52Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![thesemicolonguy](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/thesemicolonguy/32/3695_2.png) [@thesemicolonguy](https://discuss.pytorch.org/u/thesemicolonguy)\
**Post date:** [March 1, 2018, 7:29am UTC](https://discuss.pytorch.org/t/query-regarding-register-backward-hook/14184/1 "2018-03-01T07:29:52Z")

</div>

I’m working on CNN saliency maps.

```auto
def saliency_map_general(model, input, label):
    if (isinstance(model, torchvision.models.Inception3)):
        input = preprocess_inception(input)
    else:
        input = preprocess(input)

    output = model.forward(input)
    model.zero_grad()
                                       
    output[0][label].backward()

    grads = input.grad.data.clamp(min=0)
    grads.squeeze_()
    grads.transpose_(0,1)
    grads.transpose_(1,2)
    grads = np.amax(grads.cpu().numpy(), axis=2)
    
    true_image = input.data
    true_image = true_image.squeeze()
    true_image = true_image.transpose(0,1)
    true_image = true_image.transpose(1,2)
    true_image = deprocess(true_image)

    fig = plt.figure()
    plt.rcParams["figure.figsize"] = (20, 20)

    a = fig.add_subplot(1,2,1)
    imgplot = plt.imshow(true_image)
    plt.title('Original Image')
    plt.axis('off') 

    a = fig.add_subplot(1,2,2)
    imgplot = plt.imshow(grads, cmap='hot')
    plt.axis('off') 
    plt.title('Saliency Map')
    
    return grads

```

Here, I’m finding the derivative of the output “true class” w.r.t. input pixels.

Now, I want to use ‘guided backpropagation’ i.e. taking max(grad, 0) at each layer before passing the gradient to previous layer during backward pass. The documentation of register\_backward\_hook is not very detailed.

I want something like

```auto
def hookfunc(model, gi, go):
    grads = gi > 0
    return grads

h = model.register_backward_hook(hookfunc)

```

I tried doing this, but this hook callback function never runs. I think that is because I’m running .backward() on an element of the output Variable, and not on the model. I’m not sure about this, but the documentation isn’t making anything clear.

To clarify, I want the hook callback function to run for every layer during back pass. Any help on how to do this would be appreciated.

---

<div class="post-metadata">

**Author:** ![lelouedec](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/lelouedec/32/3563_2.png) [@lelouedec](https://discuss.pytorch.org/u/lelouedec)\
**Post date:** [March 1, 2018, 8:55am UTC](https://discuss.pytorch.org/t/query-regarding-register-backward-hook/14184/2 "2018-03-01T08:55:36Z")

</div>

Where do you link your hook to your layer ? (I am also working on saliency map 😉 )

---

<div class="post-metadata">

**Author:** ![tom](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/tom/32/3162_2.png) [@tom](https://discuss.pytorch.org/u/tom)\
**Post date:** [March 1, 2018, 9:35am UTC](https://discuss.pytorch.org/t/query-regarding-register-backward-hook/14184/3 "2018-03-01T09:35:42Z")

</div>

I don’t think the hook will work as expected (see this recent thread for a discussion of the values of the hook [Exact meaning of grad\_input and grad\_output](https://discuss.pytorch.org/t/exact-meaning-of-grad-input-and-grad-output/14186)).  
If you write the model yourself, you could just use hidden.register\_hook(lambda grad: grad.clamp(min=0)), similar to the [gradient clipping discussed here](https://discuss.pytorch.org/t/implement-cell-gradient-clamp-in-nn-lstm/14135)) on your activations between the layers.

Best regards

Thomas

---

<div class="post-metadata">

**Author:** ![thesemicolonguy](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/thesemicolonguy/32/3695_2.png) [@thesemicolonguy](https://discuss.pytorch.org/u/thesemicolonguy)\
**Post date:** [March 1, 2018, 10:04am UTC](https://discuss.pytorch.org/t/query-regarding-register-backward-hook/14184/4 "2018-03-01T10:04:58Z")

</div>

First of all, thanks for the awesome explanation of gradInput and gradOutput. I was trying to figure out what they were, but couldn’t find it anywhere in the docs.

I’ve found a workaround for this. It’s not an elegant solution, but it works.

```auto
def saliency_map_general(model, input, label):
    if (isinstance(model, torchvision.models.Inception3)):
        input = preprocess_inception(input)
    else:
        input = preprocess(input)
    
    h = [0]*len(list(model.modules()))

    def hookfunc(module, gradInput, gradOutput):
        print('hook callback is running')
        // do something here
    for j, i in enumerate(list(model.modules())):
        h[j] = i.register_backward_hook(hookfunc)
    output = model.forward(input)
    model.zero_grad()
                                        
    output[0][label].backward()
                                        
    for i in range(len(list(model.modules()))):
        h[i].remove()
    grads = input.grad.data.clamp(min=0)
    grads.squeeze_()
    grads.transpose_(0,1)
    grads.transpose_(1,2)
    grads = np.amax(grads.cpu().numpy(), axis=2)
    
    return grads

```

Can someone guide me as to how I should clamp the gradients to 0? (where I wrote “// do something here” in the code).

gradInput and gradOutput are both tuples, so I’m not able to do any operations on them.

I cannot convert gradInput to type “list” because it takes way too much time to do that (I grew impatient and restarted the jupyter notebook after a few minutes).

---

<div class="post-metadata">

**Author:** ![tom](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/tom/32/3162_2.png) [@tom](https://discuss.pytorch.org/u/tom)\
**Post date:** [March 1, 2018, 10:25am UTC](https://discuss.pytorch.org/t/query-regarding-register-backward-hook/14184/6 "2018-03-01T10:25:39Z")

</div>

Does `return [(None if g is None else g.clamp(min=0)) for g in gradInput]` look vaguely good? (I didn’t try, but I would hope the gradients are either None or a variable.)

Best regards

Thomas

---

<div class="post-metadata">

**Author:** ![thesemicolonguy](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/thesemicolonguy/32/3695_2.png) [@thesemicolonguy](https://discuss.pytorch.org/u/thesemicolonguy)\
**Post date:** [March 1, 2018, 10:36am UTC](https://discuss.pytorch.org/t/query-regarding-register-backward-hook/14184/7 "2018-03-01T10:36:57Z")

</div>

Was getting error:  
expected tuple, but hook returned ‘list’

This works without error though.

```auto
def hookfunc(module, gradInput, gradOutput):
            return tuple([(None if g is None else g.clamp(min=0)) for g in gradInput])

```

Thanks for the help!

---

<div class="post-metadata">

**Author:** ![thesemicolonguy](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/thesemicolonguy/32/3695_2.png) [@thesemicolonguy](https://discuss.pytorch.org/u/thesemicolonguy)\
**Post date:** [March 1, 2018, 10:40am UTC](https://discuss.pytorch.org/t/query-regarding-register-backward-hook/14184/8 "2018-03-01T10:40:36Z")

</div>

Finally got it right! Thanks @tom

 ![image](https://discuss.pytorch.org/uploads/default/original/2X/f/f6b609488e905e26ee863808a220017049c555b0.jpg)

---

<div class="post-metadata">

**Author:** ![vsag](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/vsag/32/3285_2.png) [@vsag](https://discuss.pytorch.org/u/vsag)\
**Post date:** [March 1, 2018, 10:47am UTC](https://discuss.pytorch.org/t/query-regarding-register-backward-hook/14184/9 "2018-03-01T10:47:39Z")

</div>

Looks amazing, was this the paper by the VGG group? Do you plan on doing object localisation also?

---

<div class="post-metadata">

**Author:** ![thesemicolonguy](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/thesemicolonguy/32/3695_2.png) [@thesemicolonguy](https://discuss.pytorch.org/u/thesemicolonguy)\
**Post date:** [March 1, 2018, 10:56am UTC](https://discuss.pytorch.org/t/query-regarding-register-backward-hook/14184/10 "2018-03-01T10:56:51Z")

</div>

Guided backprop was introduced in this paper: [https://arxiv.org/abs/1412.6806](https://arxiv.org/abs/1412.6806)

Yes, I’m doing a survey paper on weakly supervised localization techniques. So, this’ll be a part of that.

---

<div class="post-metadata">

**Author:** ![kindlychung](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/kindlychung/32/3679_2.png) [@kindlychung](https://discuss.pytorch.org/u/kindlychung)\
**Post date:** [March 1, 2018, 11:01am UTC](https://discuss.pytorch.org/t/query-regarding-register-backward-hook/14184/11 "2018-03-01T11:01:22Z")

</div>

Do you have a github repo for this? Would be nice to have that to look into. Thanks!

---

<div class="post-metadata">

**Author:** ![thesemicolonguy](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/thesemicolonguy/32/3695_2.png) [@thesemicolonguy](https://discuss.pytorch.org/u/thesemicolonguy)\
**Post date:** [March 1, 2018, 11:28am UTC](https://discuss.pytorch.org/t/query-regarding-register-backward-hook/14184/12 "2018-03-01T11:28:19Z")

</div>

Here it is. It’s not complete yet, still a lot of work to do.

> **[thesemicolonguy/weakly-supervised-localization-survey](https://github.com/thesemicolonguy/weakly-supervised-localization-survey)**
>
> weakly-supervised-localization-survey - Survey of various algorithms for weakly supervised object localization using pre-trained CNNs in PyTorch