Grad-cam implementation in pytorch (backward on model)

lelouedec · May 30, 2017, 2:31pm

Hello,

I am working on an implementation of the Gram-cam paper ( check it here )
The problem is that it is coded in pure torch and as you problably know with torch we can backward directly a model/Sequential like model.backward(input,target).
I am stuck at this stage as my input is the output of a conv2D ( [torch.FloatTensor of size 1x256x13x13] ) and the target is a one dimension tensor with the classe targeted at 1 and the rest at 0 .
How could I do this backward ? I tried to use an optimizer and a loss function but it seems not possible with such tensors.

Any ideas ?

Justin

smth · May 30, 2017, 6:58pm

you cannot do model.backward(input, target) whether in LuaTorch or PyTorch. LuaTorch’s actual interface is model.backward(input, gradients_wrt_output).

You need a loss function whether it is Lua or PyTorch to measure the distance between output and target

lelouedec · May 30, 2017, 8:09pm

Ohh ok i understand the lua interface now.
How could I translate it into pytorch then ? With a loss fonction ?
If I understand it well there is the attributes gradinput in modules for torch lua, but I don’t understand how I could access it with pytorch.

Thank you for your answer taken on your time, I am a beginner concerning the implementation with pytorch

ruotianluo · May 31, 2017, 5:54am

Try

output = model(input)
gradInput = torch.autograd.grad(output, input, target)

?

lelouedec · May 31, 2017, 7:48am

Oh I found out it was only available on master 21 days ago, Is it still the case ?

EDIT :
I installed from source and I have the grad function now.
But the following error :

> Traceback (most recent call last):
>   File "grad_cam.py", line 96, in <module>
>     Gradinput = torch.autograd.grad(logit,Variable(model1_output),doutput)
>   File "/home/lelouedec/anaconda2/lib/python2.7/site-packages/torch/autograd/__init__.py", line 153, in grad
>     inputs, only_inputs)
> RuntimeError: One of the differentiated Variables appears to not have been used in the graph

What I want to achieve is the following piece of code from lua:
model2:zeroGradParameters()
model2:backward(model1.output, doutput)

– Get the activations from model1 and and gradients from model2
local activations = model1.output:squeeze()
local gradients = model2.gradInput:squeeze()

Where model 1 is the first half of alexnet and model 2 the other half

ruotianluo · May 31, 2017, 10:57pm

I personally haven’t really tried torch.autograd.grad in practice, so I’m not sure if I’m correct

My suggestion is:
model1_output should be a Variable with requires_grad to be True, and logit is a function of model1_output.
For example:

model1_output = model1(input) # input can not be volatile
logit = model2(model1_output)
Gradinput = torch.autograd.grad(logit, model1_output, doutput)

BTW, there is a pytorch grad cam implementation which just came out. This uses register_hook to save the gradients.

github.com

jacobgil/pytorch-grad-cam/blob/master/grad-cam.py

import torch
from torch.autograd import Variable
from torch.autograd import Function
from torchvision import models
from torchvision import utils
import cv2
import sys
import numpy as np
import argparse

class FeatureExtractor():
    """ Class for extracting activations and 
    registering gradients from targetted intermediate layers """
    def __init__(self, model, target_layers):
        self.model = model
        self.target_layers = target_layers
        self.gradients = []

    def save_gradient(self, grad):
    	self.gradients.append(grad)

This file has been truncated. show original

lakehanne · June 1, 2017, 12:11am

How do you embed GitHub code snippets on this website?

lelouedec · June 1, 2017, 5:05am

Well man thank you I was on the right track and I stopped working at the last step of the algorithm yesterday…
I used hooks to get gradinput and output of intermediate layers. Easier than his technique I think …
But anyway thanks you I maybe didn’t do it first but at least I learned.