Cannot insert a Tensor that requires grad as a constant

jit
1

I implemented my own version of CaffeNet in PyTorch and I am trying to save the model in an onnx file. But I have the error described in the title when I run this code:

"
import torch.nn as nn
import torch

class Flatten(nn.Module):
def forward(self,x):
return x.view(x.size(0),-1)

class CaffeNet(nn.Module):
def init(self):
super(CaffeNet,self).init()

    self.layers=[]

    self.layers.append(nn.Conv2d(in_channels=3,out_channels=3,kernel_size=11))
    self.layers.append(nn.ZeroPad2d(padding=1))

    self.layers.append(nn.Conv2d(3,96,55))
    self.layers.append(nn.MaxPool2d(kernel_size=2,stride=2))
    self.layers.append(nn.ZeroPad2d(padding=1))

    self.layers.append(nn.Conv2d(96,192,27))
    self.layers.append(nn.MaxPool2d(kernel_size=2, stride=2))
    self.layers.append(nn.ZeroPad2d(padding=1))

    self.layers.append(nn.Conv2d(192, 288, 13))
    self.layers.append(nn.ZeroPad2d(padding=1))

    self.layers.append(nn.Conv2d(288, 288, 13))
    self.layers.append(nn.ZeroPad2d(padding=1))

    self.layers.append(nn.Conv2d(288, 256, 11)) #12-th layer, normally kernel_size=13
    self.layers.append(nn.MaxPool2d(kernel_size=2, stride=2))
    self.layers.append(nn.ZeroPad2d(padding=1))

    self.layers.append(nn.Conv2d(256, 256, 3)) #15-th layer, normally kernel_size=13
    self.layers.append(nn.MaxPool2d(kernel_size=2, stride=2))
    self.layers.append(nn.ZeroPad2d(padding=1))

    self.layers.append(Flatten())

    self.layers.append(nn.Linear(2304,2304)) #19-th layer, normally 4096 neurons
    self.layers.append(nn.ReLU())
    self.layers.append(nn.Dropout())

    self.layers.append(nn.Linear(2304,2304))
    self.layers.append(nn.ReLU())
    self.layers.append(nn.Dropout())

    self.layers.append(nn.Linear(2304,1000))
    self.layers.append(nn.Softmax())


def forward(self, x):
    for layer in self.layers:
        x=layer(x)
    return x

net=CaffeNet()
input = torch.randn(20, 3, 227, 227,requires_grad=False)
input_names = [ “input” ]
output_names = [ “output” ]
torch.onnx.export(net,input,“caffenet.onnx”,verbose=True, input_names=input_names, output_names=output_names)
"

I am using python2.7. The last line of the stack trace is:
“RuntimeError: Cannot insert a Tensor that requires grad as a constant. Consider making it a parameter or input, or detaching the gradient”

1 Like
2

The modules you’re adding have some Parameters, so they need to be submodules of CaffeNet. It should work if you use a torch.nn.Sequential instead of a regular Python list for self.layers.

3 Likes
3

Ok Thanks you. I’m gonna try this on monday when I’m going back to work. I hope it will work :slight_smile:

1 Like
4

Thanks you very much driazati, it works :wink:

2 Likes
5

hi，I have the same problem as you. How did you solve it，Look forward to your reply
thanks

1 Like
6

hi, the subLayers of your module should be its submodules. So:
1/ I made my CaffeNet module derived from nn.sequential
2/Then I defined a add method which adds layer to the subModule dictionary (self._modules) of a nn.sequential:
def add(self, layer):
self._modules[str(self.index)]=layer
self.index=self.index+1
3/Then I call this method with each layer of CaffeNet

1 Like
7

Thank you for your reply, you are a good man, but I don’t know the specific operation, can you provide the code to solve it?

8

Unfortunately, I seem to have the problem (and replied in another similar question). My model has a VGG (as encoder) and FCN as a decoder it’s not analogous. Any ideas?

9

Just replace self.layers =[] with self.layers =nn.ModuleList() or something else

Otherwise, model.requires_grad() won’t affect those layers within python list.

1 Like
10

I have given my solution in RuntimeError: Cannot insert a Tensor that requires grad as a constant - #3 by smkoop. We just need manually set all parameters required_grad = False .