If I modify it to:
loss = loss1 + 0.8 * loss2
then it is a weighted loss?
Yes, it is weighted. You can mix the losses as you want.
1 Like
what if I really need to get grad1 and grad2?[quote=“brisker, post:55, topic:119”]
criterion = nn.MSELoss()
x = Variable(torch.randn(1,100), requires_grad=True)
y = Variable(torch.randn(1,40))
class ToyModel(nn.Module):
def init(self):
super(ToyModel, self).init()
self.linear1 = nn.Linear(100,50)
self.linear2 = nn.Linear(50,40)
self.linear3 = nn.Linear(100,40)
def forward(self, x):
out1 = self.linear1(x)
out2 = self.linear2(out1)
out3 = self.linear3(x)
return out2,out3
model=ToyModel()
out2,out3 = model.forward(x)
print out3
loss1= criterion(out2,y)
loss2 = criterion(out3,y)
#print out2.grad
torch.autograd.backward(x, [grad1, grad2])
[/quote]
Before performing backpropagation you won’t have gradients as they are initialized when they are actually computed. (lazy initialization)
import torch
import torch.nn as nn
from torch.autograd import Variable
model = nn.Linear(5, 7)
x = Variable(torch.randn(10, 5))
y = model(x)
print(model.weight.grad) # None
y.backward(torch.randn(y.size()))
print(model.weight.grad) # Prints a 7x5 tensorwhat about the model as described here?:[quote=“brisker, post:55, topic:119”]
class ToyModel(nn.Module):
def init(self):
super(ToyModel, self).init()
self.linear1 = nn.Linear(100,50)
self.linear2 = nn.Linear(50,40)
self.linear3 = nn.Linear(100,40)
def forward(self, x):
out1 = self.linear1(x)
out2 = self.linear2(out1)
out3 = self.linear3(x)
return out2,out3
[/quote]
you need to use hooks if you want to inspect gradients of intermediate variables.
Refer to the discussion here: Why cant I see .grad of an intermediate variable?
In your case you need to attach a hook to out2 and out3, which returns the ‘grad’
Hey guys, pardon my silly question. Of what use is the FC layer? How do you generalize to localization/detection of objects using the last fclayer features?
In this line new_classifier = nn.Sequential(*list(model.classifier.children())[:-1]) from @fmassa 's post, I wonder if it was meant to read
new_classifier = nn.Sequential(*list(model.children())[:-1])
I checked the methods of the resnet18 model and it seemed to have
| children(self)
| Returns an iterator over immediate children modules.
as a direct method. Am I wrong?
resnet18 doesn’t have a classifier field, and the solution I gave was for vgg.
2 Likes
@fmassa In order to get the fc7 features of a resnet, do I need to write a class like you did with Inception, or is there a more straightforward way of doing so?
Just using:
new_classifier = nn.Sequential(*list(model.children())[:-1])
model = new_classifier
seems to work. Right?
Also, is there a way of getting both the fc7 features and the results of the softmax?
Hey @apaszke, @fmassa, if I download a resnet18 without weights for example,
resnet = torchvision.models.resnet18(pretrained=False)
Now, say I want to change the shape of the last layer to 512->2.
Which is the appropriate way to reshape the last fc layer? This:
resnet.add_module('9', nn.Linear(512, 2))?
can you please tell me how are you loading model in different file??
Has there been any updates on a feature that would allow iteration over graphs by name?
Hi fmassa! Where can I get to know model.classifier? Why isn’t it new_classifier = nn.Sequential(*list(model.children())[:-1]) ?
It depends on your model, but usually printing it is a first way to inspect (roughly) what models are used inside.
But you can always refer back to the python implementation to have a idea of the structure of the model.
Sentence of “new_classifier = nn.Sequential(*list(model.children())[:-1])” goes right!
Hi @fmassa how can i get a certain conv layer’s feature in DenseNet model? since i do not know the exact name of this layer. thanks~
@visonpon you can check the source code for exact layer names: https://github.com/pytorch/vision/blob/master/torchvision/models/densenet.py
For any model, you can use model.children() to see it’s layers. Once you know the number of layers you want to retain (based on which layer you want to get features from), use the nn.Sequential() function to create a new model. Then do a model(x) on input x to get output features. Read this thread a little more thoroughly, all of these commands have been written many times here.
3 Likes