I have a model that outputs two values, one for a classification task, and other for a regression task. Because I don’t know if it is even possible to use in a single loss function multiple output / target pairs, my model outputs a single tensor where input[:8] are the probabilities for the classification task, and input[8] is the regressed scalar, so the loss function:
def species_length_loss(input, target):
input_species = input[:,:8]
input_length = input[:,8]
target_species = target[:,0].long()
target_length = target[:,1]
if target_species != 7: #doesn't work b/c target_species is N (batch size)
loss = F.mse_loss(input_length, target_length) * 1e-4 + F.nll_loss(input_species, target_species)
else:
loss = F.nll_loss(input_species, target_species)
return loss
I’ve placed the if statement to illustrate what I want to do, but even if just leave loss = F.mse_loss(input_length, target_length) * 1e-4 + F.nll_loss(input_species, target_species) (w/o the statement) I get the error:
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation
This has me puzzled. Your code works but when I try to run the same (well almost):
def species_length_loss(input, target):
target = th.autograd.Variable(th.randn(32,2).random_(0,8).float(), requires_grad=False).cuda() # JUST TO TEST
input_species = input[:,:8]
input_length = input[:, 8]
target_species = target[:,0].long()
target_length = target[:,1]
loss = nn.MSELoss()(input_length, target_length) * 1e-4 # + F.nll_loss(input_species, target_species)
return loss
It fails as before. I’ve made sure targetrequires_grad=False but still I’m getting the same error:
Traceback (most recent call last):
File "fish.py", line 191, in <module>
trainer.fit_loader(train_loader, valid_loader, num_epoch=200, verbose=1, cuda_device=0)
File "build/bdist.linux-x86_64/egg/torchsample/modules/module_trainer.py", line 367, in fit_loader
File "/usr/local/lib/python2.7/dist-packages/torch/autograd/variable.py", line 156, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph, retain_variables)
File "/usr/local/lib/python2.7/dist-packages/torch/autograd/__init__.py", line 98, in backward
variables, grad_variables, retain_graph)
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation```
You do not need to store the values in one array. You can simply return a tuple. This will avoid the inplace operations in species_length
in your forward function you can write:
def forward(self, x):
x = self.features(x)
x = x.view(-1,self.num_ftrs )
if False:
x = F.relu(self.class_fc1(x))
species = self.class_fc2(x)
species = nn.LogSoftmax()(species)
length = self.reg_fc2(x)
# species_length = (species, length)
return species, length
else:
species_length = F.relu(self.class_fc1(x))
species_length = self.class_fc2(species_length)
# species_length[:8] = nn.LogSoftmax()(species_length[:8])
species_length = nn.LogSoftmax()(species_length)
# species_length[8] = F.relu(species_length[8])
#length = F.relu(self.reg_fc1(x))
#length = F.relu(self.reg_fc2(length))
return species_length, F.relu(species_length[8])