Hello everyone. I am trying to extra data during training Resnet18. In my case it is gender information expressed as gender=Tensor([g]), where g =[-1|0|1] for [female|unknown|male] , and some shape descriptor obtained in previous steps, which is of the form desc=Tensor([...]), desc.shape = ([64,]) .
I tried to follow the example at this thread but my loss value becomes NaN in the first epoch. Changing the learning rate does not help
Here is the modified resnet18 function that i use. in the comments i put the shape and magnitudes of tensors. As you can see the data that i want to add is already normalized:
def forward(self, x,descdata,genderdata):
#x.shape = #torch.Size([4, 3, 224, 224])
# descdata.shape = torch.Size([ 4,64])
# genderdata.shape = torch.Size([1,4])x = self.conv1(x) #torch.Size([4, 64, 112, 112]) x = self.bn1(x) #torch.Size([4, 64, 112, 112]) x = self.relu(x) # torch.Size([4, 64, 112, 112]) x = self.maxpool(x) #torch.Size([4, 64, 56, 56]) x = self.layer1(x) #torch.Size([4, 64, 56, 56]) x = self.layer2(x) #torch.Size([4, 128, 28, 28]) x = self.layer3(x) #torch.Size([4, 256, 14, 14]) x = self.layer4(x) #torch.Size([4, 512, 7, 7]) x = self.avgpool(x) #torch.Size([4, 512, 1, 1]) x = torch.flatten(x, 1) #torch.Size([4, 512]) x = self.fc(x) #torch.Size([4, 35]); torch.norm(x,dim=1) == tensor([4.4604, 1.9574, 2.0123, 1.9583]) y = genderdata.transpose(1,0) #torch.Size([4, 1]); torch.norm(y,dim=1) == tensor([1., 1., 1., 1.]) z1 = torch.cat([x,y,descdata.type(torch.float32)],dim=1) #torch.Size([4, 100]); torch.norm(z1,dim=1) == tensor([3.6243, 2.5945, 3.1520, 3.0312]) z1 = self.relu2(self.fc2(z1)) # self.relu2 = nn.ReLU(); self.fc2 = nn.Linear(num_classes+1+64, (num_classes+1+64)*2) defined in __init__ z1 = self.fc3(z1) # self.fc3 = nn.Linear((num_classes+1+64)*2, num_classes) defined in __init__ return z1
Would you know how to properly add the extra data to the layer?
Best
Chris