Adding extra data to standard convolution neural network in pytorch

Hello everyone. I am trying to extra data during training Resnet18. In my case it is gender information expressed as gender=Tensor([g]), where g =[-1|0|1] for [female|unknown|male] , and some shape descriptor obtained in previous steps, which is of the form desc=Tensor([...]), desc.shape = ([64,]) .
I tried to follow the example at this thread but my loss value becomes NaN in the first epoch. Changing the learning rate does not help

Here is the modified resnet18 function that i use. in the comments i put the shape and magnitudes of tensors. As you can see the data that i want to add is already normalized:

def forward(self, x,descdata,genderdata):
#x.shape = #torch.Size([4, 3, 224, 224])
# descdata.shape = torch.Size([ 4,64])
# genderdata.shape = torch.Size([1,4])

    x = self.conv1(x) #torch.Size([4, 64, 112, 112])
    x = self.bn1(x) #torch.Size([4, 64, 112, 112])
    x = self.relu(x) # torch.Size([4, 64, 112, 112])
    x = self.maxpool(x) #torch.Size([4, 64, 56, 56])

    x = self.layer1(x) #torch.Size([4, 64, 56, 56])
    x = self.layer2(x) #torch.Size([4, 128, 28, 28])
    x = self.layer3(x) #torch.Size([4, 256, 14, 14])
    x = self.layer4(x) #torch.Size([4, 512, 7, 7])

    x = self.avgpool(x) #torch.Size([4, 512, 1, 1])
    x = torch.flatten(x, 1) #torch.Size([4, 512])
    x = self.fc(x) #torch.Size([4, 35]); torch.norm(x,dim=1) == tensor([4.4604, 1.9574, 2.0123, 1.9583])


    y = genderdata.transpose(1,0) #torch.Size([4, 1]);  torch.norm(y,dim=1) == tensor([1., 1., 1., 1.])
   

    z1 = torch.cat([x,y,descdata.type(torch.float32)],dim=1) #torch.Size([4, 100]); torch.norm(z1,dim=1) == tensor([3.6243, 2.5945, 3.1520, 3.0312])
    z1 = self.relu2(self.fc2(z1))   # self.relu2 = nn.ReLU(); self.fc2  = nn.Linear(num_classes+1+64, (num_classes+1+64)*2)   defined in __init__ 
    z1 = self.fc3(z1)  #  self.fc3 = nn.Linear((num_classes+1+64)*2, num_classes)  defined in __init__ 
    return z1

Would you know how to properly add the extra data to the layer?

Best

Chris

If you just add one of the data parameters (descdata or gender) do you still see the issue? what loss function are you using? You might also want to check some of the other threads on nan loss to get some ideas: