After reading this: Calculating accuracy for a multi-label classification problem I managed to get the accuracy measure working by thresholding the network output. I can now see why it was not working before as my target was in binary format of 1 and 0 while the output from the nn.Sigmoid layer was a float value at every location. Thresholding now produces a more accurate IoU measure.
This now leaves the loss, with the above change my accuracy is improving as the network trains but the loss stays high. I am not passing the final convolution output from the network through a sigmoid layer and I am not thresholding it. The loss is given directly as from the model output:
output = model(input)
loss = loss_criterion(output, target)
thresholded_output = nn.Sigmoid(output) > 0.5
accuracy = accuracy_criterion(thresholded_output.to(dtype=torch.float), target)
Where target is the same binary tensor of 1 and 0 (dtype=Float32)
Should I be making any change before feeding the network output to loss criterion as well?
Thanks