Why is .clamp(min=0) used to find the max_val in the implementation of binary cross-entropy with logits loss, and why not use torch.max() function instead.
Also, why do they have used negative sign in front of (-input).clamp(min=0)
I have not clearly understood the first line of the code.
in PyTorch as defined here
max_val = (-input).clamp(min=0)
loss = input - input * target + max_val + ((-max_val).exp() + (-input - max_val).exp())
below I have shown how torch.max() should be used instead of .clamp()
input = torch.rand([2, 5])
max_val1 = (-input).clamp(min=0)
max_val2 = torch.max(input,1)
print("input:",input)
print("max_val1:", max_val1,"\nmax_val2:", max_val2)
input: tensor([[0.8552, 0.3244, 0.2635, 0.1885, 0.1494],
[0.2624, 0.8370, 0.9788, 0.8573, 0.8751]])
max_val1: tensor([[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.]])
max_val2: torch.return_types.max(
values=tensor([0.8552, 0.9788]),
indices=tensor([0, 2]))
here .clamp() doesn’t really finds the max value but instead makes all the negative value to positive and positive values to 0, that’s it.