Loss function involving max of n inputs

I am trying to implement a loss function max(f_1(x),f_2(x),...).
I know that

max(a,b)=max(a-b,0)+b

so this can be implemented with O(log(n)) layers of ReLU. Is there a function in pytorch that already implements this?

I haven’t verified but it should look like this

def hardmax(ls):
    half = []
    for i in range(0,len(ls),2):
        if i+1 == len(ls):
            half.append(ls[i])
        else:
            relu = F.relu(ls[i]-ls[i+1])+ls[i+1]
            half.append(relu)
    if len(half) == 1:
        return half[0]
    else:
        return hardmax(half)

Hi,

if I understand correctly, you want to get the maximum of two (or more) values.

Let´s say something like this

ab = torch.randint(-5, 5, (5, 2))
# Output:
# tensor([[-3, -5],
#         [ 1, -4],
#         [-3, -2],
#         [ 1, -3],
#         [ 4, -4]])

Where the left column would be a and the right one would be b in your example.

You can then use the max function to get the maximum value.

val, ind = ab.max(dim=1)

print(val)
# Output:
# tensor([-3,  1, -2,  1,  4])

By defining the dimension to 1, it will look for the maximum value in each row. (You also get the indices for the column where the maximum value is located if you want/need them).

Please let me know if this is not the answer you were looking for.

Hope this helps :smile: