Why does torch.max return one value when dim is not specified?

1

So to explain it nicely,

a = torch.randn(1, 3)
a
tensor([[ 0.6763, 0.7445, -2.2369]])
torch.max(a)
tensor(0.7445)

^ returns the maximum value.

But here,

a = torch.randn(4, 4)
a
tensor([[-1.2360, -0.2942, -0.1222, 0.8475],
[ 1.1949, -1.1127, -2.2379, -0.6702],
[ 1.5717, -0.9207, 0.1297, -1.8768],
[-0.6172, 1.0036, -0.6060, -0.2432]])
torch.max(a, 1)
(tensor([ 0.8475, 1.1949, 1.5717, 1.0036]), tensor([ 3, 0, 0, 1]))

It returns the value along with the index location of each maximum value i.e the argmax.

Why isn’t the index location returned in the first case?

2

The LongTensor that specifies the argmax values is only returned when a dimension is specified.

a = torch.randn(1,3)
# tensor([[-0.6392, -0.8940,  0.3799]])
torch.max(a, dim=0) 
#  (tensor([-0.6392, -0.8940,  0.3799]), tensor([ 0,  0,  0]))
torch.max(a, dim=1) 
# (tensor([ 0.3799]), tensor([ 2]))

I think the returned LongTensor always has 1 fewer dimension than the input tensor. Without specifying which dimension in which to get the max values, torch won’t know which LongTensor to return. When the dimension isn’t specified, the singular maximal value in the entire tensor will be returned (as a single element tensor). Such as:

b = torch.randn(4,4)
torch.max(b)
# tensor(1.2384)
2 Likes
3

The reason is historical: this is the behavior of Lua Torch. We’ve been talking about changing these functions to have consistent behavior when the dimension is specified and when it’s not, but it’s difficult to migrate without breaking backwards compatibility.

2 Likes