Why is there a clamp(min=0) and a F.relu?

1

I know it might be a pedantic question but why do we need:

y_pred = x.mm(w1).clamp(min=0)

to exist when we can do:

y_pred = F.relu( x(w1) )

instead?

(note I am assuming x = torch.nn.Linear(D1,D2) and w1 is a Variable, in terms of a NN we might switch them round and have w1 be the linear and x the layer but I just kept it like that for the sake of the example)

2 Likes
2

ReLU is a very common activation function, so it makes sense to call it by its name. Also, it makes model definitions less painful to read.

clamp is much more versatile than F.relu.

1 Like