Is torch.clamp the same as torch.nn.functional.hardtanh?

The following codes give the same results. Does it mean torch.clamp is totally the same as torch.nn.functional.hardtanh both in training and test phase?

x1 = torch.randn(10)
x2 = x1.detach().clone()


y1 = torch.nn.functional.hardtanh(x1)
y2 = torch.clamp(x2, -1.0, 1.0)

l1 = y1.sum()
l2 = y2.sum()


assert (x1.grad - x2.grad).abs().sum() == .0
assert l1 == l2

The definition of torch.Hardtanh matches the way you clamped x2. For these choices of min,max, torch.clamp seems to be identical to torch.Hardtanh.

Hi, what do you mean by “clamped x2”. I thought if min and max are the same, it means the behavior of clamp and hardtanh are identical, am I right?

I meant this line of your code:

y2 = torch.clamp(x2, -1.0, 1.0)

Yes, this is what it looks like to me.

So the implementation of hardtanh and clamp should be completely the same (both the forward and the backward) ?

I don’t know. You could look at the source code for both, or wait for someone knowledgeable to answer.