Hi, all

From the define in the pytorch document link[1], it seems the d_in should equal to dout when x equal to 3. but from the implementation[2], we can see d_in equal to dout * (1+0.5), is there any mismatch between the implementation and the defination?

[1] Hardswish — PyTorch 1.12 documentation

[2] add hardswish FP operator by vkuzo · Pull Request #34747 · pytorch/pytorch · GitHub

I assume you are concerned about the derivative at exactly `x = 3`

, which seems to use the derivative of the left part while the forward pass definition claims `f(x) = x`

for `x >= 3`

.

If so, then I think your concern is valid and you could create a GitHub issue so that the code owner could take a look at it. However, in practice you might not see any real difference at the derivative would only be different for exactly this value.

Hi, ptrblck, thanks for your reply, I have create an issue based on your suggestion, thanks for your help.