From the define in the pytorch document link, it seems the d_in should equal to dout when x equal to 3. but from the implementation, we can see d_in equal to dout * (1+0.5), is there any mismatch between the implementation and the defination?
 Hardswish — PyTorch 1.12 documentation
 add hardswish FP operator by vkuzo · Pull Request #34747 · pytorch/pytorch · GitHub
I assume you are concerned about the derivative at exactly
x = 3, which seems to use the derivative of the left part while the forward pass definition claims
f(x) = x for
x >= 3.
If so, then I think your concern is valid and you could create a GitHub issue so that the code owner could take a look at it. However, in practice you might not see any real difference at the derivative would only be different for exactly this value.
Hi, ptrblck, thanks for your reply, I have create an issue based on your suggestion, thanks for your help.