Differentiable Threhold for Conv2d output

Hi Ajinkya!

You want a differentiable, “soft” threshold function. I think that
nn.Tanhshrink gives you most of what you want. You can put in
a threshold parameter like this:

threshold * nn.functional.tanhshrink (x / threshold)

I’ve never done this, but I believe that if you make your
threshold value a nn.Parameter and include it in your model,
it should work.

The method in this post:

looks plausible to me (but I can’t vouch for it).

Best.

K. Frank