The gumbel_softmax may be feasible, but it needs input, and I just want to train a variable `delta = torch.zeros(100, requires_grad=True, device="cuda") `

, each element in delta is 0 or 1?

How to achieve this by gumbel_softmax or other methods?

The gumbel_softmax may be feasible, but it needs input, and I just want to train a variable `delta = torch.zeros(100, requires_grad=True, device="cuda") `

, each element in delta is 0 or 1?

How to achieve this by gumbel_softmax or other methods?

What do you mean? Do you wish your model to output just ones and zeros?

Yes, I want to get a [0 or 1] matrix W, and use W to choose some features.

Why can’t you use an activation function?

The activation functions (e.g. Sigmoid) do not get strictly 0 or 1, or these activation functions are not differentiable.

I need more context, like the type of problem you’re dealing with. Classification? Multi-class? Regression?. Also examples of actual target label.

Like I have a feature list: F=[f1,f2,…,fn], I want to find which is important, So, I need a [0,1] matrix W to select them. I want to select them not just get the important score, So, the W need to be 0 or 1 strictly.

Couldn’t you use floats for the training and then just use a threshold? So all the nums in F that are higher than for example 0.5 get assigned to 1/important.