I am puzzled about how I can implement a probabilistic deletion in a tensor from binary trainable layer in PyTorch. A single binary deletion looks like: 10110 -> 1010 where we don’t know if the 3rd or 4th bit is deleted.
To give some context, I am trying to solve the problem where the decoder receives the message 1010 and needs to recover the original message. This can mean a bit was deleted either in position 3 or 4. We cannot say for sure what position the deletion occurred. More on this here.
This is related to binary erasure channel where a single erasure looks like: 10110 -> 10e10. I was able to implement erasures using this.
bec_p_enc = 0.6
noise_shape = [100,7]
x = torch.from_numpy(np.random.choice([0.0, 1.0], noise_shape, p=[bec_p_enc, 1 - bec_p_enc])).type(torch.FloatTensor)
-
Is there a way to remove an element from a tensor, say based on how probable the outcome is going to be? I tried
random.sampleand other commands but they didn’t help. -
Can this layer be trainable using Straight-through estimation?