Pytorch equivalent of tensorflow keras StringLookup?

nlp
1

I’m working with pytorch and I’m really missing a layer from keras: tf.keras.layers.StringLookup that helped with processing of ids. Is there any workaround to do something similar with pytorch?

And example of the functionality im looking for:

vocab = ["a", "b", "c", "d"]
data = tf.constant([["a", "c", "d"], ["d", "a", "b"]])
layer = tf.keras.layers.StringLookup(vocabulary=vocab)
layer(data)

Outputs:
<tf.Tensor: shape=(2, 3), dtype=int64, numpy=
array([[1, 3, 4],
       [4, 1, 2]])>
2

I found a solution:

from torchnlp.encoders import LabelEncoder

data = ["a", "c", "d", "e", "d"]
encoder = LabelEncoder(data, reserved_labels=['unknown'], unknown_index=0)

enl = encoder.batch_encode(data)

print(enl)

tensor([1, 2, 3, 4, 3])