If your inputs contains categorical variables, you might consider using e.g. an nn.Embedding layer, which would transform the sparse input into a dense output using a trainable matrix.
I’m unsure what the alternatives would be and if passing these values to the model might even work in your case.