Porting Considerations for Float to Int / One Hot

1

I am in the process of porting a few PyTorch CPU generative models to GPU, and converting the import dataset from MNIST float to one hot. I am wondering what issues I am going to run up against in terms of best practices for porting such as different loss functions for integer valued data etc.

Also, I noticed that PyTorch doesn’t seem to natively support one hot encoding and sparse inputs yet, so I think I might run into trouble with GPU memory (I have 6GB but this dataset is huge due to one hot encoding). From my research it looks like the scatter_() function or the embedding module is suggested for this.

Any thoughts on the best way to proceed?

Thank you in advance!

2

One hot in MNIST? Do you mean the target labels? Some standard loss functions for this task (e.g. cross entropy http://pytorch.org/docs/master/nn.html#crossentropyloss) has native support so you only need to pass in the class index rather than the one hot vector. If you really want one hot vectors, just make one on the fly when you train on it, e.g. (if you model is float)

target = torch.FloatTensor(1, 10).zero_()
target[idx] = 1