Hi. I am training a neural network in the field of natural language processing.
To reduce runtime, I’m going to change the type of tensors from torch.long to torch.int32. Do you think that might be helpful?
Do you have another suggestion?
I wouldn’t think you will see a significant performance benefit assuming your code would work with int32. Usually integer tensors are used e.g. as the target values for nn.CrossEntropyLoss, for indexing etc. Since these targets or inputs are usually tiny compared to the model parameters as well as intermediate activations, the memory saving would also be in the same ballpark. To accelerate your model I would recommend to check mixed-precision training as well as the performance guide.
1 Like