I have a dataset consisting of gray-scale images of 16-bit pixel values. PyTorch requires the input to be a float before feeding into the model. I want to know which floating-point precision should I use to not lose any information from the images. So far, I have been using inputs.float() which casts the inputs to torch.float32. Further, does it have any effect on the training time using a higher precision float like torch.double().
You won’t lose any precision converting to float. The 32-bit floating point type can precisely represent all integers all 16-bit integers (actually up to all 24-bit integers too).
You should probably use float instead of double. Most PyTorch operations are faster on float32 and on GPUs float operations tend to be way faster than double.
You probably also want to normalize your inputs. If you’re using a pre-trained model, it’s important that you same normalization as was used to train the model. If you’re training from scratch, normalizing to [-1.0, 1.0] is reasonable or zero-mean unit standard deviation.
1 Like