RuntimeError: Input type (torch.cuda.DoubleTensor) and weight type (torch.cuda.FloatTensor) should be the same

1

It looks like pytorch initializes weights of models by default with FloatTensors. However, I would like to have DoubleTensors. I initialize my weights with

def weight_init(self):
      for module in self.modules():
        if (isinstance(module, nn.Conv3d) or isinstance(module, nn.Linear)):
          init.orthogonal_(module.weight)

What is a proper way to initialize a model’s weights with DoubleTensors? Or is this generally not recommended?

Thanks for any help.

2

You could probably transform all parameters to float64 before calling wieght_init via model.double().