Pytorch gets me crazy with float or double

data
1

Hi is there a way to tell pytorch “autocast in whatever type is good for you, for me they all are numbers it’s python not C++”, because it’s coming to the point where it doesn’t make sense:

So first I force the type of my data to be in float, and it complains because it wants double:

RuntimeError: Caught RuntimeError in replica 0 on device 0.
Original Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/parallel/parallel_apply.py", line 61, in _worker
    output = module(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/thibault/pythia/dev/pythia/GNN/interface_designer0/Models/M_ID0.py", line 58, in forward
    x_i = self.conv1[str(i)](data.x.float(), data.edge_index, data.edge_attr)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch_geometric/nn/conv/transformer_conv.py", line 176, in forward
    out = self.propagate(edge_index, query=query, key=key, value=value,
  File "/usr/local/lib/python3.8/dist-packages/torch_geometric/nn/conv/message_passing.py", line 317, in propagate
    out = self.message(**msg_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch_geometric/nn/conv/transformer_conv.py", line 211, in message
    edge_attr = self.lin_edge(edge_attr).view(-1, self.heads,
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch_geometric/nn/dense/linear.py", line 118, in forward
    return F.linear(x, self.weight, self.bias)
RuntimeError: expected scalar type Double but found Float

So I force the type to be double and now it expects floats ?? WTH ?

    raise exception
RuntimeError: Caught RuntimeError in replica 0 on device 0.
Original Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/parallel/parallel_apply.py", line 61, in _worker
    output = module(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/thibault/pythia/dev/pythia/GNN/interface_designer0/Models/M_ID0.py", line 58, in forward
    x_i = self.conv1[str(i)](data.x.double(), data.edge_index, data.edge_attr)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch_geometric/nn/conv/transformer_conv.py", line 171, in forward
    query = self.lin_query(x[1]).view(-1, H, C)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch_geometric/nn/dense/linear.py", line 118, in forward
    return F.linear(x, self.weight, self.bias)
RuntimeError: expected scalar type Float but found Double
1 Like
2

Could you try enforcing double via torch.set_default_dtype(torch.float64) and see if you get the same error? Maybe torch_geometric enforces certain dtype on certain functions

3

Thank you it worked :slight_smile: , I know it’s not pretty but somehow it works and I just need something functional.

1 Like