Expected isFloatingType(grads[i].type().scalarType()) to be true, but got false

autograd
1

Hello, I am working on the implementation of an adversarial training. The following code does not work:

    for i, data in tqdm(enumerate(train_loader), total=len(train_loader), smoothing=0.9):

        pc1, pc2 = data
        pc1 = pc1.to(device).transpose(2, 1).contiguous().float()
        pc2 = pc2.to(device).transpose(2, 1).contiguous().float()

        # train the discriminator
        net_gen.eval()
        net_disc.train()

        i_odds = torch.arange(start=1, end=pc1.shape[2], step=2)
        ...

        # train the flow extractor
        net_gen.train()
        net_disc.eval()

        flow_pred = net_gen(pc1, pc2)
        pc_pred = pc1 + flow_pred

        z_pred = net_disc(pc1[:, :, i_odds], pc_pred[:, :, i_odds])
        loss_flow = - z_pred.mean()

        opt_gen.zero_grad()

        loss_flow.backward()

I get the following error on last line: Expected isFloatingType(grads[i].type().scalarType()) to be true, but got false.

What could that be? For all I could check, the model and all the tensors are all type cuda.Float.

That is the traceback:
Traceback (most recent call last):
  File "/home/zuanazzi/anaconda3/envs/tomtom2/lib/python3.6/pdb.py", line 1667, in main
    pdb._runscript(mainpyfile)
  File "/home/zuanazzi/anaconda3/envs/tomtom2/lib/python3.6/pdb.py", line 1548, in _runscript
    self.run(statement)
  File "/home/zuanazzi/anaconda3/envs/tomtom2/lib/python3.6/bdb.py", line 434, in run
    exec(cmd, globals, locals)
  File "<string>", line 1, in <module>
  File "/home/zuanazzi/PointCloudFlow/flownet3d_pytorch/self_train_adversarial.py", line 452, in <module>
    main()
  File "/home/zuanazzi/PointCloudFlow/flownet3d_pytorch/self_train_adversarial.py", line 445, in main
    train(args, net_flow, net_disc, train_loader, test_loader, boardio, textio)
  File "/home/zuanazzi/PointCloudFlow/flownet3d_pytorch/self_train_adversarial.py", line 309, in train
    train_stats = train_one_epoch(args, net_flow, net_disc, train_loader, opt_flow, opt_disc, writer, epoch)
  File "/home/zuanazzi/PointCloudFlow/flownet3d_pytorch/self_train_adversarial.py", line 227, in train_one_epoch
    loss_flow.backward()
  File "/home/zuanazzi/anaconda3/envs/tomtom2/lib/python3.6/site-packages/torch/tensor.py", line 195, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/zuanazzi/anaconda3/envs/tomtom2/lib/python3.6/site-packages/torch/autograd/__init__.py", line 99, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: Expected isFloatingType(grads[i].type().scalarType()) to be true, but got false.  (Could this error message be improved?  If so, please report an enhancement request to PyTorch.)
2

See discussion on stackoverflow.

1 Like