libTorch C++ model not updating

vision
1

I have done a very straight forward transpiling of a PyTorch implementation I had into a C++ libTorch project. Everything seems to work fine. The forward modelling is running as expected but the model does not seemingly update any part of the model using “loss.backward()”. The offending code:

for (auto& batch : data_loader) {
auto data = batch.data.to(device), targets = batch.target.to(device);

    auto output = model.forward(data);

    auto loss = torch::binary_cross_entropy_with_logits(output, targets);

    optimizer.zero_grad();
    loss.backward();
    optimizer.step();
    cout << loss.item() << std::endl;
}

As I say everything seems to be working as is but the backward propagation isn’t either happening within the batch loop (or epoch loop). I have tried different learning rates using Adam optimisation.

Any help would be much appreciated.

Thanks

2

Try to check the gradients after the backward() call and make sure the gradients are calculated and check the magnitude of the values.
If you are (accidentally) detaching the computation graph, no gradients would be calculated. On the other hand, even if gradients are calculated they might be tiny and thus it might look as if no parameters are updated.

1 Like
3

Cheers I was “calling” into a module defined outside the main model structure. When a registered all my modules within the main model definition all worked fine.