Manual loss accumulation

autograd
1

Hi there, I’m trying to accumulate the loss manually like this:

# -- training network ----------------------------------------

num_epochs = 5

loss_function = nn.MSELoss()

optimizer = optim.SGD(model.parameters(), lr = 0.01)

model.train()

for epoch in range(num_epochs):

    _epoch_loss = []

    for i in range(len(training_data['X'][0:20])):            # all cartpole simulations (each sim. is an episode).

        optimizer.zero_grad()

        accumulated_MSE = 0.0

        model[0].hebb = torch.zeros((30, 4), requires_grad = False)
        model[2].hebb = torch.zeros((30, 30), requires_grad = False)
        model[4].hebb = torch.zeros((1, 30), requires_grad = False)

        for j in range(len(training_data['X'][i])):            # all time steps of simulation i.

            _input = torch.tensor(np.array([training_data['X'][i][j]]), dtype = torch.float32)
            _target = torch.tensor(np.array([training_data['Y'][i][j]]), dtype = torch.float32)

            prediction = model(_input)

            prediction = torch.tensor(np.array(prediction.tolist()[0]), dtype = torch.float32)

            accumulated_MSE += loss_function(prediction, _target).item()

        accumulated_MSE_tensor = torch.tensor(accumulated_MSE, dtype=torch.float32, requires_grad=True)

        accumulated_MSE_tensor.backward()
        optimizer.step()

        model[0].hebb.detach_()
        model[2].hebb.detach_()
        model[4].hebb.detach_()

        # episode ends.

        _epoch_loss.append(accumulated_MSE)

    print(f'epoch {epoch} loss: {np.mean(_epoch_loss)}')

The problem is that the loss is not changing between epochs. What’s the correct way to accumulate the loss manually?

2

You’re breaking your graph here. Don’t re-wrap tensors. Also, you shouldn’t be using the requires_grad=True on new tensors, as that destroys the history of operations leading to no gradients

1 Like
3

If I simply do accumulated_MSE.backward() I get AttributeError: ‘float’ object has no attribute 'backward. How should I do it?

4

remove .item(), its casting a Tensor to float

5

This does not work. If I remove the .item() and don’t cast it to a new tensor like you suggested (code bellow) I get the error:

Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

for epoch in range(num_epochs):

    _epoch_loss = []

    for i in range(len(training_data['X'][0:20])):            # all cartpole simulations (each sim. is an episode).

        optimizer.zero_grad()

        accumulated_MSE = 0.0

        model[0].hebb = torch.zeros((30, 4), requires_grad = False)
        model[2].hebb = torch.zeros((30, 30), requires_grad = False)
        model[4].hebb = torch.zeros((1, 30), requires_grad = False)

        for j in range(len(training_data['X'][i])):            # all time steps of simulation i.

            _input = torch.tensor(np.array([training_data['X'][i][j]]), dtype = torch.float32)
            _target = torch.tensor(np.array([training_data['Y'][i][j]]), dtype = torch.float32)

            prediction = model(_input)

            prediction = torch.tensor(np.array(prediction.tolist()[0]), dtype = torch.float32)

            accumulated_MSE += loss_function(prediction, _target)

        accumulated_MSE.backward()
        optimizer.step()
6

Do you have a grad_fn for accumlated_MSE? Can you see if your model.parameters() have a grad_fn?

7

accumlated_MSE has no grad_fn, that’s why I was wrapping it around a tensor with require_grad = True. The model parameters do have a grad_fn.

8

re-wrapping here breaks the gradient flow you should pass the output from model to accumulated_MSE

1 Like
9

Thx a lot @AlphaBetaGamma96 ! That was indeed the problem. It’s solved now :slight_smile: