How to add a learnable bias?

1

Is it something like:

bias = torch.Tensor(batch_size)
bias.needs_grad = True

x = x + bias

?

2

bias = Variable(torch.FloatTensor([…]),requires_grad=True)
…
x = x + bias
…
err = ___
err.backward()
bias.data = bias.data - learning_rate*bias.grad.data
bias.grad.data_zero()

1 Like