Forward path in nn.Linear

1

torch.manual_seed(0)
input = torch.tensor([[1.,1.,1.]])
print(torch.nn.Linear(3, 1).weight)
print(torch.nn.Linear(3, 1).bias)
print(list(torch.nn.Linear(3, 1).parameters()))
torch.nn.Linear(3, 1)(input)

I got the result:
Parameter containing:
tensor([[-0.0043, 0.3097, -0.4752]], requires_grad=True)
Parameter containing:
tensor([0.4578], requires_grad=True)
[Parameter containing:
tensor([[-0.0512, 0.1528, -0.1745]], requires_grad=True), Parameter containing:
tensor([-0.1135], requires_grad=True)]
tensor([[-1.1505]], grad_fn=)
I calculated the result by hand using weight and bias, and using parameters, but I get the two different numbers, not the one above -1.1505. Why is it?

2

Try use this:

torch.manual_seed(0)
input = torch.tensor([[1.,1.,1.]])
a = torch.nn.Linear(3, 1)
print(a.weight)
print(a.bias)
print(list(a.parameters()))
a(input)

Otherwise you will always create a new layer with different weights and biases.
Line print(list(a.parameters())) needs to contain the lines print(a.weight) and print(a.bias).

1 Like