Is there two NN are identical from a practical point of view?

1

Is there two NN are identical from a practical point of view?

Sequential(
  (0): Linear(in_features=9328, out_features=100, bias=True)
  (1): Sigmoid()
  (2): Linear(in_features=100, out_features=16, bias=True)
  (3): Softmax()
)
Sequential(
  (0): Sequential(
    (0): Linear(in_features=9328, out_features=100, bias=True)
    (1): Sigmoid()
    (2): Linear(in_features=100, out_features=16, bias=True)
  )
  (1): Softmax()
)
2

Yes, they are identical.
The only difference between them are of course the values of the parameters due to random initialization.
If you want to use the same parameters, you could try the following:

modelA = nn.Sequential(
    nn.Linear(9328, 100),
    nn.Sigmoid(),
    nn.Linear(100, 16),
    nn.Softmax(dim=1)
)

modelB = nn.Sequential(
    nn.Sequential(
        nn.Linear(9328, 100),
        nn.Sigmoid(),
        nn.Linear(100, 16),
    ),
    nn.Softmax(dim=1)
)

with torch.no_grad():
    modelB[0][0].weight = modelA[0].weight
    modelB[0][0].bias = modelA[0].bias
    modelB[0][2].weight = modelA[2].weight
    modelB[0][2].bias = modelA[2].bias
    
x = torch.randn(1, 9328)
outputA = modelA(x)
outputB = modelB(x)

print((outputA == outputB).all())
> tensor(1, dtype=torch.uint8)
1 Like