Dear experts
I wander if for instance considering a very simple Single nn.Linear(1,N) layer, Adam is optimising the weights/bias (2xN parameters) in a certain order? This means that the converging solution results in some way of a solution for the first couple (w1,b1) then (w2,b2) and so on.
best.