Loss won't decrease if batch_size > 1

[JuanFMontesinos (Juan Montesinos)]

First of all, keep in mind that the optimal LR depends on the batch size (in case u increased a lot the BS).

Secondly, you are probably having troubles with view method
check this

Difference between 2 reshaping operations (reshape vs permute)

if u have a cube c=torch.rand(3,4,5) and you use permute c=torch.rand(3,4,5) rx = c.permute(0,2,1) ry = c.permute(2,1,0) rz = c.permute(1,0,2) print(rx.size()) print(ry.size()) print(rz.size()) torch.Size([3, 5, 4]) torch.Size([5, 4, 3]) torch.Size([4, 3, 5]) you are just rotating the tensor, but order is preserved On the other hand, if you reshape you can see you are modifying the ordering because this is not rotating the cube but mapping in an ordered way from right to left. It takes n…

it may help u