Nice that they included such a plot!
I looked at it more closely, everything seems to be alright except for one detail, the convolutional layers that are followed by batch normalization should have bias=False, explained briefly here.
1 Like