Yeah I’ve since changed tune about convergence. Agree. That’s what I got when batch size was the same, so parallelisation was NOT working properly. Overheads of parallelisation don’t pay off without increasing batch size. Right?
Yeah I’ve since changed tune about convergence. Agree. That’s what I got when batch size was the same, so parallelisation was NOT working properly. Overheads of parallelisation don’t pay off without increasing batch size. Right?