Loss function and batch size

autograd
1

If I set the batch size as 1 then each iteration of dataloader will give me 1 row. However, if I add 4 more arbitrary rows in each iteration of dataloader then will the loss function (let’s say it is MSE) will be calculated as ((y-yhat)^2)/1 (since the batch size is 1) or will it be calculated as ((y-yhat)^2)/5 ?

2

Hi,

Loss function actually is not aware of batch size, It just responds to its input values which does not matter if you achieve them by using batch size or even directly reading data from storage then stacking them. So, in summary, batch size does not affect on value of loss functions directly.

Bests

1 Like
3

Thanks for the clarification