How many samples should I load per batch during training??
Is there any optimal number?
How many samples should I load per batch during training??
Is there any optimal number?
The number of samples per batch (aka “batch_size”) can have several effects on your training process:
Further reading:
Relation between batch size and learning rate: Don’t Decay the Learning Rate, Increase the Batch Size.
Visualizing the relation between batch size and learning rate can be found here.
For vaidation, you do not care about “smoothness” of gradients only about run time than you should tune your validation set’s batch_size
to be as large as you can to fit into GPU memory.