When the GPU usage is low, what is the common approach?
I know that simply increasing the batch size can affect the learning outcomes.
Are there other methods I should consider?
Increasing the batch size or using a larger model will quickly fill up your GPU memory. E.g. if your current model trains fine you could try to increase the number of layers and/or parameters to check if the training would benefit from it.
Thank you for your response.
Won’t increasing the batch size affect learning outcomes? Is increasing batch size until GPU memory is full the right choice?
My current average GPU utilization is 25%, so I was hoping to reduce the 3+ hours it takes to train my model. If my model and data are not heavy, is it impossible to reduce the training time?
Is some sense probably, but it can be for the better. Larger batch sizes mean that your gradient for the update steps is an average over more training samples. This means typically, the larger the batch, the smaller the gradient. Hence, it’s very common that increasing the batch size allows you to increase the learning rate.
Thank you.
I’ll create a larger batch size and train from it.
I’m sorry, but this may be a basic question.
When increasing the batch size, would it be okay to set a very large size like 1024, 2048, 4096, 8192, etc. I know you’ve already answered this, but I’d like to know additionally if it’s valid for such large sizes.
If this is an issue, I would also like to know what size batch sizes you typically use to increase the batch size.