[SOLVED] Invalid for input at training

Hi,

I try to replace softmax with binary-representation, repo is here;

Now it works for training, but in a way, it makes an error of

RuntimeError: shape '[64, 1, 324]' is invalid for input of size 10368

Why the input size was changed? although the input is “batch * image_size”.

Best,
S.Takano

I repaired with np.shape() to get batch size and use this for “for-loop” boundary.
But really why the batch size is changed?

Best,
S.Takano

The last batch might be smaller, if the number of samples in your Dataset is not divisible without a remainder by your batch size.
If you don’t want this behavior, you could specify drop_last=True in your DataLoader, or alternatively use the (maybe) smaller batch size for the last batch with x = x.view(x.size(0), your_size).

@ptrblck -san,

Thank you, I understand.
It means larger batch size can lost more training data, isn’t it?
So, there is optimal batch size for every model?

Best,
S.Takano

Only the very last batch can be smaller for any batch size, so if you use drop_last=True, you’ll lose (batch_size-1) samples max, which shouldn’t be a problem.
As I said, you could alternatively use the smaller batch and change your reshaping code, but it really depends on your preferences, if your model is not sensitive to the size of the batches (e.g. the batch norm layers might be updated with a noisy stats estimate for small batches).

@ptrblck -san,

Thank you very much for your advices!

Best,
S.Takano