Custom DataSet Resize and padding

I have images in horizontal and vertical orientation. What is the ideal way to resize and pad them to a common size. I tried padding all the images to maximum height and width available but that doesn’t give good results .

What type of images are they? If you are working with face images, we have actually implemented a tool (EyepadAlign in mlxtend) for preprocessing face images to align them based on eyes, and nose. This module also allows you to align the position of face images in one dataset to match with another dataset, which is useful for cross-dataset evaluation. You can see examples in the docs: https://github.com/rasbt/mlxtend/blob/master/docs/sources/user_guide/image/eyepad_align.ipynb

no its object classification the images are of different sizes and orientation. So
one images is of 236x56, other is 150x40 and 45x200 etc etc

Then in that case, I think it’s best to select a fixed size, to resize/crop images to that fixed size. Meanwhile, due to the differences in image resolution, it is necessary to apply appropriate data-augmentation techniques, like random resize, random crop, …