Hyper parameter tuning tips

Hi

I am looking for beginner tips on hyper parameter tuning in Pytorch. Is there anything within the library that can do this? I have googled a bit and most refer to other packages. Also what would one target first if your application is semantic segmentation. I am getting decent results with Cross Entropy loss between 2-5% on my training set, though it oscillates a bit. I think I would need to tune stuff like learning rate and weights to get it better than this. Not sure where to start.

Thanks in advance.