I run the code for 40 epochs. For 20 epochs, learning rate is constant and it linearly decrease for remaining epochs. During the linearly decrease mode, even if lr is in range of 10^-5, there is considerable change in validation image PSNR value around 1 during the last epochs. I think for small value of lr, PSNR should converge to near value of the previous epoch PSNR value.