I ask some questions for semantic segmentation

I use segnet to run semantic segmentaion on voc 2012 .But the result is not good.So I ask help

I use loss function is nn.NLLLoss2d(F.log_softmax(outputs), targets)

Usualy ,the loss descend to 0.12, then it hardly down,so the results is not good.
I run 100~150 epochs.
The optimizer is Adam(model.parameters(),lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight_decay=0.000)
My nvidia memory too small,GTX1060,it is only 6G memory .so I set batch_size=5 ,and every image reszie(256,256).

And I change the segnet structure by myself, The loss down faster than segnet and unet on Voc2012, but at the end, the loss stop with the value 0.12

Is there any body give me some guides
I hope some friends help me