Training error not decreasing

Hello, everyone!

I am trying to train a Densenet model, modified for personal purposes. The input is an image and the output is an array. The problem is, that after the first few epochs I have a training and validation error almost constant.

I tried to train it with a single image and the error decreases, then I used only 10 images for training, and again for a smaller Densenet model I got a decreasing training error. Then I moved to a larger training set of 3500 images and I got the flat error.

My question is, how can I find out a connection between the training set size and the number of network parameters? I am inclined to think that this might be the problem, since for example when I used only 10 samples the network had to be reduced in size for the error to decrease. I am using L1Loss since I have to calculate the error between complex quantities.