I am doing a regression problem, and right now facing a problem where my model can only be trained with small amount of data, but when I increase the amount of data, the network does not learn(loss does not decreasing).

I don’t know how small the change of loss but we can typically observe that with a small model.
It means your model is not enough big to fit the larger dataset.

How small the loss value cannot exactly explain how good your model is.
If you use nn.L1Loss, check whether you are averaging the loss by the size of batch.