Loss first decreased and then increased in the training set

I don’t know why. I tried to visualize the parameters and the gradients. I checked the distribution of some parameters, as shown in the figure below:

one layer parameters is as follows:


The graph of gradient is as follows:

But I don’t know where to start analyzing parameters and gradients。
I read some articles saying that it was because of the gradient explosion, but from the gradient and other information, I don’t know how to find out the reason。

And you can see that the parameters have changed, but my gradient image is still 0. What’s wrong with me?, In the visualization, I get the code of parameters and gradient information as follows:

Who can help me? Thanks