What is the typical scale of model parameters?

[YuA]

You made it very clear for me. Thank you very much Frank.
I am now sure that the parameter scale of my model is not a problem.
However, I still cannot solve the problem that I am facing, apparently related to the numerical accuracy. I created a new topic:

Outputs are different depending on the input size

Thank you for reading my post. My question is, how can I mitigate the small numerical errors results from input size variety to accumulate through the layers? This question may not be obvious. So I will explain it below with a figure illustrating this problem. Backgroud: Because of the limit of numerical accuracy, the input size (shape) influences the outputs even though the expected outputs are mathmatically identical. This is explained in the doc Numerical accuracy — PyTorch 2.4 documentat…

I will appreciate it very much if you can give me more insights into this problem.
Thank you very much again for your help.