I am working on a physics-informed-neural-network (PINN).
In the forward PINN (with fixed material parameter p = p_ref, input: spatial coordinates, output: pde field), the total loss represents the minimal energy stored in the system and the norm of the pde-residual.
L_total = L_energy + L_residual
This converges, regardless of the weights and bias initialization, to a fixed loss value (minimal energy) and L_residual = 0.
In the inverse PINNs, the idea is to train also the material parameter p (as a torch.nn.Parameter()) and adding an additional data loss term:
L_total = L_energy + L_residual + L_data
Ideally, the forward loss is recovered which is the case when the material parameter is optimized towards the ground truth value (p = p_ref → L_data = 0). However, the flexibility of training both the field and the material parameter can create a different minimum where the total inverse loss is lower than the total forward loss. For instance, there is a combination of “field+param” resulting in a not perfectly zero value of L_data, but a way lower value of L_energy, overall resulting in a lower total loss. So identification fails.
If I did a standard gradient-based optimization (e.g. Levenberg-Marquardt), I would only optimize the material parameter and solve a pde at every iteration to determine the field which has the minimal energy and L_residual = 0. The parameters can be re-identified.
Any advice on how to balance physics and data losses better—e.g., dynamic weighting, scaling strategies, or regularization—would be much appreciated.