Lets say I have objective functions A, B and C. A is the original (GT supervised) objective function I want to minimize, while B and C are unsupervised alternatives (resulting in the same end-goal).
Now I want to find out if B or C are better in approximating A.
Therefore, I trained my model with A and calculated B and C along the way.
Is it correct to say that the objective function with the higher correlation coefficient to A will also give the best results?
You have to be careful with the following matters:
- Association does not mean necessarily a causal relation between both variables. For example, there might be a third variable you have not considered and this third variable might be the explanation for the behavior of the other two.
- Even if there is a causal relationship between the variables, the correlation coefficient does not tell you which variable is the cause and which is the effect.
- If the coefficient is close to 0, it does not necessarily mean that there is no relation between the two variables. It means there isn’t a LINEAR relationship, but there might be another type of functional relationship (for example, quadratic or exponential).
Thank you for these points!
- Just to be clear: Are you suggesting that A influences Z and Z influences B and C? In that case, does it really matter when I just want to quantify which objective function leads to a better result?
- As I am training with A and only observe B and C, the former must be the cause - or am I mistaken?
- I trained for the first epoch using A and the correlation coefficients are: c(A,B) ~ 0.3 and c(A,C) ~ 0.5. Therefore, I believe they have a reasonably linear relationship.
Can I do something else or do I need to do ablation studies (i.e. training the whole network with B and C and compare results w.r.t. GT) ?