I have around 40 features and 4 target descriptors. I’ve trained models against each target individually, and I know there is a relationship. Even a simple linear regression does reasonably well.
I’m aware that each target descriptor has an effect on the features, such that there is function of all 4 targets that would have their cumulative effect and be more predictable than each one is individually.
It seems to me that I could discover this with a simple linear regression, but I’m not clear how I would train.
I can only imagine a training loop within a training loop. Such that the outer loop has the 4 target linear regression producing a single target, then the inner loop trains a model with the 40 features against that to see how well it can predict, with a loss produced from that prediction and used in the outer training loop.
I’m not sure I’ve explained that very well, and regardless, there may well be a much simpler solution.