Hello!
This is sort of a general machine learning question that I am not able to solve. I want to predict a 1-dimensional target variable. To do so, I use a neural net. I feed 100 features that should help the model to predict the target variable. Along those features, I feed the target itself, so my input is 101-dimensional. I train my network but it does not learn to pick up the target. What is interesting is that if I constrain my input to just 11 dimensions (10 random + target), it works. I have not done more tests but I don’t understand why this happens.
Any idea? The model is very simple, just a neural net with some activations and linear layers, without dropout or any sort of batch/layer norm. I have tried varying the capacity of my model but it still behaves this way. I tried models from 100Mi parameters to a few thousand parameters.
Thanks!