Hello,
How would one go to initialize a Conv layer for a regression problem? Input is not an image. I would like to test out and see what kind of feature engineering a conv layer can perform. Say my input is 290 features and output is 1 value.
Sounds more like a pooling layer or linear to me.
The defining feature of conv layers is shift invariance, but so when you only have 1 output, it is hard to intuit about shift invariance.
That said, you could look into using the usual conv layer + pooling approach, just avoid doing softmax / sigmoid / … at the end if you want regression.
I would probably look at scaling weights such that activations so are in a sane range using trial and error.
Best regards
Thomas
1 Like
Hi Thomas, thank you for your feedback! I am working on 1 or 3 output regression problem. Of course, my data in scaled in preprocessing. My question is, how to deal with channels in Conv layers? Assuming my input shape is say (200, 300).
Well, so what nature are these 200 by 300 inputs? Are they independent or is there some notion of locality in either dimension such that a thing and the thing next to it have more relation than the thing and the thing 100 places further? Do you expect your prediction to depend on the relation between them?
The things with locality should probably be treated as kernel dimensions similar to width and height of an image. Things without connection could either be more of “linear layer nature” (i.e. you consider all of them at once) or consider them indivdually (so you would have a kernel size of 1 in that dimension).
If you have locality in both directions, you could just use unsqueeze on the input to make it a single input channel.
Best regards
Thomas
1 Like
is there some notion of locality in either dimension such that a thing and the thing next to it have more relation than the thing and the thing 100 places further?
I am not too sure.
Do you expect your prediction to depend on the relation between them?
Yes, I believe there is but this is my goal, to prove there is such a relationship.
Ah I see. I will try to get it working and see if anything comes up.
Thank you Thomas,
Michael
I should say, just trying is cool, too. With audio people have found some success just treating mel spectrograms as images even though it can (and has been) argued that it’s not necessarily a good way to think about them.
1 Like
I see what you mean. I don’t know if I can treat my data in that way but you definitely gave me some things to think about.