We generally create parameters or layers in the init() but when we don’t know the dimensions and those should be inferred from the inputs coming to forward(), we need to create parameters in forward() .
How should I go about this ?
I have tried creating parameters and F.linear() in forward() but I don’t see those parameters getting added to the model.parameters() .
I think my post is related and partially answers your question. The way to do this currently is to use the modulelist and a bunch of IF statements in the forward, that’s what I do now and it works, but the code could be made cleaner I think. I hope that future versions of pytorch address this better.