calculate the shape of in_features using the shape formula for conv layers etc.
Yes, I suppose I am talking about a way to do that, not specific to nn.Linear.
The motivation for my question is I have seen several implementations of experimental network architectures with frustrating (anti)patterns such as
- magic numbers (e.g. a layer initialized with 1152 neurons as a result of unexplained calculations)
- user-settable parameters with unnecessary coupling (e.g. a command line parameter
--unit-size=1152which cannot be freely changed in reality, because it is determined by the input image size and other hyperparameters)
As a new Pytorch user, I am wondering if there is a “best practices” way to avoid such patterns in my own code, especially when writing custom subclasses of nn.Module.
BTW, I see my “helper method” idea is really the same as Carson McNeil’s idea here.