Say I’m creating a model that has another model as attribute. Then I set w.requires_grad = False for w in model.submodel.parameters().
I’ll get an error if I try to train SGD(model.parameters()), because model.submodel.parameters() goes on that list.
Sure I can do W = [w for w in model.parameters if w.requires_grad], but shouldn’t there be a flag in optim to filter out requires_grad automatically, and perhaps that behavior or a warn to be set as default?