So there’s an add_module method that lets me add modules, but how do I remove/replace them? For the time being, del works, but feels really dirty.
self.relu = nn.ReLU()
del self.relu #DIRTY
self.relu = fun #crashes without the del statement...
You have funny ideas of cleanliness – just like we all do.
- There is nothing wrong with
del. This is the method Python and PyTorch forsee for removing something.
- Even just overwriting a module with another without explicit
del is OK.
- What does seem a lot less clean (to me, but I’ll admit that I fancy myself to have very good tastes in this and well aligned with PyTorch at that) is that you sometimes have
self.relu be a module and sometimes a function. This is not good, as modules are superspecial to PyTorch. I would recommend to either use functions throughout (e.g.
torch.nn.functional.relu) or stay with modules when changing.
Thanks for the fast response
- As long as del is appropriate (and doesn’t introduce bugs), I can live with that.
- Indeed, I can overwrite modules with other modules; I shot myself in the foot because I’m trying to replace a module with a function.
- That is indeed dirty, and I’ll think more about it.